MrC's Convert to 1Password Utility (mrc-converter-suite)

Options
1353638404152

Comments

  • peter_jm
    peter_jm
    Community Member
    Options

    That worked perfectly!! Thanks MrC.

  • peter_jm
    peter_jm
    Community Member
    Options

    Success - thanks so much!!

  • MrC
    MrC
    Volunteer Moderator
    Options

    Excellent. Thanks for the update, @peter_jm .

    Enjoy 1Password!

  • itou_masaaki
    itou_masaaki
    Community Member
    Options

    Hi @MrC ,

    Awesome & Fantastic tool! Thank you !! I have 600 logins created in 1password successfully with macOS 10.12.6 from iCloud Keychain.

    Somehow, every login has a note like "created: 2018-01-22 06:03:12 modified: 2018-01-22 06:03:12", but i didn't add any converter options before.

    Cloud you please help with this problem (or how to remove these notes quickly)? Thank you again for helping.

  • MrC
    MrC
    Volunteer Moderator
    Options

    Hello @itou_masaaki ,

    Thank you for the nice feedback.

    These date fields are present in the Keychain export, so the converter just adds them. You can use the --modified option to place the modified date in the record's metadata (vs. in the notes section). To remove the created date, I'll have to provide you with a slightly modified converter to avoid adding these values. You would have to re-run the converter and re-import the new records.

    Let me know if you want help on this.

  • itou_masaaki
    itou_masaaki
    Community Member
    Options

    Hi @MrC ,

    Yes, that would be great with your help! Could you please generate the modified converter for me ? Thank you so much!

  • MrC
    MrC
    Volunteer Moderator
    Options

    @itou_masaaki ,

    In Testing Bits, mentioned in the first post of the converter suite thread, there is a Keychain.pm file I just dropped for you. Download it and replace your Converters/Keychain.pm file. This version simply ignores the modified and created dates. Rerun your conversion to generate a new 1PIF.

  • itou_masaaki
    itou_masaaki
    Community Member
    Options

    Hi @MrC , That worked perfectly! Thanks a lot .

  • AGAlumB
    AGAlumB
    1Password Alumni
    Options

    Really glad to hear it all worked out. Cheers! :chuffed:

  • dpdannemiller
    dpdannemiller
    Community Member
    Options

    MrC - I was able to use your converter to transfer the contents of my wife's PasswordWallet to 1Password. Worked like a champ. Thanks.

  • MrC
    MrC
    Volunteer Moderator
    Options

    Excellent @dpdannemiller, thanks for the update!

    Enjoy 1Password.

  • matthew_frederick
    matthew_frederick
    Community Member
    Options

    I want to thank you for all of your hard work here. Amazing effort.

    Here's my issue: I want to export my Chrome passwords and import them into 1P. I've followed the Chrome flags export process, but unfortunately I'm not prompted to enter my macOS credentials, so I get an empty file (just column names).

    I then followed the sqlite export process described here: https://www.axllent.org/docs/view/export-chrome-passwords/

    It created the csv file, but again, I wasn't prompted for my credentials, so the password fields are all encrypted.

    On a whim I examined and then ran this python script: https://github.com/manwhoami/OSXChromeDecrypt. I reformatted the data so it was tab-delimited, then converted to CSV.

    It exported all of the passwords, but it's missing a lot of fields that your script expects (it seems). I started down the path of modifying the script — which I can do if necessary — but I've already burned 3 hours on the process so far, so I figured I'd first ask you about the lack of credential prompts.

    If you don't have a quick solution for that, alternatively will you please point me to the format (columns) your script is expecting so I can export the right fields in the right order with the right column names?

  • MrC
    MrC
    Volunteer Moderator
    Options

    @matthew_frederick ,

    Sorry to hear you've had to spend so much time on this - I always feel badly for folks that have so much of their lives consumed by these stupid decisions by software developers. Being one, I take some responsibility for this too.

    My mea culpa's out of the way, I haven't spent much time on Chrome password exports, so am not an expert in this. Recently Chrome even removed the ability to export the passwords (ugh).

    Can you tell me which fields the Python script exports? The csv converter should be able to handle them. First, please be sure you are using the 1.10 version of the converter suite. It is in Testing Bits, mentioned in the first post. The README.pdf explains which fields are required per category type.

  • matthew_frederick
    matthew_frederick
    Community Member
    Options

    It only exports the origin url, username, and password.

    However, I'm pretty sure I can get to all of these (Chrome's database field names), just need to add them to the script:

    origin_url
    action_url
    username_element
    username_value
    password_element
    password_value
    submit_element
    signon_realm
    preferred
    date_created
    blacklisted_by_user
    scheme
    password_type
    times_used
    form_data
    date_synced
    display_name
    icon_url
    federation_url
    skip_zero_click
    generation_upload_status
    possible_username_pairs
    

    Looking at Converters/Csv.pm, it looks like it wants these columns for logins:

    [ 'title',      0, qr/^title$/i, ],
    [ 'url',        1, qr/^login url$/i, ],
    [ 'username',       1, qr/^login username$/i, ],
    [ 'password',       1, qr/^login password$/i, ],
    [ 'notes',      0, qr/^notes$/i, ],
    [ 'tags',       0, qr/^tags$/i, ],
    

    Are they the ones? if so, should I use the "textname" or the "fields" name?

  • matthew_frederick
    matthew_frederick
    Community Member
    Options

    Oh, and I looked in the readme but couldn't find the fields the previous times, sorry. Looks like it's Title, Login Username, Login Password, Login URL, Notes. I'll try that, thanks.

  • MrC
    MrC
    Volunteer Moderator
    edited January 2018
    Options

    @matthew_frederick

    That's correct. The csv converter works from the column header names. There are some fixed names, as per the README.pdf. Other columns will be imported as custom fields using your header names.

  • matthew_frederick
    matthew_frederick
    Community Member
    Options

    Perfect. Turns out the script is exporting what I needed, just had to reorder and add the appropriate additional columns.

    (Then run some regexp in BBEdit to turn the site domain names into titles, then separate all of my work passwords from my personal passwords with some more-complex regexp, then delete all of the completely useless passwords, then transform to csv, then run the converter for each to pull them into separate vaults. But y'know, what else was I going to do today?)

    Thanks so much!

  • matthew_frederick
    matthew_frederick
    Community Member
    edited January 2018
    Options

    Oh, and FWIW, if other folks are having problems getting stuff out of Chrome, that script does work great.

    https://github.com/manwhoami/OSXChromeDecrypt

    To change the "pretty" output format used by the script to a plain csv that your script should be able to convert to a 1P_import.1pif file, change this chunk of the script:

                print("{}Passwords for Chrome "
                      "Profile{} -> [{}{}{}]".format(blue, end, violet,
                                                     profile.split("/")[-2], end))
    
                for i, entry in enumerate(db_data):
                    entry["pass"] = chrome_decrypt(entry["pass"], safe_storage_key)
    
                    print("  {}[{}]{} {}{}{}".format(green, i + 1, end, bold,
                                                     utfout(entry["url"]), end))
                    print("\t{}User{}: {}".format(green, end, utfout(
                        entry["user"])))
                    print("\t{}Pass{}: {}".format(green, end, utfout(
                        entry["pass"])))
    

    into:

                print("Title,Login Username,Login Password,Login URL,Notes")
    
                for i, entry in enumerate(db_data):
                    entry["pass"] = chrome_decrypt(entry["pass"], safe_storage_key)
    
                    print("\"\",\"\","  "," + utfout(entry["user"]) + "," + utfout(entry["pass"]) + utfout(entry["url"]) + ",\"\"")
    

    (Note that this change will only modify the way passwords/logins are exported... if you have other things stored in your browser like credit cards and such, they'll still format the "pretty" way.)

    I'm absolutely not endorsing this script... I only know enough Python to google the syntax needed to make this change, and there are likely problems with it, especially if your username or password include a double-quote character ("). But hey, if nothing else is working, this seems to do the trick.

    (Edited to fix the replacement code.)

  • MrC
    MrC
    Volunteer Moderator
    Options

    Thanks @matthew_frederick ,

    Maybe I will take a look at that Python script and just create a chrome converter based on it.

  • matthew_frederick
    matthew_frederick
    Community Member
    Options

    That would be brilliant!

  • DariusR
    DariusR
    Community Member
    Options

    @MrC
    Thank you so much for this! I can only begin to imagine how much time and energy you've saved me with this!

    I couldn't get 'Stable bits' to work but 'Testing bits' let me successfully import around 400 entries, including files, last modified dates, and custom fields from Keepass2.

    Sorting through this in a Standalone vault before moving them to my 1Password Teams account is going to so much quicker and more practical.

    This seriously made my week :)

  • MrC
    MrC
    Volunteer Moderator
    Options

    @DariusR ,

    Stable Bits works with Perl version 5.24 and previous. Version 5.26 broke old, long standing behavior. I really need to promote Testing Bits to Stable Bits.

    Testing Bits also had some Keepass2 changes, which you found beneficial.

    I'm really happy to hear this saved you some time. Thanks for the feedback.

    Enjoy 1Password!

  • tajchiu
    tajchiu
    Community Member
    Options

    Hi @MrC Thank you so much for creating a converter to allow folks who mistakenly chose a password manager other than 1Password to convert over to the light side!

    I am trying to convert a RoboForm csv file into 1Password using your utility.

    I exported the CSV directly from RoboForm version 8.4.6 on my Mac.

    When I try to convert it with version 1.09 of your utility I receive the following error:

    Unexpected Roboform print list format; please report your platform and version of RoboForm

    Did I export the wrong file from RoboForm?

    Thank you,
    Taj

  • sjk
    sjk
    1Password Alumni
    Options

    Hi @tajchiu,

    Hi @MrC Thank you so much for creating a converter to allow folks who mistakenly chose a password manager other than 1Password to convert over to the light side!

    Surely MrC appreciates your kind words, as I do. :)

    I am trying to convert a RoboForm csv file into 1Password using your utility.

    I exported the CSV directly from RoboForm version 8.4.6 on my Mac.

    You won't need to use MrC's conversion utility for data exported to CSV format from RoboForm 8. After exporting, simply remove this first header line:

    Name,Url,Login,Pwd,Note,Folder,Rf_fields

    Then you'll be able to import it with 1Password for Mac using these instructions:

    Import your CSV data into 1Password

    The Import screen will look similar to this before selecting the CSV file in step 3:

    Make sure the Login item type is chosen in step 1 and the correct vault in step 2.

    Alternatively, you can import your data on 1Password.com if you have an account:

    Move your data from RoboForm to 1Password

    When using that method the header line in the exported CSV file doesn't need to be removed before importing.

    Did I export the wrong file from RoboForm?

    Nope! As far as I know it's only possible to export passwords from RoboForm 8 (to HTML or CSV format).

    Let us know if you have any questions or trouble with this. Cheers!

  • MrC
    MrC
    Volunteer Moderator
    edited February 2018
    Options

    @tajchiu / @sjk ,

    Roboform is tragically flawed with respect to exporting data.

    1. It will only export certain categories.
    2. It has three forms of "export" - 1) the newer Export -> CSV export, which will only export Logins and Bookmarks, 2) the Export -> HTML export which will only export the URLs for Logins and Bookmarks in a simple fashion, and 3) Print List which will allow exporting via HTML Logins/Bookmarks, Identities, and Safe Notes.

    The converter uses the Print List form of export (3), since it works for Roboform versions back to version 6. This allows you to also get your Identities and Safe Notes, something the CSV export does not support.

    You can get better import control (with respect to naming your fields), if you use the csv converter from the converter suite. Use the 1.10 version in Testing Bits, not the 1.09 version you are using. But 1Password's CSV import will work too.

    I don't know why these guys won't get their act together, so we're all left with the safe assumption they really don't care, nor want you to leave (without some pain).

  • tajchiu
    tajchiu
    Community Member
    Options

    Yeah! That worked!

    It would be awesome if the 1Password importer gave me an error indicating that I need to remove the header from the file!

    Taj

  • MrC
    MrC
    Volunteer Moderator
    Options

    It would be even more awesome-er if 1Password just removed the header for you!

  • tajchiu
    tajchiu
    Community Member
    Options

    That's fair - I was sort of thinking the same thing!

  • bpiec
    bpiec
    Community Member
    Options

    Thank you for you awesome tool!

    I noticed one thing though. I am trying to import from CSV with the first column named "Title". When I saved my file as UTF-8 with BOM it didn't match the title column and thus all my records were marked as "Untitled". So the problem is that it tries to compare "Title" to "Title" and fails while it should just handle BOM correctly.
    After removing BOM it went well.

  • MrC
    MrC
    Volunteer Moderator
    Options

    @bpiec ,

    You're very welcome.
    Thanks for bringing this to my attention.

    Typically, UTF-8 does not contain a BOM. But, I'll add support for it into the csv converter, as I already have routines for file-reading, encoding, and BOM removal.

This discussion has been closed.