Import via CSV with Tags?

Scoundrel
Scoundrel
Community Member
edited November 2019 in Lounge

I just signed up for a 1Password trial and so far every thing looks good except the import function. I have about 700 passwords to import from an unsupported source (Secret Server by Thycotic). My import CSV will need heavy editing and there are a lot of duplicate Titles because Secret Server had a folder structure and duplicates were not an issue.

I want to use tags to organize my passwords since folders are not available. But manually entering 700+ items is daunting, and manually editing them after the import to copy the folder name from a "custom field" to the Tags field is also problematic. I would like to create a CSV with a column that the import recognizes as a tag.

It seems like I'm missing something, because this seems like such a basic feature to not be supported. But I searched the help and the forum trying to figure out how to do it. The closest thing I found was some sort of converter utility, but when I looked at it, I was overwhelmed by it.

On the website, I can only import “Login” items, and it seems that the Tags field is not supported.

In the 1Password v7 program, I found the export to CSV function and exported some test passwords to CSV and saw that the file included a tags field. So then I attempted to import that file I had just exported as a test, and was astonished to find that the program does not have options for importing the very same file type it just exported. That makes no sense to me.

This is a large hurdle, and it may halt my forward progress unless someone can help me make this happen. Is this do-able?
Thank you for your time.


1Password Version: 7.3.712
Extension Version: Not Provided
OS Version: Windows 10 version 1803
Sync Type: Not Provided
Referrer: forum-search:import tags

«1

Comments

  • MrC
    MrC
    Volunteer Moderator

    @Scoundrel ,

    I’d be happy to help you with the converter to setup your custom tags. Let me know if you are interested.

  • AGAlumB
    AGAlumB
    1Password Alumni

    @Scoundrel: Regarding export, that's intended for moving data out of 1Password since otherwise it's taking encrypted data and dumping it into plaintext, which is not a great idea unless it's absolutely necessary and temporary. And importing exported 1Password data back into 1Password is not at all a common use case for that reason. That said, there are some things we'd like to improve in that area. Most other software doesn't structure data the same way 1Password does, so having it expect tag data will have strange consequences in many cases, but it's something we'll continue to evaluate as we develop future versions. Thanks for your feedback! :)

  • Scoundrel
    Scoundrel
    Community Member
    edited November 2019

    MrC -
    I realized that in my case, I could use vaults the way I used folders in the old software so I've side stepped this problem. Using vaults in this case is a good solution as it gives me more granularity for sharing and working with batches of passwords, and security boundaries.
    This still involved doing several dozen separate imports instead of one, but that was manageable, and did not involve tags.

    brenty -
    Are you sure export is for moving data out? Really? OK I know what the word export means and that's kind of insulting

    I just figured if a program has functions to export, and to import, it would at least support importing its own export format, so I looked there to try and find my solution.

    Exporting a few innocuous test passwords of various types with placeholder data in them to get examples of the format, and then using that to organize data to import during a migration does not seem to me like an uncommon use case.

    Sorry to hear that unexpected things happen when tags are included in an import.

  • AGAlumB
    AGAlumB
    1Password Alumni

    Are you sure export is for moving data out? Really? OK I know what the word export means and that's kind of insulting

    @Scoundrel: I didn't mean any offense. Please don't take any unnecessarily. It's just better to be clear when making a point, rather than taking for granted that everyone visiting this forum knows what I mean without me saying what I mean. It's an important consideration for a public support forum. I hope you'll understand.

    I just figured if a program has functions to export, and to import, it would at least support importing its own export format, so I looked there to try and find my solution. Exporting a few innocuous test passwords of various types with placeholder data in them to get examples of the format, and then using that to organize data to import during a migration does not seem to me like an uncommon use case. Sorry to hear that unexpected things happen when tags are included in an import.

    CSV is not 1Password's export format. It's entirely generic, which is good for interoperability, but bad when it comes to app-specific data. What I meant was that people exporting data from 1Password and then importing that same data back into 1Password is not a common use case. Usually people export to take the data somewhere else entirely, rather than to put it right back again. I don't mean that to be flippant or insulting, just to be clear on that fact, and to illustrate why it's not something we prioritize designing around. There are definitely ways in which export and import could be improved though (as MrC knows well), and I'm sorry for the inconvenience in your case. Thanks for sharing your experience.

  • Scoundrel
    Scoundrel
    Community Member

    Yes, it's good to leave nothing ambiguous. Forums are an imperfect form of communication.

    I'm sure that makes perfect sense from an insider's perspective. But from the perspective of a new customer with 700+ passwords trying to figure out how to efficiently get them in and organize them without excessive data entry, coming from a data source that is not among the list of expected/supported sources, it was a frustrating experience. But it's all good now, I got them all in there and organized by vault instead of folders or tags.

    Everything else about the software has been good so far with the exception of the trash refresh, but that's another topic.

  • Thanks for the update @Scoundrel. Obviously it is in our best interest to have the process of importing from a different solution to 1Password be as smooth and pain free as possible. There is definitely room for improvement here, and hopefully as we expand our development teams this is an area in which we can focus. MrC has done a great job for us with his converter tooling, completely voluntarily I might add, but we recognize that it isn't a frictionless situation.

    Ben

  • AGAlumB
    AGAlumB
    1Password Alumni

    @Scoundrel: I'm sorry for the rough start, and also wanted to apologize for the Trash (not) refresh(ing) issue in 1Password for Windows. We have a fix for that coming in what's become a rather big update to the Windows app and therefore needs our time and attention testing it. Thanks for your patience while we work on that!

  • CoreyCorey
    CoreyCorey
    Community Member
    edited January 2020

    There are many reasonable use cases for supporting tags during the import process. Please add (optional) support for it. 1Password could add another import option "Custom CSV with tag support" if needed.

    It's an important organizational feature that is missing from 1Password's import process.

    For me, I want to convert the export from another password manager that used folders, and I want to preserve that data and set it to tags. I could easily accomplish that a multitude of ways if 1Password supported it: I could write a script; use Sublime's bulk editing features; use Excel to import the CSV, move it around, and then export it as CSV again; or even a Find & Replace feature in another word processor to reformat as needed to preserve this data. I just need 1Password to provide a way to import tag data.

    Another obvious use case is people who store their passwords in a spreadsheet will often have a column for categorizing them and there's currently no way to get that data properly into 1Password.com in the tags area using the import process.

    Please add (optional) support for tags in CSV imports.

  • MrC
    MrC
    Volunteer Moderator

    @CoreyCorey

    What password manager?

  • CoreyCorey
    CoreyCorey
    Community Member

    @mrC, I'm in the process of consolidating from several different password managers, including encrypted plain-text formats. I'm standardizing everything into a single format for the import into 1Password, and that's why I'm focusing on the lack of tag support rather than specific password managers that aren't supported by 1Password.

    I realize a solution won't be available in the time I need it, so my specifics aren't as important as providing this general functionality for the next person who will need it.

  • MrC
    MrC
    Volunteer Moderator

    @CoreyCorey

    I get it.

    1Password's native CSV import has never handled a Tag colum (and from your other thread, doesn't label custom fields in a CSV per the column label - they are generically named).

    The csv converter in the converter suite can handle these two items - it will populate tags from a Tags column and will use your custom labels to create custom fields.

  • CoreyCorey
    CoreyCorey
    Community Member
    edited January 2020

    @MrC

    What password manager?
    The csv converter in the converter suite can handle these two items

    Have you considered adding ESET Password Manager?

    It's one of the most impressive, security-wise, password managers that I've ever come across, so it may not be as straightforward to add as the others. 1Password could probably benefit from studying them a bit.

  • MrC
    MrC
    Volunteer Moderator

    @CoreyCorey

    Have you considered adding ESET Password Manager?

    I don't recall ever reading a request for this. Typically I take a lazy approach, and develop a converter when there is a need, and a user who can test out the code I just developed.

  • charlieholder
    charlieholder
    Community Member

    I would also like to +1 support for a tags column in the CSV import. I would like to use tags and find myself about to import 200 passwords from Google Chrome. I find it easier to edit a CSV file instead of manually visiting each entry in 1P to add them.

    Sorry to feature creep, but every time I test import I get a dummy Login with all the CSV column headings. Would be nice to ignore that line.

  • ag_ana
    ag_ana
    1Password Alumni

    @charlieholder:

    Thank you for taking the time to share this feedback with us! :+1:

  • Shannakae
    Shannakae
    Community Member

    I have been trying to use the converter suite to bring in my Logins. I am using windows 10. I am using the basic CSV converter. I have followed all of the steps but after i import all of my logins are entered in as Secure Notes and my Tags are also imported. I don't write or really understand code, so i just followed the steps and did my best to figure out what to add for the final step in the converter process.... this is what i entered (my csv file is on my desktop)

    perl convert.pl -v csv ..\middinc.csv

    Can someone please help me figure out what i am doing wrong. Also here are my top headings on my spreadsheet (i have the specific ones in BOLD on my spreadsheet.
     Login Username Tags Title Login Password Notes Email Product Key Phone Login URL

  • Shannakae
    Shannakae
    Community Member

    It would be really amazing if 1 password would make the Tags importable with the regular import.

  • MrC
    MrC
    Volunteer Moderator

    Hello @Shannakae ,

    Can you tell me what program you used to create the CSV file, and what file encoding you saved it in?

    Also, is the CSV file using Comma or Tab as the separator character?

  • Shannakae
    Shannakae
    Community Member
    edited August 2020

    Hi I am using excel. I exported csv UTF-8(comma delimited)(*.csv)

  • MrC
    MrC
    Volunteer Moderator

    @Shannakae

    Ok, good. Let's see what's going wrong. Here's a sample file, shown as a screenshot:

    I made one change to your "Email" title, since that will cause auto-detection to an Identity (I can explain this later, and show you a workaround, but for now, I just changed that single title).

    Here's a screenshot showing the converter run, with debugging enabled so you can see what is happening:

    And here's the import:

    Can you run the debug mode -d to see what you are getting?

  • Shannakae
    Shannakae
    Community Member

    Hi, sorry i think this might be above my pay grade LOL. I don't really write or understand code. I can follow step by step instructions but beyond that i think i am lost. I can not understand what you are saying above. Sorry! : (

  • MrC
    MrC
    Volunteer Moderator
    edited August 2020

    @Shannakae

    I get it, I'm just trying to walk you through.

    Can you email me a sample CSV file with those same titles? Just copy your file, remove all but the column header row, and one sample row. Please obfuscate any private data. This will give me a file with two rows, and I can test it for you here.

    My email is at the top of the convert.pl script.

  • Shannakae
    Shannakae
    Community Member

    Sure Thing, thanks so much! Ok now i really feel dumb. When you say at the top of the convert.pl script where exactly is that?

  • MrC
    MrC
    Volunteer Moderator

    The convert.pl script is in the mrc-converter-suite folder you have. It will be the 4th line, in the copyright notice.

    @Shannakae

  • Shannakae
    Shannakae
    Community Member
    edited August 2020

    Ok still cant find it sorry! here is my email can you email me and i will reply? [ redacted ] Thanks!

    [ Edit: Email address removed for you - email sent ]

  • MrC
    MrC
    Volunteer Moderator
    edited August 2020

    @Shannakae ,

    Ok, I received your file and tested it. When I run it as is, your item gets detected as Identity items (as I mentioned above, due to the field named Email).

    But, when I change that field label to E-Mail, to avoid detection as Identity, I get a Login entry as expected:

    So, I don't see how you would get Secure Notes from this. Perhaps you can make that one label change to from Email to E-Mail and see what you get.

  • Shannakae
    Shannakae
    Community Member

    Ok i think i follow! I am headed out for a bit but when i get back i will give it a try! Thanks!

  • MrC
    MrC
    Volunteer Moderator
    edited August 2020

    @Shannakae

    OK, at your convenience. If you really want to use the title Email, I can send you an updated CSV converter file that will make accommodate you (I will change this anyway, for others, since the title Email is too common, and I don't want to auto-detect Identity for such a simple title).

    We can continue to work via email if that will be more convenient, and summarize here when we're done.

  • Shannakae
    Shannakae
    Community Member

    Yay it worked! However, i did have about 17 of my files end up as secure notes rather than login. I checked it out and the thing they all have in common is that they do not actually have a login or password (yet). I guess i can add a dummy login and password to trick the system, or do you have another suggestion? Thanks so much.

  • MrC
    MrC
    Volunteer Moderator
    edited August 2020

    @Shannakae ,

    Seems like good progress.

    So long as at least one of the column labels Login URL, Login Username, Login Password or Additional URLs exists in your header row in your CSV, all the data in the file will be treated as a Login:

    The Category is set when the header row is read, and should never change after that.

    You can see in my sample data above that empty cell values does not affect the Category detection.

This discussion has been closed.