How to migrate to 1Password from Enpass 5?

pedrogarcia
pedrogarcia
Community Member
edited June 2019 in 1Password 7 for Windows

Hello,
I have read the forums, and I was unable properly convert the enpass .csv to 1password, with all the correct categories. This is the error message I get:

perl convert.pl -v enpass ..\pm_export.json
couldn't find diagnostic data in /usr/share/perl5/core_perl/pods/perldiag.pod /c/Users/phg/OneDrive/Agilebits/mrc-converter-suite/. /c/Users/phg/OneDrive/Agilebits/mrc-converter-suite/lib /usr/lib/perl5/site_perl /usr/share/perl5/site_perl /usr/lib/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib/perl5/core_perl /usr/share/perl5/core_perl convert.pl at /usr/share/perl5/core_perl/diagnostics.pm line 259, <POD_DIAG> line 711.
Compilation failed in require at /c/Users/phg/OneDrive/Agilebits/mrc-converter-suite/./Utils/PIF.pm line 14.
BEGIN failed--compilation aborted at /c/Users/phg/OneDrive/Agilebits/mrc-converter-suite/./Utils/PIF.pm line 14.
Compilation failed in require at convert.pl line 18.
BEGIN failed--compilation aborted at convert.pl line 18.

1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Windows 10
Sync Type: Not Provided

Comments

  • MrC
    MrC
    Volunteer Moderator
    edited June 2019

    Hi @pedrogarcia ,

    Enpass 5 does not have JSON output (that was first available in Enpass 6). The JSON export over version 6 is better than the CSV of version 5 (which is ambiguous).

    Also, it seems like your Perl installation is broken. What exact version and variant did you install? You may not be able to install it on OneDrive - try installing it to a local disk instead. You'll need to reinstall the modules also.

  • pedrogarcia
    pedrogarcia
    Community Member

    I installed perl from scoop.

    I now get the following error:

    C:\Users\phg\OneDrive\Agilebits\mrc-converter-suite>perl convert.pl -v enpass ..\pm_export.txt
    Can't locate Date/Calc.pm in @INC (you may need to install the Date::Calc module) (@INC contains: C:/Users/phg/OneDrive/Agilebits/mrc-converter-suite/. C:/Users/phg/OneDrive/Agilebits/mrc-converter-suite/lib C:/Users/phg/scoop/apps/perl/current/perl/site/lib C:/Users/phg/scoop/apps/perl/current/perl/vendor/lib C:/Users/phg/scoop/apps/perl/current/perl/lib) at C:/Users/phg/OneDrive/Agilebits/mrc-converter-suite/./Utils/PIF.pm line 18.
    BEGIN failed--compilation aborted at C:/Users/phg/OneDrive/Agilebits/mrc-converter-suite/./Utils/PIF.pm line 18.
    Compilation failed in require at convert.pl line 18.
    BEGIN failed--compilation aborted at convert.pl line 18 (#1)
        (F) You said to do (or require, or use) a file that couldn't be found.
        Perl looks for the file in all the locations mentioned in @INC, unless
        the file name included the full path to the file.  Perhaps you need
        to set the PERL5LIB or PERL5OPT environment variable to say where the
        extra library is, or maybe the script needs to add the library name
        to @INC.  Or maybe you just misspelled the name of the file.  See
        "require" in perlfunc and lib.
    
    Uncaught exception from user code:
            Can't locate Date/Calc.pm in @INC (you may need to install the Date::Calc module) (@INC contains: C:/Users/phg/OneDrive/Agilebits/mrc-converter-suite/. C:/Users/phg/OneDrive/Agilebits/mrc-converter-suite/lib C:/Users/phg/scoop/apps/perl/current/perl/site/lib C:/Users/phg/scoop/apps/perl/current/perl/vendor/lib C:/Users/phg/scoop/apps/perl/current/perl/lib) at C:/Users/phg/OneDrive/Agilebits/mrc-converter-suite/./Utils/PIF.pm line 18.
            BEGIN failed--compilation aborted at C:/Users/phg/OneDrive/Agilebits/mrc-converter-suite/./Utils/PIF.pm line 18.
            Compilation failed in require at convert.pl line 18.
            BEGIN failed--compilation aborted at convert.pl line 18.
    
  • MrC
    MrC
    Volunteer Moderator
    edited June 2019

    @pedrogarcia

    I don't know who "scoop" is. Use the directions in the README.html, part of the converter suite package. The README.html will guide you.

    Re-run the install_modules.bat file, as per the README. This is required.

  • pedrogarcia
    pedrogarcia
    Community Member

    Ok, new error log:

    Looking for required perl modules...
    ---- App::cpanminus
    ----    Good
    
    XML::XPath ----
        Already installed
    
    Bit::Vector ----
        Needs installing
    ! Installing Bit::Vector failed. See C:\Users\phg\.cpanm\work\1560886465.14276\build.log for details. Retry with --force to force install it.
    +++ Module installation failed
    
    Date::Calc ----
        Needs installing
    ! Installing Bit::Vector failed. See C:\Users\phg\.cpanm\work\1560886470.13884\build.log for details. Retry with --force to force install it.
    ! Installing the dependencies failed: Module 'Bit::Vector' is not installed
    ! Bailing out the installation for Date-Calc-6.4.
    +++ Module installation failed
    
    Win32::Unicode::File ----
        Needs force installing
    ! Installing Win32::Unicode::File failed. See C:\Users\phg\.cpanm\work\1560886475.8104\build.log for details. Retry with --force to force install it.
    +++ Module installation failed
    
    Done installing required modules.
    
  • MrC
    MrC
    Volunteer Moderator

    @pedrogarcia

    Sorry for the delay.

    I don't know what state of Perl installation you now have.

    If you used the script provided by "scoop", then it installed the portable version of Strawberry Perl. I no longer support that version in the converter suite. It may be just fine, but I found users would get themselves into trouble using this method. You may be in that situation. Again, I don't know "scoop", nor understand what led you to that script.

    You could try forcing the installation of the modules that failed:

    cpanm --force  Bit::Vector  Date::Calc  Win32::Unicode::File
    

    but I'm worried about this, because the install_modules.bat failed when trying to force the install of the Win32::Unicode::File module.

    What I would recommend instead - remove the portable version of Strawberry Perl, and follow the README.html instructions. I just tested that it works:

    and you can see on a Win 10 x64 system all the modules installed as expected.

  • pedrogarcia
    pedrogarcia
    Community Member

    I will install Perl using the readme method and report the results.

  • pedrogarcia
    pedrogarcia
    Community Member
    edited June 2019

    @MrC Scoop is like Homebrew, but for Windows. I uninstalled the perl version from scoop and installed the recommended version.

    Installation still fails with the strawberry perl from the README.
    This is the build.log:

    cpanm (App::cpanminus) 1.7044 on perl 5.030000 built for MSWin32-x64-multi-thread
    Work directory is C:\Users\phg\/.cpanm/work/1560911388.6072
    You have make C:\Strawberry\c\bin\gmake.exe
    You have LWP 6.39
    You have C:\WINDOWS\system32\tar.exe, C:\msys64\usr\bin\gzip.exe and C:\msys64\usr\bin\bzip2.exe
    Searching Win32::Unicode::File () on cpanmetadb ...
    --> Working on Win32::Unicode::File
    Fetching http://www.cpan.org/authors/id/X/XA/XAICRON/Win32-Unicode-0.38.tar.gz
    -> OK
    Unpacking Win32-Unicode-0.38.tar.gz
    Entering Win32-Unicode-0.38
    Checking configure dependencies from META.yml
    Checking if you have ExtUtils::MakeMaker 6.59 ... Yes (7.36)
    Checking if you have ExtUtils::ParseXS 2.21 ... Yes (3.40)
    Checking if you have Devel::PPPort 3.19 ... Yes (3.52)
    Configuring Win32-Unicode-0.38
    Running Makefile.PL
    Writing ppport.h
    Checking if your kit is complete...
    Looks good
    Generating a gmake-style Makefile
    Writing Makefile for Win32::Unicode
    Writing MYMETA.yml and MYMETA.json
    -> OK
    Checking dependencies from MYMETA.json ...
    Checking if you have Devel::PPPort 3.19 ... Yes (3.52)
    Checking if you have ExtUtils::MakeMaker 0 ... Yes (7.36)
    Checking if you have XSLoader 0.02 ... Yes (0.30)
    Checking if you have Test::Mock::Guard 0.08 ... Yes (0.10)
    Checking if you have Test::Exception 0.31 ... Yes (0.43)
    Checking if you have ExtUtils::ParseXS 2.21 ... Yes (3.40)
    Checking if you have Test::Output 1.01 ... Yes (1.031)
    Checking if you have Test::More 0.98 ... Yes (1.302164)
    Building and testing Win32-Unicode-0.38
    cp lib/Win32/Unicode/Process.pm blib\lib\Win32\Unicode\Process.pm
    cp lib/Win32/Unicode/Console.pm blib\lib\Win32\Unicode\Console.pm
    cp lib/Win32/Unicode/Error.pm blib\lib\Win32\Unicode\Error.pm
    cp lib/Win32/Unicode/Dir.pm blib\lib\Win32\Unicode\Dir.pm
    cp lib/Win32/Unicode.pm blib\lib\Win32\Unicode.pm
    cp lib/Win32/Unicode/XS.pm blib\lib\Win32\Unicode\XS.pm
    cp lib/Win32/Unicode/Native.pm blib\lib\Win32\Unicode\Native.pm
    cp lib/Win32/Unicode/Util.pm blib\lib\Win32\Unicode\Util.pm
    cp lib/Win32/Unicode/File.pm blib\lib\Win32\Unicode\File.pm
    cp lib/Win32/Unicode/Constant.pm blib\lib\Win32\Unicode\Constant.pm
    Running Mkbootstrap for Unicode ()
    "C:\Strawberry\perl\bin\perl.exe" "-Iinc" -MExtUtils::Command -e chmod -- 644 "Unicode.bs"
    "C:\Strawberry\perl\bin\perl.exe" "-Iinc" -MExtUtils::Command::MM -e cp_nonempty -- Unicode.bs blib\arch\auto\Win32\Unicode\Unicode.bs 644
    "C:\Strawberry\perl\bin\perl.exe" "-Iinc" "C:\Strawberry\perl\lib\ExtUtils/xsubpp"  -typemap C:\STRAWB~1\perl\lib\ExtUtils\typemap  xs/Console.xs > xs/Console.xsc
    "C:\Strawberry\perl\bin\perl.exe" "-Iinc" -MExtUtils::Command -e mv -- xs/Console.xsc xs/Console.c
    gcc -c  "-I." "-I." -s -O2 -DWIN32 -DWIN64 -DCONSERVATIVE -D__USE_MINGW_ANSI_STDIO -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -fwrapv -fno-strict-aliasing -mms-bitfields -Wall -Wextra -Wdeclaration-after-statement -Wc++-compat -s -O2   -DVERSION=\"0.38\" -DXS_VERSION=\"0.38\" -o xs/Console.o  "-IC:\STRAWB~1\perl\lib\CORE"  -DUSE_PPPORT xs/Console.c
    gcc: fatal error: no input files
    compilation terminated.
    gmake: *** [Makefile:376: xs/Console.o] Error 1
    -> FAIL Installing Win32::Unicode::File failed. See C:\Users\phg\.cpanm\work\1560911388.6072\build.log for details. Retry with --force to force install it.
    
    
  • pedrogarcia
    pedrogarcia
    Community Member
    edited June 2019

    @MrC I managed to get it working by temporarily removing my gcc install from my path, because perl does not work very well with it.

    But the converter failed to properly import my bank accounts (the "Finance" category from enpass,), all wifi stations (the computer category), and documents. It imported all this stuff a text note instead.

    It also fails to convert any and all custom categories from all entries, and it fails to detect the category "E-mail" from any entry.

    This difficulty and unwillingness to properly import data from competitors seem to be exclusive to 1Password.

    For example, Bitwarden did not have any problem to properly import my data, it detected custom categories and fields and did not corrupt a single byte from the enpass exported data.

  • MrC
    MrC
    Volunteer Moderator

    @pedrogarcia

    Your build is behaving differently than mine, likely due to the additional development environment you have installed on it:

    Compare two key lines from our builds, mine which is successful, yours not (because the source file isn't being detected on the command line):

  • pedrogarcia
    pedrogarcia
    Community Member

    @MrC I managed to get it built by commenting the gcc environment from my path. It runs, but fails to convert some of the data I mentioned above in my comment. What should I do now? Is there any way to properly import my Enpass data to 1Password?

  • MrC
    MrC
    Volunteer Moderator
    edited June 2019

    @pedrogarcia ,

    Exactly correct re: gcc, and your development environment.

    Ok, now we can get onto the converter.

    The converter isn't "failing to detect" - it can only know about Enpass' stock categories, and the CSV contains only field labels and values. So the converter uses these field names to try to detect the category:

    As mentioned on the converter suite post and README.html, I do not work for Agilebits/1Password, so do not speak for them. But the issue is not a 1Password problem, nor are any conversion / importation issues unique to them. The data exported by Enpass has no semantics. Here's a "bank account" record:

    The fields can contain total nonsense values, the labels are sometimes duplicated (especially in non-English languages), field / value pairs are not in any strict order, nor are they always present.

    It is trivial to create Enpass data that will trip up Bitwarden, or any other conversion. It is also trivial to create an Enpass export which contains broken CSV - I've demonstrated this several times. It is trivial to create Enpass records for which it is impossible to know which fields belong to which part of an Enpass record.

    But I'm not going to argue these points anymore - if you want help customizing the converter to work with your customized data, to work best with 1Password, I'm obviously happy to help.

  • pedrogarcia
    pedrogarcia
    Community Member

    @MrC What do I need to do to customize the converter?

  • MrC
    MrC
    Volunteer Moderator
    edited June 2019

    @pedrogarcia

    The only thing the CSV importer of the Enpass converter can do is match field labels (the JSON importer can do more). At the top of the converter is a table of categories and their field labels as regular expressions. Above, I showed an example of a "bankacct" type. It has labels such as "Bank name". If this label is matched, and that category only needs 1 matching field name (see the 0's and 1's in the record), then bankacct will be selected as the target category. A 0 indicates the label is insufficient to match the category, a 1 means only 1 label is required. Some categories need 2 or more matches, because the fields are ambiguously labeled across all categories.

    This converter requires English labels, since the non-English labels are poorly translated, and label matching becomes nearly impossible. So examine your CSV data for those field labels (like the example above).

    Each category in the %card_field_specs table can be customized, and the RE labels can be altered. These lines also define the 1Password internal key for a field (these are well-defined). So if you've customized some label, it just needs to be changed in that part of the table to force a match.

    So if you had no match for a bank account, that's suggesting to me your export may not be in English as per the README.html (see the Enpass converter's Notes section), or you've highly customized your data (since the bankacct type is a very relaxed category - many stock labels will cause a match).

    You can enable --debug mode, and see how this CSV matching is done:

  • pedrogarcia
    pedrogarcia
    Community Member

    I can provide the .json files.
    Assuming I have the .json fron enpass, what else do I need to fix?

  • pedrogarcia
    pedrogarcia
    Community Member

    @MrC the json importer is a bit better, but it still misses my bank accounts and any custom field I created.
    The csv importer get the bank account category correctly, but puts all information and custom fields into a "secure note".
    The csv importer also gets all logins for all websites incorrectly identified as "Identities", with all the information such as email and password as a secure notes.
    But how do I fix them so that the 1Password converter creates the custom categories correctly?

  • MrC
    MrC
    Volunteer Moderator
    edited June 2019

    Sorry again for the delay - I was done working for the night.

    I recall now that the previous users of this newer conversion mode (JSON) didn't have any / many of the more esoteric record types, so they were satisfied. Since there are many categories in Enpass, I work on an as needed basis; thus, not all of the category definitions were defined. So let's define what you need.

    Run the converter using the --dumpcats option.

    This will tell us the Enpass categories you have, and the fields present for each along with their section, order, and count.

    Let's work to chip away at these, hitting the most important categories for you. Let's do one or two at a time. You give me the dumpcats output for a given category, and I'll update the table accordingly.

    The JSON importer portion of this Enpass converter works nearly the same way as the CSV converter, but it has the additional category name (this is unavailable to the CSV importer).

    I just added 'bankacct' :

    You can download this updated Enpass.pm file in the Custom folder (at the Dropbox link provided in the converter suite thread). Replace your Converters\Enpass.pm file with it.

    To get extra fields that do not or cannot be mapped into 1Password fields automatically added as custom fields in a custom section, use --addfields option. These can instead be alternatively mapped in the table, but this is a little more complex.

    You can also tell the converter which imported categories to deal with via the --imptypes option. You'll see my use of that above to process only Enpass' "bankacct" exports.

  • MrC
    MrC
    Volunteer Moderator
    edited June 2019

    The csv importer also gets all logins for all websites incorrectly identified as "Identities", with all the information such as email and password as a secure notes.

    Let's see what --dumpcats provides for these.

    Feel free to email these directly to me, and/or we can continue correspondence via email. My email is at the top of the convert.pl file.

  • pedrogarcia
    pedrogarcia
    Community Member

    @MrC Thanks. I have emailed you.

  • MrC
    MrC
    Volunteer Moderator

    @pedrogarcia ,

    Replied. Let's handle this offline and we can summarize here when we are done.

  • MrC
    MrC
    Volunteer Moderator

    I think we may have closed in this matter.

    @pedrogarcia persuaded me to re-evaluate conversion of Enpass’ TOTP field for logins, and so now the converter supports this.

    And also as requested, the password history is now handled.

    Thank You!

    But I’ve stubbornly resisted arbitrarily adding the https URI protocol to protocol-less URL fields, and just using the first non-empty Email address field when a Username field is empty (e.g. for Logins).

    Sorry!

  • pedrogarcia
    pedrogarcia
    Community Member

    Yeah, that wraps it up. :)

  • AGAlumB
    AGAlumB
    1Password Alumni

    @pedrogarcia: Glad that MrC was able to work his magic to help you get your data converted! If you have any questions about 1Password, just let us know. :chuffed:

This discussion has been closed.