mrc-suite-converter importer history [keepass2]

PasswordNuker
PasswordNuker
Community Member
edited September 2019 in 1Password 7 for Windows

Hello, I have KeePass 2 and can confirm entries are exported by KeePass 2 as XML and I see them. But when importing the 1password file into 1password it does not transfer my historical passwords which is very important. Any solution to this?


1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Sync Type: Not Provided

Comments

  • MrC
    MrC
    Volunteer Moderator
    edited September 2019

    I'm so sorry, @PasswordNuker . This was a bug I introduced inadvertently. I've just posted an update.

    If you don't want to do a full update, the only line that changed that you'd care about is line 169 in Converters\Keepass2.pm, which was:

    my $histentry_data = get_entrydata_from_entry('History element', $xp, $histentrynode, 1);
    

    and you want it to be:

    my $histentry_data = get_entrydata_from_entry('History element', $xp, $histentrynode, 0);
    

    The only change was the last argument to the function call - it should have been a 0 and not a 1 (since it tells the function whether or not to ignore time stamp values, and these are needed for the converter to consider these values).

    From the Changes.txt file:

    2019-09-12 -----------------------------------------------------------------------------------------

    • Fix [keypass2]
      The passwords history was not being imported (caused inadvertently by the change that supported
      --notimestamps).
  • PasswordNuker
    PasswordNuker
    Community Member

    Thanks that worked.

    Use of uninitialized value $_ in substitution (s///) at I am getting this while running with verbose on an attachment, is there a way to know which one or is it the one above before this error appears?

  • MrC
    MrC
    Volunteer Moderator

    @PasswordNuker ,

    Add the -d option (or --debug). I'd be interested in what is causing this.

  • PasswordNuker
    PasswordNuker
    Community Member
    edited September 2019

    I see a lot of data in the debug, is there any way to just do attachments?

    I think I managed to narrow it down to the gmail backup codes .txt attachments.

    1. 111 111 111 6. 666 666 666
    2. 222 222 222 7. 777 777 777
    3. 333 333 333 8. 888 888 888
    4. 444 444 445 9. 999 999 999
    5. 555 555 555 10. 999 999 999

    This is how it looks in Keepass and when exported out it is

    1. 111 111 111 6. 666 666 666
    2. 222 222 222 7. 777 777 777
    3. 333 333 333 8. 888 888 888
    4. 444 444 445 9. 999 999 999
    5. 555 555 555 -10. 999 999 999

    Notice the extra dash.
    @MrC

  • MrC
    MrC
    Volunteer Moderator

    @PasswordNuker

    There isn't that type of debug granularity available.

    Near the top of the Converters\Keepass2.pm script, find this line:

    use MIME::Base64;
    

    and add the following line beneath it, and re-run the script

    $SIG{__WARN__} = sub { die $_[0] };
    

    It will cause the script to die when that warning is issued. Use with -d mode to see which entry is causing the issue. Let me know the line number that is causing the issue, and provide me with the full error messages (bleep out anything confidential).

  • PasswordNuker
    PasswordNuker
    Community Member

    find_card_type : type detected as 'login' (key='Password')
    Use of uninitialized value $_ in substitution (s///) at
    C:/Users/PasswordNuker/Desktop/mrc-converter-suite/./Utils/Utils.pm line
    102 (#1)
    (W uninitialized) An undefined value was used as if it were already
    defined. It was interpreted as a "" or a 0, but maybe it was a mistake.
    To suppress this warning assign a defined value to your variables.

    To help you figure out what was undefined, perl will try to tell you
    the name of the variable (if any) that was undefined.  In some cases
    it cannot do this, so it also tells you what operation you used the
    undefined value in.  Note, however, that perl optimizes your program
    and the operation displayed in the warning may not necessarily appear
    literally in your program.  For example, "that $foo" is usually
    optimized into "that " . $foo, and the warning will refer to the
    concatenation (.) operator, even though there is no . in
    your program.
    

    Uncaught exception from user code:
    Use of uninitialized value $_ in substitution (s///) at
    C:/Users/PasswordNuker/Desktop/mrc-converter-suite/./Utils/Utils.pm line 102.
    Converters::Keepass2::ANON("Use of uninitialized value \$_ in
    substitution (s///) at C:/Us"...) called at C:/Users/PasswordNuker/Desktop/mrc-
    converter-suite/./Utils/Utils.pm line 102
    Utils::Utils::fs_safe(undef) called at C:/Users/PasswordNuker/Desktop/mrc-
    converter-suite/./Utils/Utils.pm line 113
    Utils::Utils::create_attachment(SCALAR(0x402ee30), undef, "1112223331.jpg",
    undef) called at C:/Users/PasswordNuker/Desktop/mrc-converter-
    suite/./Converters/Keepass2.pm line 153
    Converters::Keepass2::do_import("..\pm_export.xml", undef) called at
    convert.pl line 175

  • PasswordNuker
    PasswordNuker
    Community Member
    edited September 2019

    https://pastebin.com/XLnwq0i9 link to the error.

    @MrC

  • MrC
    MrC
    Volunteer Moderator
    edited September 2019

    Ok, great @PasswordNuker , thanks.

    This was caused by an untitled entry you had in Keepass2. The update I just posted should resolve this.

  • PasswordNuker
    PasswordNuker
    Community Member

    Thanks this worked. Thanks for all the assistance.

  • PasswordNuker
    PasswordNuker
    Community Member

    I just have a question regarding the - being added to the number when exported txt, is there any way to prevent that?

  • MrC
    MrC
    Volunteer Moderator

    You're welcome, and thanks for the assist there, @PasswordNuker

    The change in the encoded data attachment confuses me. Is there some way I can reproduce it? Can I just create a .txt file with those values?

  • PasswordNuker
    PasswordNuker
    Community Member

    Make a file called Backup-codes-mrc (1).txt and import into KeePass 2 and then use converter.

    Have this content https://pastebin.com/fVMQrejs. You will see the 10 gets - added to it after export.

  • MrC
    MrC
    Volunteer Moderator

    @PasswordNuker

    I followed your instructions, but don't see any issues.

    What encoding did you select when you attached the .txt file?

  • PasswordNuker
    PasswordNuker
    Community Member

    I am not entirely sure, it was an old attachment that was added long time ago. I see I have two entries of this one so I assume it was the older copy. I think it may have been imported incorrectly using some weird encoding long time ago so I think we shouldn't look too deeply into it.

    Thanks for the great work on converter.

  • PasswordNuker
    PasswordNuker
    Community Member

    I am not sure why but deleting it all and importing again I didn't get the strange - on the numbers. @MrC

  • MrC
    MrC
    Volunteer Moderator

    @PasswordNuker

    That's great news. The corruption of the attachment decoding made no sense to me since it is being done by well-tested modules, and any such issues would have been detected long ago. Those darned Gremiins!

    Thanks for the updates.
    Enjoy 1Password

  • PasswordNuker
    PasswordNuker
    Community Member

    I do have one question @MrC, how easy would it be to import all 2FA entries?

    The key is saved as KEY=NUMBER under otp custom field in KeePass 2 and used by OTP plugin.

  • MrC
    MrC
    Volunteer Moderator

    @PasswordNuker

    I just implemented TFA for another converter. I'll take a look today.

  • MrC
    MrC
    Volunteer Moderator
    edited September 2019

    @PasswordNuker

    I've implemented the OTP feature from the keeotp plugin. If you can test out the update, I'd appreciate it.

  • PasswordNuker
    PasswordNuker
    Community Member

    @MrC
    Thank you for the awesome great work, I do have one more question. I have a field called email on most entries. I see you have a sample there with custom fields but I don't understand the arguments of how I would add my field to the export. It's called email.

  • MrC
    MrC
    Volunteer Moderator
    edited September 2019

    @PasswordNuker

    You're welcome.

    All non-stock fields will go to the Notes section, unless you use the --addfields option, whereby these are placed into a separate section as custom Key / Value pairs.

    You can add additional custom definitions to a table at the beginning of the converter to place items into particular sections, etc.

    Where / how exactly do you want your email field to be resolved?

  • PasswordNuker
    PasswordNuker
    Community Member
    edited September 2019

    For entries where there is username I have a custom field called email, I will place it as a custom field and think that will suit my purposes.

    Can I also skip fields like keefox setting stuff that are unnecessary?

    @MrC

  • MrC
    MrC
    Volunteer Moderator

    @PasswordNuker

    It sounds like --addfields is what you want then.

    There's nothing currently in the converter to allow user-defined Ignore / Skip functionality. This could be dangerous, since users aren't necessarily skilled in understanding the implications of creating such definitions. To be most flexible, I'd code this in such a way that a user could supply a list of patterns to match (to ignore).

    If you know the exact field names, and you are certain you want these fields ignored in all items, I can show you have to add a few lines of code to do that.

  • PasswordNuker
    PasswordNuker
    Community Member
    edited September 2019

    I would like to ignore KPRPC JSON field and also I noticed I have some entries with an older KeePass OTP version.

    https://i.imgur.com/yN0hRT6.png

    The TOTP Seed contains the key only. Is it possible to have this field added as well?

    I assume that it reads the older compatible entries without issues in KeePass but it isn't being imported into 1password cause of the different format.
    @MrC

  • MrC
    MrC
    Volunteer Moderator

    I would like to ignore KPRPC JSON field and also I noticed I have some entries with an older KeePass OTP version.

    I have placed a custom version of the converter for you into the Custom folder where the converter suite lives. Just replace your Converters\Keepass2.pm file with it.

    You can edit the entry at the top of that file, which looks like:

    my @ignored_keys = (
        'KPRPC JSON',
    );
    

    To ignore more keys, just add similar strings, such as:

    my @ignored_keys = (
        'KPRPC JSON',
        'A Key To Ignore',
    );
    

    The TOTP Seed contains the key only. Is it possible to have this field added as well?

    What would be the purpose of this, since you have the OTP now via another key? What would I do if both exists? What do you want done with this field?

    I assume that it reads the older compatible entries without issues in KeePass but it isn't being imported into 1password cause of the different format.

    Sorry, I don't understand. Can you clarify?

  • PasswordNuker
    PasswordNuker
    Community Member

    @MrC
    Thank you I missed the notification, I have some entries that one either has new format and one old, but I could just check each entry manually as I don't have that many OTP entries.

    Before I was using tray OTP I believe which is what generated these fields in the entries with the TOTP seed. Then I started using keepass OTP and I got the new format that you added and it also reads entries with the old format cause it's backwardscompatible. At one point I was also using KeePass2 XC which uses another OTP plugin so not sure if that also added the old format.

    It's not a big bother but depending on what plugin users used they'd have one of two formats. I don't think any entry would contain both considering they are both read and if readded OTP I assume it removes the old field.

    I will give this new version a try. Thank you so much!

  • MrC
    MrC
    Volunteer Moderator

    @PasswordNuker ,

    I've posted an update to the converter in the Custom area for you to try. It adds support for Tray TOTP's TOTP Seed value.

  • PasswordNuker
    PasswordNuker
    Community Member
    edited September 2019

    Thank you it worked, I just had to hide TOTP Settings, I am grateful for the swift response.

    @MrC Do you have a PayPal or something of the sorts?

  • MrC
    MrC
    Volunteer Moderator

    @PasswordNuker ,

    Great, thanks for the prompt feedback. I think I'll add the ignore code into the keepass2 main code, but comment-out the patterns, so it will be available for use, but not active by default.

    Your thanks was sufficient payment.
    Enjoy 1Password!

This discussion has been closed.