mrc-converter-suite and handling carriage return/line feed

kamion
kamion
Community Member

mrc-converter-suite version (2020-04-22-1730)

I'm trying to import a description field into the "notes" section. The entry has some new line characters '\n', for example:

"First line of text.\nA new line of text."

I can't get this to display correctly after importing into 1Password.

I looked in the login-example.csv file, and in the example, it splits up the "Recovery Codes" with a real carriage return/new line, and mrc-converter-suite can handle this as well. This seems like a break from accepted CSV formats, doesn't it?

The generated 1P_import.1pif for the login-example.csv shows that it converts the carriage return and line feed into \r\n, but when the text itself contains '\n', it double escapes it '\\n'.

Is there a way to do this without printing the literal new line character into the generated CSV file?


1Password Version: 1Password 7 Version 7.5 (70500003)
Extension Version: Not Provided
OS Version: OS X 10.14.6
Sync Type: Not Provided

Comments

  • kamion
    kamion
    Community Member

    I posted a question here a few hours ago, regarding importing CSV into 1Password, and line feeds/carriage returns. I made an edit, and now the post I made is gone.


    1Password Version: Not Provided
    Extension Version: Not Provided
    OS Version: Not Provided
    Sync Type: Not Provided

  • ag_ana
    ag_ana
    1Password Alumni

    Hi @kamion!

    I think I managed to recover your post. Is this the one you were referring to?

  • kamion
    kamion
    Community Member

    Yes, @ag_ana . Thanks fo recovering my original post.

  • MrC
    MrC
    Volunteer Moderator

    @kamion

    Can you help me better understand.

    The login login-example.csv is a Windows CRLF example, and it converts / imports correctly:

    When I convert that login-example.csv to Unix-style line endings, it also converts / imports correctly:

    You can see both platforms here:

    So, am I missing something?

  • kamion
    kamion
    Community Member

    Hi @MrC ,
    Instead of hex 0x0a and 0x0d, I have the representation \n for a new line.

    For your easy reference:

    Title,Login URL,Login Username,Login Password,Notes,Member Number,Recovery Codes
    GitLab,https://www.gitlab.com,wendycodes,SJcv;W*$S,"This item has a custom field recording 2FA recovery codes\nA new line",,"c584debad339c771\na9238ce3f9e96abc\n925b98f25ec7ec76\ndf32e0d015285b36"

    This gets converted to:

    See it becomes double escaped in the 1pif file?

    Finally, when it is imported into 1Password, it still shows the \n character.

  • MrC
    MrC
    Volunteer Moderator

    @kamion ,

    Thanks for explaining.

    The two character sequence “\” followed by “n” is not the same as a newline character in a CSV file. Backslash is not an escape.

    Where did this sequence or CSV file come from?

  • kamion
    kamion
    Community Member

    @MrC

    It came from an export from a mySQL database.

    Yes, CSV shouldn't care about the \n, so it should just import it verbatim, but it doesn't, as it appends a double escape to it instead.

    If it had just ignored it and imported it verbatim, it will work fine as 1Password print will respect the \n.

    Can an option be added to your converter to so say "ignore \n\r in text fields"?

    Thanks.

  • MrC
    MrC
    Volunteer Moderator
    edited May 2020

    @kamion ,

    It’s not the CSV import decoding that causes the doubling of the backslash, but the encoding into the 1PIF, which is JSON. A backslash must be escaped with another backslash. So, \n gets encoded as \\n. There is no way to know that the original source was meant to be a newline (as it was not properly encoded).

    1Password should be decoding this on import into a single backslash.

  • kamion
    kamion
    Community Member

    @MrC ,
    OK, at the end, I just did a search and replace on the 1PIF file.

    I had to put in a placeholder for the double quote " character, as that's not accepted by the CSV importer.

    I searched 1PIF file for double back slash \\ and replaced it with a single one \.

    Seems to have worked. Thanks for your time in looking at this and your help in explaining how it works.

  • Hey @kamion. Thanks for the update here. I'm glad to hear you were able to find a solution. And thanks as always to @MrC for helping out. :)

    Ben

  • MrC
    MrC
    Volunteer Moderator
    edited May 2020

    @kamion

    I had to put in a placeholder for the double quote " character, as that's not accepted by the CSV importer.

    A double quote in a CSV needs to be escaped by another double quote.

  • kamion
    kamion
    Community Member

    @MrC

    Yes, a "double double quote" works with the CSV importer.

    Anyway, this was that I was trying to do: https://github.com/tanyanghan/passphrase_to_1password

    I was tasked with extracting the whole Phabricator Passphrase database and importing it into 1Password.

    Clunky, but it seems to work. Thanks again for your help!

  • ag_ana
    ag_ana
    1Password Alumni

    On behalf of MrC, you are welcome @kamion!

    If you have any other questions, please feel free to reach out anytime.

    Have a wonderful day :)

This discussion has been closed.