How to remove all duplicate entries after an import?

Options
invertedlight
invertedlight
Community Member

Trying to get all devices on the same sync I exported from Drop Box vault. Then switched to iCloud vault and imported the entries from the Drop Box export. Now I see duplicate entries with a few that were unique between the two vaults.

Is there an automated way to remove the duplicate entries?


1Password Version: 6.3.5
Extension Version: 4.6.1
OS Version: 10.12.1
Sync Type: iCloud
Referrer: forum-search:Imported into a new sync vault and now I see lots of duplicate entries. How do I easily remove dupli

Comments

  • Drew_AG
    Drew_AG
    1Password Alumni
    Options

    Hi @invertedlight,

    Sorry to hear you're having difficulty with some duplicate items!

    Trying to get all devices on the same sync I exported from Drop Box vault. Then switched to iCloud vault and imported the entries from the Drop Box export.

    Just to make sure I understand: Are both of those vaults in 1Password on the same Mac? Or is each vault on a different Mac, such that one Mac has a Primary vault that syncs with iCloud, and the other Mac has a Primary vault that syncs with Dropbox? It sounds like each vault is on a different Mac, but please let us know for sure.

    Also, it sounds like most items in each vault are the same. If each vault is on a different Mac, did you create all the items on one Mac, and then create them all again on the other Mac? Or did they used to sync with each other at one point? If most of the items are the same on both Macs because they used to sync with each other, that's good news, because what you can do is restore from a recent backup (i.e. to get you back to the state you were in before the import), then sync those vaults with each other, and 1Password should merge copies of the same items with each other instead of keeping them all separate. (Items are never merged when you do an export/import, which is why you ended up with duplicates.)

    Once we hear back from you about this, and if those vaults uses to sync with each other, we can give you the steps to do what I described above. Thanks in advance! :)

  • invertedlight
    invertedlight
    Community Member
    Options

    Both Macs were using a dropbox and then an iCloud sync. Thought I had switched both from dropbox over to iCloud... apparently not.
    I ended up having different entries in each set of devices.

    Strange thing was I saw the iPhone do a merge when i pushed it to iCloud. But the Mac did not. That is when I tried the export / import.
    Didn't think to push a backup before trying the import.

  • invertedlight
    invertedlight
    Community Member
    Options

    I see a auto backup from the 26th. If I restore from that then I'll loose the entries from the past two days.

  • invertedlight
    invertedlight
    Community Member
    Options

    As soon as I resync with the iCloud source I get double entries. Sync with the Dropbox after the restore and not much changes.
    Try to sync with iCloud and the entries double again.

  • Drew_AG
    Drew_AG
    1Password Alumni
    Options

    Hi @invertedlight,

    Sorry for the delay in getting back to you about this!

    Both Macs were using a dropbox and then an iCloud sync. Thought I had switched both from dropbox over to iCloud... apparently not.

    Ok, so it sounds like you just needed to switch to iCloud in the 1Password sync settings on the other Mac as well. That would have worked the same way it did when you switched from Dropbox to iCloud on your iPhone (in other words, no duplicates). But an export/import works very differently from the sync options - when you do an import, 1Password treats all the items in the imported file as brand new items, which is why you ended up with duplicates. But don't worry, we should still be able to help get things back to normal without you needing to manually delete each duplicated item.

    I see a auto backup from the 26th. If I restore from that then I'll loose the entries from the past two days.

    If you know exactly which items you created or updated after the 26th, you can export just those items to a .1pif file (select just the new/updated items in your vault and go to File > Export > Selected Items). After saving the .1pif file, go to File > Backup, which will create a new backup (just in case) and open the Backup preferences. Restore from the October 26 backup, and then import from the new .1pif file to get back your new/updated items (you might need to delete just a few duplicates).

    That should get you back to normal on that Mac. But don't re-enable sync with iCloud yet, because you'll need to do something else first:

    As soon as I resync with the iCloud source I get double entries.

    That's likely because those duplicates had already synced to your iCloud account. Before re-enabling sync on that Mac, you can reset the 1Password sync data in iCloud:

    • On that same Mac, open 1Password and go to Help > Troubleshooting > Reset iCloud Data.
    • Confirm you want to reset it by clicking "Yes, Remove Remote Data".
    • After a moment you should see a message that confirms the data was reset and sync was disabled (click OK).

    Now you can turn on iCloud in the sync preferences in 1Password again. After doing that, wait for the sync to complete (it will take a little extra time because it's copying your entire vault to iCloud again).

    On your iPhone and/or other Mac, if 1Password was still set to sync with iCloud, it will stop syncing when it realizes the data in iCloud was reset, and will give you an error message about that on each one (which is expected). Just re-enable sync with iCloud in 1Password on your iPhone and other Mac, and choose to merge when prompted to do so.

    Let us know how it all goes, thanks! :)

This discussion has been closed.