Synching changes item layout

Options
mschmitt
mschmitt
Community Member

This is the data corruption issue I alluded to in a comment I made 5/24 to https://blog.agilebits.com/2018/05/22/1password-7-for-mac-the-best-ever (which for some reason appears to be stuck in moderation, even though other blog entries later have shown up).

In the beginning, the Company field on Software Licenses was under the Publisher heading. So that's what I entered there: the name of the company that produced the software.

(This is distinct, in my view, from the publisher of the software. The publisher is where you got it, the company is who makes it. For example, when you get a game from Steam the company is who created the game but the publisher is Steam. You could get the same game from other sources, which would be other publishers. At least, this is how I categorize the data.)

Later the default for new Software Licenses changed, to put the Company under the Customer. I (the customer) am an individual, so I have no company. So, when I create a Software License entry, I move the Company field under the Publisher heading.

Well, earlier this year I noticed something: on one machine most of the software licenses have the Company moved back to the Customer field. And there's a problem on the main machine too: while on most of the licenses the Company field is where I put it, on 5 of the licenses it too has been moved to the Customer heading.

I suspect it has to do with synching: I think that when the data is pushed to the other machine, it changes the position of the Company field, so that it isn't where I want it. This would explain why the incidence of problems is higher on the second machine, since I don't usually create the licenses here.

I don't know if this happens with every update or just when it creates new license entries.

How can I fix this? I'm kind of scared to touch anything; I don't want to have the sync mess up more data. For example, if I try to fix an entry on the secondary machine, I don't want that fix to propagate to the main machine by moving the Company up to Customer. What I mean is, if the data sync is what screws up the data, then causing more data synching would screw up more data.

And what will happen when I upgrade to 1Password 7? Is it going to get worse? Is it going to have conflicts because as it is now, the two machines are not exactly in sync? (This is why I asked in [missing] blog comment if there was a way to buy the 1Password license without installing it first; I don't want to upgrade until I'm sure it won't harm my data.)

Note: I don't know if this is just a problem with Software Licenses. It could affect every type of entry. But software licenses are the only entry type where I usually move fields in the layout.


1Password Version: 6.8.9
Extension Version: Not Provided
OS Version: 10.12.6
Sync Type: Dropbox

Comments

  • AGAlumB
    AGAlumB
    1Password Alumni
    Options

    @mschmitt: Thanks for getting in touch. Certainly we do see sync issues with Dropbox from time to time, but I'm not familiar with any of this specific sort. 1Password 7 won't make a difference in this regard though, as all of that is handled by Dropbox itself. I'm also not aware of changes to the Software License template, but depending on how long ago you originally created the item that's potentially something I've forgotten — perhaps around the 1Password 3-4 time frame. How many items are you having this trouble with? It might be best to create a brand new item to save the data.

  • mschmitt
    mschmitt
    Community Member
    Options

    Correction: I think originally the software license template had no field groupings, so the meaning of the Company field was ambiguous.

    I don't think we can blame this entirely on Dropbox. This weekend I ran an experiment, for the purpose of testing 1Password 7:

    1. Installed 1Password 6 on a High Sierra virtual machine, which did not have a) 1Password or b) Dropbox. So this is a clean install.
    2. Copied the agilekeychain to the virtual machine, and...
    3. Told 1Password 6 to use it.
    4. The result was that every entry now had the Company field in the Customer area (or so I thought). So I started moving the Company to the Publisher area.
    5. Then I realized that actually what had happened was that the installation had duplicated the Company field: every entry had the Company in two places: Customer and Publisher. (This is with entries that in the source machine were OK.)
    6. But remember that I had already moved the Company from Customer to Publisher on about half of the entries, before realizing it was already there. So on these entries, it now has the Company twice in the publisher area: once as the first field and again as the last.
    7. OK, so now I tried deleting the extra Company. Here's where it gets really weird: when I deleted the extra Company, it deleted both. That was not a fluke. It did that every single time.

    So what we can see here is that there are weird things going on even with synching completely out of the picture. This is simply issues in attempting to edit fields in the license.

    Eventually, after a lot of work, I cleaned up all of the licenses, and then upgraded the virtual machine to 1Password 7. I can say that that the upgrade did not make the licenses worse.

    I have issues on one machine or the other with every single license I have, which is about 75. The problems are not just with old licenses; I first noticed this with licenses I created a few months ago.

    I just ran a test to see if it is possible to correct the data when synching is involved. The answer is no. On any machine, if I correct it there, it screws it up on the receiving end of the sync. If I fix it there, it then corrupts the data again on the first machine.

    It is like the sync isn't respecting the field layout. If 1Password is going to permit the user to move fields around, then the arrangement should be maintained on the sync.

  • Lars
    Lars
    1Password Alumni
    Options

    @mschmitt - thanks for the follow-up!

    Eventually, after a lot of work, I cleaned up all of the licenses, and then upgraded the virtual machine to 1Password 7. I can say that that the upgrade did not make the licenses worse.

    Reading your exchange with brenty, that's more or less what i was expecting. Not certain of, but expecting. It sounds as if there may be a bug with the Software License category in terms of the database schema and field-mapping with respect to Agile Keychain. And because we've officially retired Agile Keychain as of 1Password 7 on both Windows and Mac, I suspect this isn't one we're going to be spending time trying to fix retroactively, since it doesn't appear to actually cause data deletion and anyone upgrading to 1Password 7 will be using either 1password.com accounts or the OPVault format.

    I'd be curious to know whether turning off syncing to the Agile Keychain(s) you have in Dropbox and writing out new OPVaults from your instance of 1Password 7 (which is all it can/will write), using "good" data results in any similar errors. To be a "fair" test, what you'd want to do is make sure you've got backups of everything, then not just disable sync but actually start over on all devices except the one that has the "clean" data. Make sure you write out a new OPVault file by turning off sync to the Agile Keychains (and deleting them if you want, or just moving them), and then re-establishing sync using the OPVault format, and then using that to sync and import the data anew on the other devices.

  • mschmitt
    mschmitt
    Community Member
    Options

    Oh man, was this a pain.

    1Password 7.1 is out, so time to install the upgrade.

    First mistake was to not clean up all the data in one of the computers first; I assumed I could do that in 1Password 7. Wrong. I hadn't counted on that 1Password 7 doesn't let you edit items.

    So reinstalled 1Password 6, cleaned up the data.

    Then had to struggle to get 1Password 7 to reimport from 1Password 6. First confusion was that File > Import > 1Password 6 won't import from a 1Password 6 keychain or vault; that item really should say 1Password account or something like that. But it is a moot point, because importing from a keychain doesn't work either, even if you first delete the old items you want to replace.

    Restoring from a 1Password 6 backup was a challenge too, because the last backup I made wasn't in the list. I suggest that what the conversion should have done is move the 1Password 4 backup folder to the new location and leave an alias behind, so that when you take another backup in 1P6 it shows up in 1P7. (Assuming, of course, that 1Password 6 knows how to follow the alias.)

    Anyway, I finally got it to restore the 1P6 backup with the corrected data.

    No fool I, on the other Mac I reset the local data, reconnected it to the Dropbox opvault created by 1P7, then did the upgrade. That worked. All done, right?

    Then I changed one of the iDevices to use the Dropbox opvault. It immediately uploaded its data to that vault, corrupting all the data again.

    So then I had to do it all over again: disable dropbox and delete the data, remove the 1password data from all devices but one, uninstall dropbox from the iDevices, restore the primary computer from the backup again, reestablish sync.

    On the secondary computer that took two tries. For some reason the first try only loaded part of the data back from the Dropbox opvault.

    But I finally have all the data corrected, in 1Password 7 opvault, on all devices. Only problem is that since 1P7 won't let me edit the entries, if they get out of whack again, I won't be able to fix it.

  • Lars
    Lars
    1Password Alumni
    Options

    @mschmitt - sorry to hear you ran into some trouble.

    I hadn't counted on that 1Password 7 doesn't let you edit items.

    I'm not sure what you mean by this; if you have a read-only copy of 1Password 7 for Mac because it's not yet paid for, that would make sense, but otherwise, 1Password allows you to edit items. Can you clarify what was giving you trouble?

    The File > Import dialogue works as follows:


    I'm glad to hear you have everything (more or less) sorted, but I'm still confused about the "won't let me edit." Do you see a Read Only badge in the upper right hand corner of your items? Does the Edit button not appear at all? What specifically happens when you try to edit an item?

  • mschmitt
    mschmitt
    Community Member
    Options

    Your picture says that "1Password 6 ... imports users's entire 1Password 6 for Mac database (all vaults) directly from the .sqllite file". That doesn't work for me. When I select it, I get a prompt "Select your 1Password account and a 1Password database file to import into it.", with an "Import into:" selector. But, the import into selector is greyed out. Nothing happens if I try to click it.

    Note that I don't have a 1Password account.

    If the intent is that you would use this option to import from a database or opvault, it doesn't work. And, from your description, it doesn't sound like it is intended to import from opvault. So how would you import from an opvault?

    It seems to me that you should be able to import from any 1Password data store: opvault, agilekeychain, agilekeychain_zip, 1p4_zip, sqllite database, whatever 1Password 3 used to use. May the import should have one item for all 1Password files.

    . . .

    By "1Password 7 won't let me edit", I mean that if there is a way to rearrange fields in 1Password 7, I can't find it. And remember, the entire point to this post was that 1Password synching was messing with the fields: moving them to different sections, duplicating them, moving them to different positions in the section -- all things that in order to correct, you need to be above to move the field.

    The funny thing is that there are support posts that reply to people asking why they can't move sections with "you can't move sections but you can move fields". Now you can't even do that.

    That is the most disappointing thing about 1Password 7.

    I did realize that I may be able to use 1Password 6 for editing going forward, as long as I'm very, very careful: Start 1P6, reset its data, reload from the Dropbox opvault, make change, then turn off sync. The key is to ensure it never syncs data from an old local database back to the dropbox opvault.

  • AGAlumB
    AGAlumB
    1Password Alumni
    Options

    But, the import into selector is greyed out. Nothing happens if I try to click it. Note that I don't have a 1Password account. If the intent is that you would use this option to import from a database or opvault, it doesn't work. And, from your description, it doesn't sound like it is intended to import from opvault. So how would you import from an opvault?

    @mschmitt: It sounds like you just need to open the existing OPVault in 1Password by selecting it during setup, rather than trying to "import" other stuff.

    The funny thing is that there are support posts that reply to people asking why they can't move sections with "you can't move sections but you can move fields". Now you can't even do that. That is the most disappointing thing about 1Password 7.

    I honestly don't know what you mean. 1Password for Mac has never supported moving sections, but it has supported moving fields for many years. Are you not seeing these drag handles for some reason?

  • mschmitt
    mschmitt
    Community Member
    Options

    The field drag handles are gone from Software License items in 1Password 7, as is the widget for deleting fields.

  • Lars
    Lars
    1Password Alumni
    Options

    @mschmitt -- 1Password 7 for Mac no longer allows you to delete "default" fields -- fields that are what define a particular record type. You can delete/move/alter any custom fields you've created, but not the default fields.

This discussion has been closed.