Changing master password and re-encyption of 1P data

pomme4moi
pomme4moi
Community Member

From reading other posts, I understand that changing the 1P master password does not cause 1P data to be re-encrypted with a new encryption key. I think I further understand that if one wants to change the 1P master password and re-encrypt the 1P data, then one has to export the existing 1P data and start over with a fresh install of 1P. Just curious I guess, but why is 1P designed this way? When a user changes the master password, why not create a new encryption key and re-encrypt the 1P data so that the old master password no longer works?

«1

Comments

  • Hi @pomme4moi,

    Good question. :)

    I've taken Khad's explanation from another similar discussion:

    Background

    The solution we recommend is that people pick a good Master Password to begin with and then never change it. But of course there are cases where people do need to change a Master Password:

    • If your Master Password is weak, a strong one should be used.

    • If your Master Password is reused, a unique one should be used.

    What seemed like a good Master Password in 2008 may not pass muster today, and your Master Password should never be used anywhere else.

    There is a marginal case in which changing your Master Password weakens your security. When your keychain is created a truly random encryption key is generated, and that key is used to encrypt your data. Let's call it your master key. The master key does not change when you change your Master Password. Your Master Password is used to encrypt your master key, and your master key is used to encrypt your data.

    So if an attacker can get hold of an "old" copy of your encrypted master key, they can attack that by going after the old Master Password. If they get hold of a copy of a new copy of your encrypted master key, they can attack that by going after your new Master Password. So depending on what an attacker has access to, a Master Password change can increase the attack surface.

    1Password is like other high security tools such as SSH and PGP/GPG in this regard. The actual key doesn't change with a password change, but how that key is encrypted.

    Since your Master Password protects a secret key that never actually changes, old copies of your key files (1password.keys, .1password.keys and encryptionKeys.js) will remain protected only by your old Master Password. Because your more recent data are encrypted with the same decryption key, then even those can be gotten at by cracking an old Master Password.

    How to re-encrypt

    You can export all your 1Password data to a 1Password Interchange File (1PIF) (File > Export). Be sure to export all items and choose the 1PIF option. Then move your old data aside and create a fresh 1Password data file (which 1Password will prompt about if it doesn't find a data file where it expects one). That new, empty, data file will have a new random key encrypted with your (new) Master Password. Then import from the 1PIF file. All the data will now be encrypted with the new random key.

    Note that, as the warning states when you create the 1PIF, the 1PIF is not encrypted; treat it very carefully.

    Be sure that your import works and that you have everything you need before wiping your old 1Password data. On a Mac you can securely erase the 1PIF using Secure Empty Trash or srm from the command line.

    Caveat

    • Old data will still be available on Dropbox if you have been syncing with Dropbox. To delete files from Dropbox backups, see their guide on this here:

      https://www.dropbox.com/help/40

      That process will remove the file and all of the previous versions from Drobpox. Also be aware that 1Password makes backups of your data on your computer. See Preferences > Backups > Show Files in 1Password to see where those are kept.

    Why this hasn't changed (yet)

    One thing to keep in mind is that our design "solves" a number of security problems that might not be obvious. Here are a few:

    1. There is a limited amount of data that should be encrypted under the same key.

      Now 1Password databases aren't getting that big (yet), when you have a structure that allows people to add an unlimited amount of data, you need to use multiple keys.

    2. To get the full strength of 128-bit (or 256-bit) keys, those keys should be generated completely at random.

    3. To have a system where only the minimum amount of data necessary is decrypted at any single time requires separate keys for each item.

    4. Changing the Master Password should not be a process that takes many minutes during which a power failure or computer crash might leave data unusable.

    But this does leave the problem that a Master Password change does not have the effect that it might seem.

    For us the real problem is how to have something that is simple and straightforward to use, without people needing to study the details, that works in a way so that it makes it easy for people to behave securely and hard to behave insecurely. This really is the overriding goal of what we do here. As a consequence, 1Password presents itself to people as much much simpler than it really is. In nearly all cases that is a very good thing. But there are still those small number of cases where what is presented to people by 1Password can be misleading in a way that can lead them astray.

    And there is another problem. The problem is that sometimes we do have to make tradeoffs of defending against one kind of threat versus another. Although I acknowledge the downside of the choice that we've made here, it was not a tough decision. The security benefits of our design choice are overwhelming.

    We'd love to have a system that did everything, including conforming to intuitions about changing Master Passwords. Perhaps the next time you need to change your Master Password we will have a smoother process. (Though neither SSH nor PGP/GPG seem to have come up with anything better when faced with an analogous problem.) We'll keep exploring ideas, but we require that any such system do more good than harm.

  • Plato
    Plato
    Community Member
    edited November 2014

    @JasperP

    These instructions, quite simply, do not work.

    When I try to _"Then import from the 1PIF file."_either of two things occur.

    The import function is grayed out.

    The import function is not grayed out but my only subsequent option is to open the .1pif (1Password %2014-%23-%20 %19_%11_%9) folder. Nothing gets imported and, in fact, the "import" text is never present after the original decision. My only choice is "open" and when I make that choice, nothing happens. (The folder doesn't open.)

  • Plato
    Plato
    Community Member
    edited November 2014

    @JasperP‌

    Well, I kinda got somewhere but backwards. I was able to import the .1pif file but I went through so many gyrations I'm not sure what finally worked for me. I then realized that the command-backslash didn't work. I tried everything that I could think of but couldn't make it work. I also discovered my Master Password and hint were one version PRIOR to what they were before I started.

    I gave up and restored to the configuration that I had when I started five hours ago (fortunately I had saved the data from "Containers"). Unfortunately, iCloud still has the old Master Password and hint and I'm unable to configure to sync automatically. Apparently the sync had not been functioning properly before but I didn't know it. Below is what I see when I try to sync. I can put in the old Master Password and it will sync once. When I try again, that screen appears again and the old Master Password is again required.

    How can I force iCloud to update itself to the current iMac configuration?

  • Plato
    Plato
    Community Member

    @JasperP‌

    Well, I'm not sure what worked but I think that I finally have everything functioning properly. However, I suspect that I have fourteen old master keys that the bad guys can attack. A big part of my concern right now is that I am clueless regarding whether or not I removed all of the "old data" as instructed in your procedure. Unfortunately, you didn't identify exactly where all that "old data" resides.

    Hopefully you can shed some light on what data I should have removed and where I should go from here.

  • littlebobbytables
    littlebobbytables
    1Password Alumni

    Hi @Plato‌

    I had to stop and get coffee before tackling this one, hope you don't mind :wink:

    Okay, so if I've read all your posts correctly everything is working after your experiments, you just need to ensure exporting and moving stuff around hasn't left any data lying around, is that the case?

    Given the import window screenshot you've supplied the one to be most careful about is the .1pif folder.

    What I'd recommend there is emptying your Trash as you'd normally do, drag the .1pif folder to Trash and then in Finder > Secure Empty Trash... just to be safe. The reason for emptying your Trash beforehand is Secure Empty takes longer so you only want to use that for the stuff you need to know is deleted.

    The other files visible in that screenshot are the two Containers folders but I'm guessing you moved them back into Containers and overwrote the existing ones that were just created? If that assumption is correct that's probably all there was. Now if you'd had success with the .1pif import then you'd be wanting to delete those folders as you have new ones.

    So if you've securely erased your .1pif folder I think you should be okay.

  • Plato
    Plato
    Community Member

    @littlebobbytables‌

    Well, your assumptions are kinda correct. The 2BUA... folder and the com.agilebits... folders that you see on the desktop were the originals that were inside Containers. I had believed that they constituted the data that I was to (eventually) delete.

    However, it was completely impossible for me to create a brandy new vault with those two folders not in Containers. 1password would open with a blank screen and a useless menu. I put the com.agilebits... folder back inside Containers and tried again. At that point, my options were to create a brand new vault or import from iCloud (which, of course, had a vault from several days ago). I created a new vault and then imported the .1pif file. So far, so good.

    The next issue was the inability to sync (reference the "Merge with iCloud vault" screen above). I did the merge and then tried another sync. Same screen appeared. I repeated this perhaps four or five times.

    Well, at this point I had no clue what impact the merge had on my vault on the iMac so I went back to Containers and deleted the new 2BUA... folder. I emptied the trash securely because I would have no need for this new folder (the original 2BUA... folder was still on the desktop). I then repeated the above except that I did not allow the merge. On a hunch, I checked the Dropbox sync checkbox. A message appeared telling me that the vault in iCloud would be erased (which I wanted). I went back to iCloud sync and allowed a sync. All OK now. After checking the bulk of my logins, I deleted the .1PIF file and the original 2BUA... folder (secure erase for both).

    I then went to my MacBook and first deleted the 2BUA... folder. Feeling confident, I trashed it and did a secure erase. I opened up 1password and selected the option to copy the vault from iCloud. Worked like a charm.

    Finally, I went to my iPod and delete and re-installed 1Password which deletes data. I did a sync and all OK with the iPod.

    All of the above took me about eight hours of pure agony. You guys have a good product but your ability to provide user instructions absolutely stinks. YOU STILL HAVE YET TO ADVISE ME EXACTLY WHAT DATA JASPER WAS TALKING ABOUT WITH THE WORDS "Then move your old data aside..." Heck, he didn't even mention the Containers folder!

    Further, I still do not know if I have resolved the issue that Jasper was alluding to. CAN YOU CONFIRM THAT ALL IS OK NOW?

  • littlebobbytables
    littlebobbytables
    1Password Alumni

    @Plato‌

    Old data are any copies of the following:

    • Exported vaults e.g. .1pif folders.
    • Any copies of 2BUA8C4S2C.com.agilebits.onepassword-osx-helper and com.agilebits.onepassword-osx not currently in ~/Library/Containers/

    That's why I asked about all three in my previous post, as I could see you have all three on your desktop.

    The instructions didn't specifically mention the Containers folder because MAS applications store their files in a different location to non MAS ones. If you had been running the web store version you would have needed to look in ~/Library/Application Support/1Password 4.

    Given your latest post it sounds like you've created a new vault with new encryption keys and Master Password and successfully imported the .1pif folder but in one of your previous post I read:

    I gave up and restored to the configuration that I had when I started five hours ago (fortunately I had saved the data from "Containers").

    So I was under the belief that you'd reverted to your old data. Given that you've synced with two devices now and no trouble it would seem safe to say you've done it correctly, even if the route seemed torturous.

    While the guide could definitely use additional steps and an awareness of the different file locations, it is only aimed at people very comfortable with how 1Password works from a technical perspective. In fact we would probably discourage most people due to the hassle if something goes wrong. Automating the procedure in a way safe to your data is something we're working on though.

  • Plato
    Plato
    Community Member
    edited November 2014

    @littlebobbytables‌

    1. As I indicated previously, my com.agilebits.onepassword-osx folder is the original com.agilebits.onepassword-osx folder. DO I STILL HAVE THE WEAKNESS DESCRIBED BY JASPER?
    2. "The instructions didn't specifically mention the Containers folder because MAS applications store their files in a different location to non MAS ones." Would it have been all that difficult to state "If you have the MAS version,,," and "If you have the non-MAS version...?" Also, doesn't my use of iCloud give you a hint?
    3. Also, please be reminded that you not only didn't identify the location of the data, you also did not identify the name of the folder(s) that should have been replaced. SHOULD I HAVE REPLACED THE com.agilebits.onepassword-osx FOLDER?
    4. "Given your latest post it sounds like you've created a new vault with new encryption keys and Master Password and successfully imported the .1pif folder but in one of your previous post I read:" Please realize that the latest post is the most recent post and, if any information there differs from that in a previous post, that is the result of action that took place subsequent to the preceding post. Because of the action described in my latest post, you should have concluded that I now have a new vault with new data. HOWEVER, BECAUSE I AM STILL USING THE ORIGINAL com.agilebits.onepassword-osx FOLDER I DO NOT KNOW IF I HAVE THE PROBLEM THAT JASPER ALLUDED TO.
  • Stephen_C
    Stephen_C
    Community Member

    It seems to me there is a very simple way to check if you have successfully changed the encryption key. Try using your old master password. If it doesn't work then, as I understand it, the key has been changed.

    Stephen

  • Plato
    Plato
    Community Member

    @Stephen_C‌

    Is Jasper_C incorrect?

    Since your Master Password protects a secret key that never actually changes, old copies of your key files (1password.keys, .1password.keys and encryptionKeys.js) will remain protected only by your old Master Password. Because your more recent data are encrypted with the same decryption key, then even those can be gotten at by cracking an old Master Password.

  • Plato
    Plato
    Community Member

    @Stephen_C‌
    @littlebobbytables‌

    Please explain. I am totally confused here.

  • Fairgame
    Fairgame
    Community Member

    If you did not read this post yet, it might answer some questions.

    discussions.agilebits.com/discussion/20236/why-doesnt-a-master-password-change-sync-to-my-mac/p1

  • littlebobbytables
    littlebobbytables
    1Password Alumni

    @Plato‌

    On this occasion Stephen_C is mistaken. The keys only change if you start with a brand new primary vault, hence the instructions. It's an easy enough mistake to make unless you understand the entire process and what is encrypting what.

    I've had to read and re-read your posts to try and follow them. As far as I can tell you've never mentioned what version of OS X or 1Password you're running. 1Password 4 Agile Web Store version was able to sync to iCloud so knowing iCloud isn't enough to guarantee the MAS version.

    The sqlite file representing your vaults is stored in the 2BUA8C4S2C.com.agilebits.onepassword-osx-helper folder. As long as the final sequence that led to your existing vault did include having to create a new primary vault then new keys were generated. Returning the original com.agilebits.onepassword-osx back into the Containers won't affect that.

  • Plato
    Plato
    Community Member

    @littlebobbytables‌

    Thank you. Finally this is making some sense. First, I have up-to-date Yosemite on two computers (iMac and MacBook) and 5.02 1Password. Let me repeat this back to you to ensure my understanding. If I explain this properly, all you need say is "Yup."

    On the iMac, remove the 2BUA folder and put it where I can get it if something goes wrong.

    Leave the com.agilebits folder as is.

    Start up 1Password and create a brandy new vault.

    Import the previously exported .1pif file.

    If everything works fine, delete the original 2BUA folder.

    Temporarily select Dropbox so that iCloud trashes its old vault information. Then immediately select iCloud again and allow the sync to complete.

    Repeat the procedure on the MacBook EXCEPT do NOT create a new vault. Allow 1Password to obtain the vault from iCloud.

    The encryption key now matches my new Master Password and I don't have any worries about the multiple passwords that Stephen_C alluded to.

  • littlebobbytables
    littlebobbytables
    1Password Alumni

    @Plato‌

    Is it your intent to once more follow a sequence of instructions with the goal to start again with fresh encryption keys?

    If it is, there are modifications and additions to that sequence I would like to make and I will happily write them down step by step.

    If you're just wanting to know if that sequence will have yielded new keys then the answer is yes.

  • Plato
    Plato
    Community Member

    @littlebobbytables‌

    Thanks for the response. That's exactly what I wanted to know about the keys. I kinda almost did exactly what my steps indicated. One minor difference is that I initially removed the com.agilebits folder and, when I determined that it was the wrong thing to do, I simply put it back (untouched).

    I don't expect that I'll do it again the near future but I am curious what additions you would recommend. If you're interested, I'll take the combination of my procedure and your changes and reword everything to make it more suitable as a tech note.

  • littlebobbytables
    littlebobbytables
    1Password Alumni
    edited November 2014

    General Warning to all users. This is an advanced topic and you had better know exactly what you're doing. Your data is at risk if you don't.

    Hi @Plato‌

    Sorry for the delay. We had a surge of posts and as this was more about edification I figured you wouldn't mind it I got round to after ensuring the forums were in a reasonable state.

    So what would I do differently?

    1. Open up 1Password and use the File > Export > All Items... on each vault you have. It is important the file format is set to 1Password Interchangeable Format (.1pif). If you find you can't click on the Save button just press the return key when the text editing cursor is in the File name field (I believe it's a bug). This should result in a .1pif folder per vault.
    2. Disable any syncing you are currently doing on all of your devices. You can do this in 1Password's preferences in the Sync tab. You can disable sync using the Change Syncing... button and tick the Delete data from XXXXX checkbox as you want the old sync data removed.
    3. Follow the steps in our Starting Over guide. I have one exception. If you are a Mac App Store customer then after step 2.2 (where you've moved the 2BUA8C4S2C.com.agilebits.onepassword-osx-helper folder I would also move the com.agilebits.onepassword-osx folder in ~/Library/Containers/.
    4. Follow the steps in our Running 1Password for the first time guide and create a new primary vault.
    5. From your primary vault create as many new secondary vaults as you require using 1Password > New Vault...
    6. With each vault active, import the .1pif file for that vault using File > Import... option. Selecting the .1pif folder (not the contents) will enable the Open button.
    7. With all .1pif files imported, check your vaults to ensure everything looks good.
    8. If all is correct move the .1pif folders to your Empty Trash and then use the Secure Empty option which you can find in Finder > Secure Empty Trash... We want to do this because .1pif data is unencrypted and so not safe to keep. You can delete the folders dragged to your desktop in step 3. although if you want to keep them for a little longer it isn't as bad because they are at least encrypted (although using your older encryption keys).
    9. Re-enable Syncing in the copy of 1Password where you created the fresh new vault. This .agilekeychain will have the new encryption keys from your fresh vault.
    10. Personally I'd then follow the Starting Over guide for all my other Macs and then follow the Getting started again on a new computer guide. For iOS devices running version 4 we have this Starting over guide, for version 5 users they can clear data and settings by using Settings > Advanced > Erase Data and Settings. For both versions you can then follow the Existing 1Password user guide.

    That would result in fresh vaults on all devices using the new synced .agilekeychain with the new encryption keys. Now Plato, I know step 3 is going to stand out to you. I know you had troubles but I suspect as you didn't know about that guide you didn't do step 2.3 Reboot your Mac. Aggressive caching can produce some really odd behaviour if you don't reboot.

    On my test machine (which suffers some awful abuse) I've used these steps (including removing that additional folder in step 3.) running 1Password 5.0.2 on Yosemite and it all worked.

  • Plato
    Plato
    Community Member

    @littlebobbytables‌

    When I get a bit more time, I'll read your post with a bit more concentration. However, you're correct in your comment regarding step #3. I had not turned off the syncing (per step #2 of your post) at that point in time. Also (per step #2 of Starting Over), I didn't reboot. I did discover the benefit of rebooting after I had tore out all six of my remaining hairs. After that, I rebooted every time that I breathed but I didn't bother removing the com.agilebits folder again.

  • Plato
    Plato
    Community Member

    @littlebobbytables‌

    I tried your procedure. It worked just fine but there were two weird problems, one with the export and a similar one with the import. When I first tried to export, the following window appeared:

    Unfortunately the "Save" button didn't work! (Everything else worked.)

    I opened it up to get the following screen and the "Save" button worked there!

    The import had a similar problem although the good/bad windows were different.

  • littlebobbytables
    littlebobbytables
    1Password Alumni

    Hi @Plato‌

    Yeah, I found the same thing (from my step 1.)

    If you find you can't click on the Save button just press the return key when the text editing cursor is in the File name field (I believe it's a bug).

    I'm off to check and see if we have a bug report given you're reporting the same quirky behaviour.

    Other than that it all went okay and you found the instructions easy enough to follow? (given there were no useful screenshots to guide).

  • Plato
    Plato
    Community Member
    edited November 2014

    @littlebobbytables‌

    You are correct. Other than the export/import issue, everything went fine. It's weird. I went back to the export function just to take a picture and the "Save" button was then functional! For some reason, the issue affected me only the first time. I totally missed your comment about the "return" key.

    I do wish to underscore your previous statement about the reboot. I repeated the procedure several times while deliberately NOT doing the reboot. Not only it wouldn't work without the reboot, the conditions were not predictable.

    My one remaining issue with the procedure in your post is that the user is bouncing to links for other procedures. Also, it's my suspicion that a factor in me missing your comment about the "return" key is that the procedure (including the links) is lengthier than it need be. I really believe that I can streamline the instructions without losing any essential information. I need to do some leaf raking now but I'll get to it when I'm bored.

  • littlebobbytables
    littlebobbytables
    1Password Alumni

    Hi @Plato‌

    You're correct, the links to the other guides could be removed and merged for a single set. In this instance I used the links as somebody had gone to the effort of creating a series of screenshots which may have proven useful when following the guide.

    As for the reboot. I wasn't aware of this until I joined AgileBits but since something like Mountain Lion, OS X has had a fairly aggressive caching system which means when you delete something from the Library folder you may think it's gone but OS X has a copy that it's reading. At that point all bets are off. There is a way to clear that cache from a command prompt but a reboot achieves the same thing and every user of every level knows what a reboot is so we go with the reboot :smile:

  • MagicalUnicorn
    MagicalUnicorn
    Community Member
    edited December 2014

    Thanks for the information outlined in this thread I found it really useful/insightful :)

  • Megan
    Megan
    1Password Alumni

    Hi @ElectricWombat‌

    I'm so glad this thread helped! Because I'm paranoid, I'll re-iterate a caution here (just in case anyone has been skimming):

    The solution we recommend is that people pick a good Master Password to begin with and then never change it. Unless your Master Password is weak or used elsewhere, you shouldn't need to follow this epic thread to re-encrypt your database.

  • MagicalUnicorn
    MagicalUnicorn
    Community Member

    Hi @Megan‌

    Yes it was very helpful indeed, as I mentioned I've been searching the forum and found some really interesting articles on cryptography and the nature of attacks etc, there's a plethora of information in the 'Agile' archives!

    I do appreciate that this is not a required everyday/routine procedure though, I was just curious as to whether it could be achieved on IOS with less effort but it appears that's not the case. However it's still good to know what is entailed should the need arise in the future etc.

    Thanks for your help :)

  • Hi @Plato,

    I'm sorry that that Export window was misbehaving. We fixed two bugs in it for 5.1... the first being the Save/Cancel buttons not working correctly in some cases, and the other being the default filename having percent symbols in it for no apparent reason.

    If you have any other questions about how this stuff works, please don't hesitate to ask. We try to be as open as possible about how the machine works.

    Rick

  • Lamplighter
    Lamplighter
    Community Member

    Has any of this changed in the 1P for Mac updates to 5.3 ?

    I had previously decided to wait on changing my Master Password
    due to "complications" described in this thread. I'm lazy and unsure of
    following all the directions exactly. I would much prefer to simply click on
    the "Change Master Password" button, and have everything magically
    proceed perfectly under direction of the 1P5 software. :blush:

  • littlebobbytables
    littlebobbytables
    1Password Alumni

    Hi @Lamplighter,

    There hasn't been any significant change in version 5.3. There are a number of side-effects to forcing the use of new encryption keys with no clean and friendly way of handling them. While the steps above are involved, it forces the user to understand the gravity of what they're doing and ensure a clean start over all of their devices. Hopefully we can come up with a better solution at some point in the future.

  • Lamplighter
    Lamplighter
    Community Member

    @littlebobbytables

    OK, and thanks for your replies. That is what I thought... but hope never dies.

  • littlebobbytables
    littlebobbytables
    1Password Alumni

    Fingers crossed @Lamplighter :smile:

This discussion has been closed.