My Mac is not syncing with my 1Password account

drrich711
drrich711
Community Member

My Mac will not sync passwords and other edits I make on my iOS devices in real time. I need to restart the Mac for the updates to be made. I appear to have the correct vault in the sync settings, and nothing I can find is not set correctly.

Any idea why I need to restart the Mac to update, and a cure?

Thank you.


1Password Version: 7.1.2
Extension Version: 4.7.3.90
OS Version: OS X 10.13.6
Sync Type: 1 Password account

Comments

  • Lars
    Lars
    1Password Alumni

    @drrich711 - my guess would be something in your local network setup, though without looking more closely, it's difficult to say. You should not need to actually restart, however: in 1Password 7 for Mac, locking and then unlocking with your Master Password or Touch ID should force a sync with the 1password.com servers. Let me know if that does the trick (you can lock 1Password with the ^⌥⇧⌘L keyboard combo).

  • iamecho
    iamecho
    Community Member

    @Lars @drrich711 I am also having this problem and have tried locking and unlocking 1Password. I mentioned it in my question this morning. I also cannot get into my 1password account on the web for the reason listed below.
    https://discussions.agilebits.com/discussion/96654/trouble-accessing-web-version-of-1password#latest

  • drrich711
    drrich711
    Community Member

    @Lars Thank you. Doesn't the Mac version continuously sync? Do I have to sign out and in to have it sync? Is this normal behavior, or should I put in a support request?

  • Lars
    Lars
    1Password Alumni

    @drrich711 - 1Password for Mac doesn't "continuously sync" even when you're using a 1Password account, that would be problematic. What it does do is push updates to the server when changes are made, and there is a notifier which should receive changes made on other devices which have been pushed to the server and need to be changed locally. That's why I mentioned something either on your device or in your network setup, because on their own without interference, that should happen pretty smoothly and quickly; you should definitely not need to lock/unlock 1Password for Mac in order to sync. That means something's likely interfering with the notifier. Do you run AV software or other network filters/proxies like Little Snitch or HandsOff?

  • iamecho
    iamecho
    Community Member

    @Lars I am still not syncing either and do not have any software running that you mentioned. And, are you able to help me with the 2FA question I have that is preventing me from logging into 1password.com? I submitted a question (see above) yesterday morning but haven't had a response yet. Thank you.

  • Lars
    Lars
    1Password Alumni

    @iamecho - let's solve the other question in the separate thread first. :)

  • iamecho
    iamecho
    Community Member

    @Lars I am anxiously awaiting your response over there!

  • Lars
    Lars
    1Password Alumni

    @iamecho - yep, replied. :)

  • BLD
    BLD
    Community Member

    Aaargh. I ran into this too while trying to navigate updating with stuff with my son away at school remotely over the phone. This is really a bad one folks. I hope you can track it down. I got him out of it by having him "Shut down 1Password Completely" on his Macbook, quit it completely on his iPhone and relaunch both -- and then things sync'ed. But that was pretty touch and go there -- and at the worst possible moment, of course. How about a button on clients in the Advanced Settings that says "Force Sync Now" or something?

  • Lars
    Lars
    1Password Alumni

    @BLD - we've had requests for that in the past, and always resisted adding it, because if there are issues with sync, they need to be addressed directly instead of papered over with a "force sync now" button. We also place a pretty high bar on adding another button/option/toggle into the 1Password interface that comes with increased complexity and the everpresent possibility that someone will not understand it or attempt to use it in a way it wasn't designed to work. Given that you were working with your son remotely by telephone and that you solved the problem with a quit/restart approach, I don't think there's much I'd be able to glean directly from any kind of diagnostics report or logs. But I will say that although we continue to get reports of sync delays in various 1Password native clients that are using 1password.com accounts, we've yet to find one "in the wild" that turned out to be a bug in the 1password.com server/client notifier itself; it always ends up being either a local network configuration issue or having shut off wi-fi temporarily (on a mobile device, for example), or similar issues. That's not to say you couldn't have discovered such a bug, only that it would be a novel experience for us. What I'd recommend is that your soon look into what his setup is on the school (dorm, whatever) wi-fi. Was he in the library, using school wi-fi? Or on his own, tethered to an iOS device using a cellular network? Something else? If he's interested (and can get back to that specific network setup), you can try having him sign into his 1password.com account in a browser and updating the title of an item (for example) and seeing whether/how quickly it syncs to his Mac/iOS device.

  • BLD
    BLD
    Community Member
    edited January 2019

    @Lars I largely agree with everything you just wrote. It's entirely possible something in my son's network environment prevented 1P from syncing properly when it wanted to. However, as hard as I know these types of bugs are to track in the wild, I still feel that the issue is 1P's to solve. 1P should have detected it was out of sync within some reasonable amount of time after connectivity was resumed. It had been hours since I had made a change to a shared vault, and my son definitely had connectivity to the Internet during that time.

    Ensuring that multiple copies of a database have consistent copies in the presence of sporadic connectivity is a well known problem (understandably made more difficult by databases like yours where each local copy must remain usable during periods of no connectivity -- and keeping bandwidth consumption as low as possible and not overwhelmed by constant pings to check for synchronization). But I don't think it's asking too much to bury an option in the Advanced section -- saying essentially "Force Sync when things don't seem right." That looks a lot better than having to quit the app and restart from scratch -- even if that's exactly what such a button does behind the scenes.

    I just want to add that as an engineer myself, I feel your pain. But from an end user's perspective, the data being out of sync is always a bug. Having to tell my family members to relaunch everything if things don't look right doesn't go over well. Personally, I've had to "Quit 1P Completely" completely a few times to get the browser extensions even locally to start behaving again. I would totally believe that something the browser is doing is giving you heartburn --- but agian from the user's perspective, it's 1P's bug. It should "just work" -- or at a bare minimum make it as easy as possible for me to kick it in the head to start from scratch. There is no way that two 1P clients both with connectivity to the server (even if at different times) being out of sync with each other can be viewed as anything but a bug in 1P. I hope you will take that view -- I'm working very hard to evangelize this stuff with my non-tech family, and stuff like this makes my "IT job" really hard. (I promised my son last night during the middle of busy homework that I only needed 10 minutes of his time to help him get his insecure AppleID password into 1P and his laptop and phone back in shape. This problem turned that 10 into 45 -- and I was literally holding my breath when I told him to try to restarting the 1P clients on both machines. Needless to say, 1P made "IT Dad" look bad. :-P)

  • Lars
    Lars
    1Password Alumni

    @BLD

    However, as hard as I know these types of bugs are to track in the wild, I still feel that the issue is 1P's to solve.

    How? I don't mean that flippantly, I mean it quite seriously: how should we, in advance, troubleshoot a near-infinite variety of local network issues, each often specific to hardware and/or configuration (that's out of our control or even visibility) of a single user's setup? We already do take steps to mitigate or solve a variety of issues that need not be detailed here but could potentially be problematic for numerous users, and we're willing and able to help when individual users have specific problems. What I don't think we should try to do is attempt in advance to anticipate individual users' issues and try to provide code-based work-arounds for them all. It's just not possible, and would require too much of our time and effort that could be otherwise used improving 1Password in other aspects.

    To be clear, locking 1Password for Mac and unlocking it should always force a sync. Fully quitting 1Password - as you did - would amount to the same thing, since that also locks the local database.

    I certainly hear you in regard to taking up your son's time at an inopportune moment and thereby casting doubt (even if momentarily) on your "IT Dad" cred. We use a websockets-based notification service to notify client apps that there are changes for them to pull down from the server, and what you're describing seems to me as if the connection between it and 1Password for Mac is being interfered with in some way; if you look in your IPS/firewall logs, the hostname that you should see would be b5n.1password.com, on port 443. That connection is what tells 1Password that there is new data, so when you get a free moment (and your son does too), you may want to see if that's being blocked by something in his network environment. Hope that's helpful. :)

  • BLD
    BLD
    Community Member

    @Lars Lol, if we were sitting in a room, I bet we'd agree more than it sounds than it sounds in text which loses so much. It's good to know that a 1P client lock will force a sync. I wasn't aware of that, thanks.

    Yes, of course, 1P can't possibly anticipate every usage scenario. But if the server or a client tries to push a change, I hope there is a handshake involved that the change was accepted and processed. And if that handshake fails, whoever was trying to deliver the message schedules an event to reattempt, either at a fixed interval of time, or when triggered, such as by a new connection. The way you described it -- it sounds like 1P server or client sends its changes but doesn't ensure they were received. I hope I've got that wrong -- because that wouldn't be defensive coding. As an engineer myself, I sympathize all too well that far more effort goes into covering error scenarios than the actual features every developer would rather be working on.

    Again, I can't stress enough, that from an end user's perspective, even if an intermittent spotty connection is the root problem, when 1P clients are out of sync looks like a 1P bug and a very bad and scary one. If a 1P client can't deliver its changes to the server after a reasonable number of reattempts, etc., it should alert the user. It should not leave the user in a state where they believe the changes have been made, when they really have not been. This is nothing unique to 1P, this is a classic problem with cloud-based computing. And the sharing across clients is a pretty fundamental feature of 1P -- so I really hope it gets serious architectural attention.

  • Lars
    Lars
    1Password Alumni

    @BLD - you're not wrong: it WILL feel like a bug to end users -- or at least (if they're not conversant in computer conventions/nerdspeak) just something that doesn't work correctly all the time and therefore is only partially trustworthy. And that's not a great reputation for a security product to acquire, even if it is only one person at a time. I've passed along your thoughts to the 1Password for Mac development team, and while I'd be straight-up fibbing if I told you I had any idea what the outcome would be, I think it's worth re-evaluating. We actually do quite a bit of that around here, much more than it probably seems from outside as a user: the landscape is always changing, and so it would be tantamount to malpractice for us NOT to do so. But we aren't always thinking of everything and we don't always get it right on the first try, which is why feedback from users - both knowledgeable and less so - is SO valuable to us. Thanks again, very seriously, for being willing to take a part of your own time and knowledge - not to mention your experience as an end-user - to share with us.

  • BLD
    BLD
    Community Member

    @Lars You and your cohorts are responsive and passionate about your product. As long as you remain that way, I'll be your enthusiastic customer. All software has problems -- and as I've said before, the bulk of what you've done so far is simply great -- I was sold within hours. So in that vein take my absurd volume of comments as wanting it to be successful. When I finally get all my "users" transitioned and in a stable usage pattern, I'll stop spewing noise all over your forum. :-)

  • Lars
    Lars
    1Password Alumni

    :) :+1:

  • Good morning @BLD ! Thanks for keep up such a great dialog. We (the dev team) have been chatting about this one this morning. We came up with a pile of questions I was hoping you could answer for us to try and lock down where things are going sideways for you.

    • Is this at work or at home? (is it a corporate environment setup issue, proxy servers, vpns, firewalls, network configurations)
    • If at home, do you have any special software you have to run as part of using these devices for work? (in an attempt to capture proxy servers, VPNs, etc)
    • Are you using any antivirus or security software like firewalls, VPNs, proxy servers, system-wide ad blockers, network-wide ad blockers, etc?
    • Does sync actually work if you lock and unlock?

    //cc @ag_kevin @AGKyle

  • BLD
    BLD
    Community Member

    @MrRooni Unfortunately, I have not been in rigorous data gathering mode when this has happened (a few times now) -- but I will attempt to get you something resembling reproducible the next time this occurs. I fully understand trying to troubleshoot issues like this is extremely difficult. That said, here are a few approximate observations from memory, so take them with that grain of salt:

    • I have observed this between two 1P instances even on the same home network -- all machines on the same physical Ethernet with a single simple router, e.g., saving an item on a Mac and not seeing it appear on another without explicit action to force a sync. The case that caused my son grief was from his Mac laptop to his iPhone within the same wireless network at his college within his dorm room (same WiFi hotspot, but of course I have very little idea of the network topology there). So, that leads me to empirically believe network issues are not at play here, but of course I can't be certain.
    • On my home network at least, I am not using any proxies or network software such as a VPN when I've seen this happen. So, it should be the simplest network path 1P can hope for: client1 -> router -> Internet -> 1P server -> Internet -> router -> client2. I can't speak to my son's college network, but again he had the problem between two devices on the same WiFi hotspot with no other indications of network issues on either device.
    • I run a browser based ad blockers as well as Pihole for DNS-level ad blocking on my home network, but I am pretty sure I have seen this happen when I had Pihole turned off. Each Mac and the router are running the standard firewalls built into OS X and the router (an Airport Extreme in my case). Again, I can't speak to what my son's college is doing on their network.
    • I think sync may work if I lock and unlock, but in my son's case, it seems we had to quit 1P entirely. I can't be sure at this point.

    Sorry to not be able to give you more specific data yet.

  • Thanks so much for all the detail, @BLD. We're going to attempt to recreate these issues here as well.

  • AGKyle
    AGKyle
    1Password Alumni

    @BLD

    Thanks for the information. Something that stuck out to me was the Pihole here, so I'm wondering if maybe it's doing something to make this interesting enough sometimes.

    I'm curious if you can take a look through the logs on your Pihole when this happens or see if you can determine if the following URL is ever blocked:

    wss://b5n.1password.com/

    It'll probably look a bit like gibberish at the end, you'll have something like: wss://b5n.1password.com/<account uuid>/<user uuid>/<device uuid>

    I'm guessing it's a bit of a long shot here, though other tools that may try to interpret URLs as being malicious could possibly see the UUIDs in the URL as strange, which could cause blocking as well.

    At this point I'm hoping we can try to find a way to actually rule Pihole out entirely with sufficient evidence that it's not the problem. It'll be one less thing on the list.

    In the few reports we've seen of this the logs do not indicate there being a problem, so it's almost as though traffic is being lost in transit.

  • BLD
    BLD
    Community Member

    I saw no trace of any URLs even partially matching what you describe in my Pihole logs, as we suspected.

    Just to reemphasize what I said earlier, while you can't predict when the connection between the two endpoints will fail or how to mitigate all types of failure modes -- you can certainly detect when failures occur and keep trying or display an error. Failure to complete a sync with no user visible alert at the very least is most certainly a bug that you need to address, and without seeing your codebase, I know should be do-able even without knowing why the transmission failed (you don't declare success until the recipient acknowledges your message, and this wait for ack must of course be done asynchronously for the best user experience, i.e., responsive to other events during slow and intermittent network connections while still allowing for slow syncs to make progress).

  • AGKyle
    AGKyle
    1Password Alumni

    @BLD

    Thanks for that information about pihole. Lengthy response incoming so you can see things from my perspective, which I hope will make a lot more sense of how I'm approaching this issue.

    In this case, what we are seeing is that the web socket connection between the app and the notifier is staying open. We're seeing no disconnections or anything take place. This is why I suspected that Pihole wasn't the cause, as I imagine it would outright block the original connection. So far as the app is concerned it seems things are "fine."

    With that in mind, the questions I need to answer are basically:

    1. Is there a bug preventing the server from sending a message to the clients to sync?
    2. Is there a bug in the client apps that prevent it from processing the request to sync?
    3. Is there something blocking the message from the server to the clients such that the client doesn't know it needs to sync?

    Number 1 is possible, but I think we'd see more widespread issues here. We did make some changes to the notifier server recently to try to reduce the load on our servers so it's entirely possible in some small way we've made a mistake there. I'm not going to rule that out but we'd also see this on Android and Windows devices if this were the case... so far, at least that I've seen, it's limited to Mac and iOS devices I don't want to go down this route yet.

    Number 2 is far more plausible but I'd also expect it to be more widespread... and on top of that we haven't made any direct code changes to this I don't think recently, which means either this is a long standing bug or one that has somehow been impacted by moving of code. We have been doing a fair bit of moving code around in 1Password 7. Certainly not unlikely but again if something is broken like this I'd think we'd see more widespread issues because something breaking in this scenario is outright breaking every time, not failing after some period of time.

    Given that this is happening to a fairly small number of people I'm less inclined to believe it's anything other than 3 at the moment, but that's why I'm trying to rule these things out. In the past, 9 times out of 10 these issues are almost certainly client side on user's devices and due to other tools causing havoc. That doesn't mean this is guaranteed to be that, but my experience troubleshooting issues tells me it's where we should start because if we assume it's a bug on our side instead and we have no way to recreate it we're going to go down a rabbit hole that we never get out of chasing a bug that doesn't exist.

    Regarding displaying sync errors. This is tricky, we want the experience to be seamless for users. We don't even call it "sync" anywhere for good reason. 1Password.com Accounts aren't simply a sync solution so saying it's a sync solution, or implying it, is kind of disingenuous and starts making comparisons to other sync solutions which aren't even in the same ballpark with regard to features and power. We really want to avoid that and would much rather find a solution that doesn't require showing the user anything. I'll happily agree that silently failing is bad as well, but this isn't as easy as showing an error I don't think. At least from our perspective, I'll also concede that it seems easiest to you but that conflicts with our desires to hide complexity and provide the type of user experience we are aiming for.

    If you're willing I'd love to get a diagnostics report from one of your machines while the issue is happening. Instructions here:

    https://support.1password.com/diagnostics/

    Once you've sent the report in, you'll get an email response with a ticket ID in it. Please simply respond back here with the ticket ID and we'll be able to find it in our support system. I can then take a look at the report and see what I can determine, if anything.

    Also, can you give some basic idea of network configuration where you're encountering this? Router make and model, any particular networking hardware that may exist in between your devices and the internet. That type of thing. I'm not looking for a full topology of your network or anything but like Pihole, application firewalls and other software could potentially be too aggressive and see some requests as potentially malicious and block them. Right now this information will mostly be asked for everyone so we can see if there's any consistency with what we're seeing there. Is everyone using the same brand router? If so that's a possible clue.

  • BLD
    BLD
    Community Member

    @AGKyle If I can catch this live, yes, I will certainly grab diagnostics. It's interesting to learn that you have only heard so far about incidents of this on MacOS / iOS devices.

    I understand the design goal not to display any errors, but I'm glad you agree that silently failing altogether is the worst failure mode.

    I'm not sure why you're loathe to avoid the term "sync" -- I consider that one of the key strengths of the product.

    If I understood you correctly, your best guess right now is that client1 makes an update that the server successfully receives and that the server then pushes down the web socket with client2. client2 for whatever reason does not process the message from the server and no indication is made on the web socket (connection dropped, etc.).

    I think at a minimum you can avoid the worst case failure mode and make the client that's out of date aware. My hope is that there is application layer logic as well guaranteeing the delivery of data, in other words, the server will know through lack of response from the client that the data was not processed to completion. The question is what should the server do about it -- and I agree, that's a tough call. Broadcast to all clients that an error occurred? Multicast only to the original sender(s)? And/or forcefully drop the web socket with client2, so that when client2 successfully reattaches it will know to receive a stream of errors from the server to surface to the UI unless it can't fully sync?

    As far as addressing the root problem -- why is the data not making it down the web socket in the first place -- that's where I agree the problem becomes quickly like shooting in the dark. As far as networking configuration goes, I already gave you in my earlier post the high level view. Within my home network, I'm pretty sure I've seen this with one client, OSX Mojave, connected wired to an Apple Airport through a Netgear unmanaged switch and another client (OS X or iOS, can't remember which at the time) connected wirelessly to the same router. In my son's case, he had an iOS device and an OS X device both connected wirelessly to the same WiFi hotspot with no other indications on either client of any kind of network issue.

  • AGKyle
    AGKyle
    1Password Alumni

    Hi @BLD

    We have another customer contacting us via email in our support system who may have helped us discover what is causing this. Thing is, I'm not familiar enough with the Apple Airport router to determine how to possibly test it.

    What we've discovered is that some routers/firewalls will close out connections when there's no activity on the connection for a period of time. What may be happening is that the router/firewall is destroying the routing table information to map one to the other but our client may simply keep the connection open and never close it out, thus never get a reconnection.

    I'm not sure how familiar you are with the Apple Airport routers but it may be worth looking through the settings to see if there's anything related to keep alive type of settings that you can change.

    I'm using a Ubiquiti EdgeMax router and am not experiencing this issue, so I suspect there's some setting somewhere that some routers have enabled that others may not that is causing this to happen for some users but not the others. Assuming we're on the right path here of course.

  • BLD
    BLD
    Community Member
    edited February 2019

    I'm extremely familiar with Airport routers (pun intended) and no setting like that rings a bell. But I'll investigate.

    However, recall my son experienced this between an iOS and MacOS device on his school's WiFi. While I don't know for a fact, I find it quite unlikely they're using routers not supported by Apple anymore.

    Update: I see no settings related to KeepAlive in the router's configuration, nor do I see anything suspicious in the logs for it.

    But @MrRooni I have to say that even if what you describe is happening, one end or the other of the web socket should be getting a timeout at least. I still stand by my assertion that regardless of what is causing a network transmission error 1P should be aware that data was lost in transit. And I still think it may not be a network issue at all -- you may have a bug where data received is dropped by the application itself, or never sent in the first place.

  • Hi @BLD,

    It's not so much as "Apple compatible routers", routers silently dropping websocket connections. Some routers do this if there's no activity on the socket for an amount of time. The client and server are not notified; messages just aren't passed through any more. We are looking into improvements to help keep the socket alive. So this may solve your issue.

    While there's always a possibility of a bug in the client, from looking into the code where this happens, I can't see it only happening for some customers and not all. And it's definitely only an issue for some customers. We can/will look into putting additional diagnostic information in that section of the code, so that if it is a failure there, it'll log it, and show up in the diagnostic reports.

    Cheers,
    Kevin

    ref: apple-3115

This discussion has been closed.