A candid discussion on WIFI sync architecture

Options
bryce_bryce
bryce_bryce
Community Member
in Mac

First of all, I LOVE using 1Password. It's an incredibly robust and ever interesting product. I read every changelog and blog post that AgileBits writes and am thankful for them!

One question that I find myself aggravated by of late is how sync is implemented over WIFI – a necessary feature for some who are less trusting on cloud-y days. The basis of this frustration is that 1Password does not seem to be aware of password changes across more than 2 devices (server-client). For all intents and purposes, let us agree that these devices are authorized to sync data and that I own and operate an unknown number of devices of various types. In that case, it would seem to be a suboptimal pattern to simply sync devices between a server and a client: these creds can get quickly out of sync and overwritten. It would seem better to maintain state in a separate database in which creds have been updated at a specific time, with a piece of metadata distributed over both servers and clients. A local broker service would consume published sync messages from the server and flush them to disk to the subscribed client only if the metadata indicates that the message has not yet been consumed by the client ("at least once" pub-sub pattern). Furthermore, such a mechanism could allow for cool features ('out of sync' devices, background notifications, etc.) because it could operate on the message metadata (less secure) rather than the data itself. These ideas are based highly on Apache Kafka, Avro, & Zookeeper.

Would love any feedback :)

1Password Version: Mac 5.4, iOS 6.1
Extension Version: 4.4.3.90
OS Version: OS X 10.11.1, iOS 9.1
Sync Type: WIFI

Comments

  • rickfillion
    rickfillion
    Options

    Hi @bryce_bryce,

    Thanks for writing in, I love it when users want to talk nuts and bolts like this. :)

    I have to admit that I'm not familiar with Kafka, Avro, nor Zookeeper. I just googled them and now have a rough idea of what they are, and how they relate to what you're saying. I think if we were to rebuild Wi-Fi sync today, it would look a fair bit different than what's currently implemented. That's just how things go as you learn more about the problem.

    The current Wi-Fi sync solution does work with more than 2 devices. You can have as many client devices as you'd like and they should all sync with each other fine. The server is also still free to sync with other devices via other means (AgileKeychain, OPVault, or CloudKit). If you're limiting yourself to Wi-Fi only syncing, it can be frustrating that there's a line that can't be crossed on the other side of the server though. If we allowed the Mac to be a client as well, it'd then be possible to make for more interesting configurations.

    What you're describing is actually fairly similar to how we implement iCloud/CloudKit sync. With CloudKit sync every device is a peer, and the CloudKit server is responsible for stopping a client from irresponsibly overwriting another change without first considering a merge. CloudKit also provides the "give me changes since X" mechanism, and since it ties into OS X and iOS push notifications we get background notifications and all that jazz.

    In our Wi-Fi Client/Server model, items shouldn't be getting overwritten as you mentioned. They should be getting properly merged should changes happen in multiple locations at once. That's one of the responsibilities of the server, to make sure that this happens. Personally, I'm still in the camp of having a server to help with sync in that way as opposed to leaving it all be truly peer to peer.

    If you have any specific questions about how we've implemented anything, let me know.

    Cheers.

    Rick

This discussion has been closed.