Problems with Syncing, does not maintain A.C.I.D. properties!
Request for change.
Problems with Syncing, does not maintain A.C.I.D. properties! Can the items described below be considered?
1Password's Syncing algorithm is faulty. Let me explain:
- Syncing does not guarantee A.C.I.D. = (Atomicity, Consistency, Isolation, Durability).
1a. Consistency and Isolation are two attributes that the Syncing protocol violates.
1b. Consistency is the property that guarantees that an update to the data will maintain the data so that multiple updates will keep the data "Valid." This property means that updates will be executed serially. This implies that update at time "T" will be displayed to user for update at time "T+1." Syncing just assumes that the latest update to the data is the valid data regardless of the time "T" update. If I update 1Password at time "T," with information on a specific item "A" on my cell phone, and then update 1Password with information at time "T+1," on the same item "A" on my laptop with different information, since the information which I update item "A" at time "T+1", is the latest, the information which I updated item "A" at time "T" is lost. The data at time "T" is important, the data at time "T+1" is important. Syncing violates the validity of data at time "T" by ignoring it since it is older than data at time "T+1."
1c. Isolation is the property that forces data that is being updated simultaneously to be forced to be executed serially. That is, item "A" is being updated by two updaters at time "T." Isolation forces one updater to execute at time "T," and the other updater is delayed. The delayed updater is then presented the changed data from the updater at time "T," and then updates the time "T" data at time "T+1." The update at time "T" is not lost. In fact, the time "T+1" update caused the person updating data "A" to see the time "T" updates, and decides what to keep and what to change! Again, Syncing violates this property by loosing the time "T" update. The "simultaneous update" property is an issue even without "simultaneous update!" Why? Since a "distributed database system" is used, "T" updates must be replicated across all distributed nodes of the DB system. This takes time. However, as this action occurs, a "T+1" update may be made. If the "T" data is not updated on the node that the "T+1" data reads for update, loss of update "T" will occur on the node in which "T+1" is updated. The "T+1" data will then be updated, and at some time in the future, will be replicated across the nodes of the DB.
1d. Syncing's purpose is to allow fast updates across a "distributed database!" In other words, the entire database is spread across multiple servers. Syncing can be valid and has been shown to be so in certain situations. However, the algorithm becomes invalid in the situation I described in "1b" and "1c." This is unacceptable and has caused me loss of 1Password updates on multiple occasions! There is a solution described in "1e" following!
1e. Syncing requires a "tie breaker" to determine what is the Valid data for Consistency and Isolation. This "tie breaker" can not be "Validly" programmed in the Syncing method. The "tie breaker" must be a human. Online transaction processing systems, which must guarantee A.C.I.D., for example financial transactions, inventory control systems, NASA launch control and mission control systems, etc., are required to maintain "A.C.I.D.," will typically offer to suffer time queues to maintain "A.C.I.D.." The time quests slow updates, but force A.C.I.D. Syncing's purpose is "Speed" across a "Distributed Data Base," not "A.C.I.D.."
So here is the solution: keep both, or all updates, for "A" at time "T" thru time "T+ N (any number of updates), and let the human user review the data and edit it as appropriate. Syncing's purpose can still be maintained. I.E., rapid update to data "A" across a distributed database. Yes, this will cause an update at time "T+N+1" which will be spread across the nodes of the distributed database. However, the user will now have data for time "T," "T+1", all of the updates for the set "T+N" and the update at "T+N+1." Since the data for time "T+N+1" is the correct data, the user can simply delete the other "T's" data! Voila, the user's interaction with the multiple updates simulates the properties of "A.C.I.D.."
There is a problem, however, with Syncing! The "tie breaker" is often programmed into the database servers upon which the "Syncing" data resides. If this is the case, then as the 101st Airborne Division, Battle of the Bulge, (1 star) Brigadier General McAuliffe said to the German General von Lüttwitz's representatives who requested General McAuliffe to surrender, since the 101st was surrounded, General McAuliffe replied NUTS. The Germans did not understand the meaning of NUTS. When asked, Col. Joseph Harper explained: "In plain English? Go to .....!" Gen. McAuliffe used NUTS, because he did not use profanity.
1f. The issues described above in "1b" and "1c" are a severe problem for me. I use 1Password constantly. I have used 1Password since at least 3.8.2. I find 1Password to be a fantastic password tool. However, the multiple platform syncing issue should be reviewed!
tim b
May the blessing of Jesus Christ abound to you and yours! Romans 13.
1Password Version: OSX 5.3.2, Android 4.5
Extension Version: Not Provided
OS Version: OSX 10.10.4, Android 5.0.1
Sync Type: Dropbox
Comments
-
Hi @tim_b,
It would seem you're looking for an implementation where 1Password communicates with a database server for the purposes of synchronisation. While 1Password would have some code regarding this unless I'm mistaken it's up to the server to ensure certain aspects of this are correct.
1Password doesn't work like this though, we leverage Dropbox and iCloud and so all we do is write to a local copy sync container, a simple file structure and ensuring everything is synchronised is up to the service. Information shouldn't simply disappear though as Dropbox for example creates conflict files and we store conflicted information in custom sections when we merge. This allows the user to manually resolve the conflicts as we find them.
So what you're looking for isn't a minor adjustment or, at least in my opinion, a bug in the code. It sounds like you're after a more robust syncing mechanism that I would say Dropbox would offer. I could be mistaken so please do post back if that's the case.
0 -
Hi littlebobbytables,
Thank you for the feedback.
Yes, this is what I thought. I mentioned this in the next to the last paragraph: "There is a problem, however, with Syncing! The "tie breaker" is often programmed into the database servers upon which the "Syncing" data resides. If this is the case ... NUTS. "
If the "tie breaker" is in the "Syncing" provider, control is out of your hands!
Yes, I understand my suggestion would require a more robust syncing mechanism. Syncing is a protocol, and I doubt that Dropbox, or any Syncing Provider, would want to develop their own protocol. My solution in "1e." would present inconvenience to users, and probably confusion.
The syncing problem I describe has bitten my 1Password data on many occasions. I do not carry my laptop with me everywhere. My cell phone is an appendage to my anatomy. However, with that said, my laptop is my work center. The cell phone is a handy tool for on the fly update to 1Password. The time "T" and "T+1" updates to item "A" are adhoc and therefor "T" and "T+1", and their resident platforms, are unpredictable! The laptop takes precedence, but my cell phone has timely availability.
Thank you,
tim bMay the blessing of Jesus Christ abound to you and yours! Romans 13.
0 -
If the "tie breaker" is in the "Syncing" provider, control is out of your hands!
@tim_b: While it will vary to some degree depending on the sync service, as lil bobby mentioned, this is what the "conflicted copy" files are for; nothing is thrown out, and the user can decide for themselves if the sync service — Dropbox in this example — has chosen the "correct" canonical copy. This is one reason I prefer Dropbox, personally: the user still has the control.
In the end, 1Password isn't data synchronization software, so architecting a sync mechanism is outside of its scope; but we can certainly consider other services and options going forward, especially if current solutions prove to be limiting or unsatisfactory for both us and our customers.
Thanks so much for your detailed (and passionate!) feedback! While this isn't currently something that AgileBits does, we'll never say never, and should we choose to go in that direction in the future, it's important to know what you and the rest of our awesome customers want. Cheers! :)
0 -
Hello again lil bobby, and brenty.
Brenty, you cleared up an area of confusion on my part, and caused me to re-read lil bobby's reply. Dropbox's "conflicted copy" did not at first sink into my hard headed brain helmet. My wife and everyone I know says I am hard headed, so now we see I am.
Off that, and back on track. Dropbox's "conflict" link (https://www.dropbox.com/help/36) helped me to understand what is available. Therefor Dropbox provides both a method for forcing "Consistency" and "Isolation," and therefor forcing serial application of DB update (maybe). Dropbox does provide a "Dropbox for Business provides a Service Organization Controls 3 (SOC 3) assurance report" at (https://cert.webtrust.org/pdfs/soc3_dropbox.pdf). This is the SOC for the "Business" product. There is still a question in my mind wether this method provides "READ" integrity. It appears that although it would force serial update, it may not force a read of the new updated data for the queued user. If this does not happen, there would still be a inconsistency and therefor a conflict in the data.
Back to Dropbox's "conflict" link information:
_"How can I prevent conflicted copies from happening in the future?
One workaround that can be used to prevent conflicted copies is to move the file out of the Dropbox folder while editing it. Doing this causes the file to appear as deleted and doesn't allow other users to edit the file. Once you're done editing, you can move the file back into its original location in the Dropbox folder. It will reappear with your edits for all users who have access to that file.
Another option is to move the file into a second folder designated for "In Use" files. Other members of the shared folder can still access it if they need to, but will know not to make changes."_
This is inconvenient, but effective. However, this is one way AgileBits could force "C.I." compliance into update protocol by forcing the movement of the "actual folder db" to an "update folder db." However, you would need to effect the handling of:
1. Creating an "Update Folder" and the "Actual Folder."
2. Moving the Container to the update folder. However, if 1Password is designed to multi-task, this would be multi-task bottleneck requiring queuing between tasks. This would probably cause user's question regarding "Why is 1Password so slow." I have some interesting customer support stories about "slowdown must-tasking" issues.However, a second solution would be to take the "non conflict" container, and the "conflict" container, and allow process similar to my "1e" solution. Keeping in mind user desires and the way they utilize 1Password on multiple platforms. This should be a "User Option!" Describing this option would definitely take an excellent wordsmith. Also, it may not be worth your while.
Another "however," if READ consistency or integrity is maintained, then the "1e." solution is unnecessary. However, based on experience this is not so.
I am only one user. Development is expensive. Errors in complex code tend to the side of one error per twenty to one hundred lines of code. This is not something to ignore or take lightly. I realize my request may be very low priority. I wish it were high, but 1Password is your business.
Thank you all very much for you insightful and kind replies.
The best to y'all,
tim bMay the blessings of Jesus Christ about to you and yours!
0 -
My wife and everyone I know says I am hard headed, so now we see I am.
@tim_b: You and me both!. :lol:
There is still a question in my mind wether this method provides "READ" integrity. It appears that although it would force serial update, it may not force a read of the new updated data for the queued user. If this does not happen, there would still be a inconsistency and therefor a conflict in the data.
Good point! I'm afraid I'm not familiar enough with their Business product to say one way or the other, but it's certainly worth pursuing with Dropbox.
Another option is to move the file into a second folder designated for "In Use" files. Other members of the shared folder can still access it if they need to, but will know not to make changes."_
It's certainly an interesting idea, and one of many we can consider if we decide to take on sync management ourselves in the future. As is stands, 1Password is a local app with local data first and foremost, and then users can choose to sync if they wish.
- Moving the Container to the update folder. However, if 1Password is designed to multi-task, this would be multi-task bottleneck requiring queuing between tasks. This would probably cause user's question regarding "Why is 1Password so slow." I have some interesting customer support stories about "slowdown must-tasking" issues.
Indeed! One concern with this approach is that, while modern computers are certainly capable of eating this overhead, mobile devices are less accommodating in this regard — both due to sheer dearth of resources, and the fact that we want to minimize impact on responsiveness and battery life, even if there are adequate hardware resources.
I am only one user. Development is expensive. Errors in complex code tend to the side of one error per twenty to one hundred lines of code. This is not something to ignore or take lightly. I realize my request may be very low priority. I wish it were high, but 1Password is your business.
Absolutely! These are all excellent points, but ultimately we're here doing what we do only because of you and the rest of our awesome customers; and so we depend on thoughtful suggestions like you've given to help us steer development in the future. While I can't say for certain that we'll embark on the path you've laid out in such detail, it's feedback like this that is often foremost in our minds as we do make these decisions.
Thank you all very much for you insightful and kind replies.
You're most welcome, Tim! It means so much to my teammates and me that we have such passionate users, and that we're able to make a living doing what we love as a result. This is truly a gift. Thank you for your support! :chuffed:
0
