To protect your privacy: email us with billing or account questions instead of posting here.

German consumer report page tested 1Password

telephoneman
telephoneman
Community Member
edited September 2017 in Memberships

Hi Guys. the german consumer report institute ("Stiftung Warentest", Magazin of Oktober 2017) tested a bunch of Password Managers and Agilebits 1Password landed on 5th place after Dashlane, Intel Security, Keeper and Lastpass. They tested stuff like usablity, setup process and security. I don't know the others but I think they gave you an unfair rating in regards to your security concept - as far as I understood they said you don't have a 2factor auth. Which is for your web sync not true ... Futhermore they seems to have ignored the Tower-Features and security audit features. The security has 40% weight in test result. As this test reports have high awareness in German Market (Germany , Austria, Switzerland) you should check this report and call that magazine to correct at least this point, Currently I don't think the results for 1Password are correct.


1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Sync Type: Not Provided

Comments

  • @telephoneman: I saw some folks talking about this over my weekend and I believe our security team is aware of it and investigating. I can't tell you how delighted I am that you disagree with this article's assessment though. One of our primary security goals is to be transparent about our security practices and do our best to educate folks about how we keep your data safe so y'all can make up your own minds about security. That you've taken the time to learn about 1Password's security for yourself a draw your own conclusions puts a huge smile on my face. Thanks for being awesome and for letting us know more about the potential impact of this article. I've shared your post with the security team as well so they're aware of the sway it may hold. Much appreciated. :chuffed:

  • telephoneman
    telephoneman
    Community Member
    edited October 2017

    I mean there might always be some space for improvement - and always others might be better or equal in comparing tests - but here you got "minus points" ...

  • AlwaysSortaCurious
    AlwaysSortaCurious
    Community Member

    lol. There is always space for improvement (btw, I hated Dashlane, I trialed it). Anyway, when I was doing my research, most of the reviewers focused on usability or UI or perceived security improvement. They need a reviewers guide, not too dense, that they can easily download and review. In the bad old days before ubiquitous websites and content, a reporter or writer for a magazine would ask for these talking points to educate themselves, get the vendors perspective on their niche in the world and why their POV is the best. (and sometimes nail a product to a wall with them).

    The Two-factor thing would take the wind out of their sails on that issue. Keep the secret key of course, it just makes even dumb passwords so much better, but 2FA is on their editor's choice checklist. You can just position it differently than everyone else, Access control, not real security. And then you have two layers of access control,

    I don't think I've ever seen a technical discussion in a review, not at a "Tom's Hardware" or other sites where this is discussed. Not that LastPass hints at security audits and you guys post the teams and a summary of the results, Bet the reviewers are not aware that level of detail exists because they are on a deadline.

    1Password needs to educate The Tom's and PC Mag's of the world (proactively? These sites don't reach out and let you know it is happening) and then those other reviewers will use those sites as source as well (like the fact in the Elcom password cracking trial, 1Password was the slowest in giving up its brute force secrets).

    My two cents.

  • AGAlumB
    AGAlumB
    1Password Alumni

    I mean there might always be some space for improvement - and always others might be better or equal in comparing tests - but here you got "minus points" ...

    @telephoneman: We couldn't agree more. Certainly we see the ways in which 1Password needs improvement more than anyone. All I can say right now is that we've got some cool stuff in the works for future versions, and that it's really great to get constructive criticism from anyone. It helps us make 1Password even better. Cheers! :)

  • @AlwaysSortaCurious: We do try, but since nothing is ever truly deleted from the internet I've definitely seen some folks end up with outdated or incomplete info when writing reviews. Of course, when we see misinformation out there, we do try to reach out and work with reviewers and others to correct things and most are fairly receptive. There are always exceptions, but we are lucky to have an awesome security team that puts a lot of time and effort into documenting our security practices for experts and laymen alike, so if folks want to learn for themselves and form their own opinions, there are some awesome resources available. :+1:

    @telephoneman: To add to brenty's comments, our Chief Defender Against the Dark Arts has a saying I'm fond of, "Security is a process." The minute our security practices stop evolving and improving is the minute we're doing it wrong so we definitely aim to be improving every day, bit by bit. :chuffed:

  • AGAlumB
    AGAlumB
    1Password Alumni

    They need a reviewers guide, not too dense, that they can easily download and review. In the bad old days before ubiquitous websites and content, a reporter or writer for a magazine would ask for these talking points to educate themselves, get the vendors perspective on their niche in the world and why their POV is the best. (and sometimes nail a product to a wall with them).

    @AlwaysSortaCurious: It's an interesting idea. I'm not sure that it would be a good thing in practice though. Journalistically, if reviewers care about independence and integrity, one of the worst things they can do is rely on the info/criteria the creator of a product offers them to review said product. I know that's not what you're suggesting, but we do also put a lot of effort into representing 1Password honestly and trying to educate people about security in general instead of just saying 1Password = security. I think anyone should get info from as many sources as possible, if for no other reason than a different perspective. Customers aren't paid to do that though, so they're our primary audience when we write things.

    The Two-factor thing would take the wind out of their sails on that issue. Keep the secret key of course, it just makes even dumb passwords so much better, but 2FA is on their editor's choice checklist. You can just position it differently than everyone else, Access control, not real security. And then you have two layers of access control,

    Yeah, that's the tough thing. Certainly it's different enough from traditional multifactor authentication that it isn't a 1:1 comparison, and can therefore be confusing. But we care enough about this that our priority is security and not merely optics — a checkbox on a list, as you say. So we're willing to take some flak for decisions like this that we make for the right reasons.

    1Password needs to educate The Tom's and PC Mag's of the world (proactively? These sites don't reach out and let you know it is happening) and then those other reviewers will use those sites as source as well (like the fact in the Elcom password cracking trial, 1Password was the slowest in giving up its brute force secrets). My two cents.

    As I mentioned earlier, I'm not sure it's appropriate for us to try to actively educate people whose job it is to essentially judge us. I'm not sure they should listen if we tried.

    But security education is something we care about deeply. We do that already, but you'd be forgive for not noticing. We've got a really great team working on the main website, our support site, and our blog. And a big part of what we do here on the forums is in that vein as well, but on a more personal level. I love it, but when we're replying to someone else's question and you already know the answer it probably seems less educational. And conversely we have to be careful about making the content we publish (as opposed to replies to email and forum messages) accessible to a large cross-section of people.

    Everyone has a different level of familiarity with, and interest in, security and technology in general, so we focus on a dual strategy of general security information relevant to all 1Password users, and personal replies to questions from individuals. Not perfect, but we'll keep at it. :)

  • AlwaysSortaCurious
    AlwaysSortaCurious
    Community Member

    Reviewers guide is ok, not every technical journalist is an expert in everything or has time to dig deep. Giving them food for thought and describing your philosophy and the vocabulary, might just make them smarter....

  • AGAlumB
    AGAlumB
    1Password Alumni

    Maybe. I wish they'd read the white paper then. That covers pretty much everything about how 1Password works under the hood. A lot of other great info on our support site as well. I'm just not sure how we can know someone is thinking about writing about us and preemptively reach out unless they tell us. Sometimes they do, and we're always happy to point them to these resources and answer any questions they have. Always gives us a fresh perspective. Cheers! :)

  • telephoneman
    telephoneman
    Community Member
    edited October 2017

    @brenty im sure you always works on new stuff and improve constantly. But I don’t thinks it’s that bad and they updated also ratings in past when there was something not right

  • AlwaysSortaCurious
    AlwaysSortaCurious
    Community Member

    @brenty you make it part of the press kit. I read your white paper and those of the competitors. White papers are often seen as marketing drivel. They won’t waste their time. I searched security audit, and saw “we do them” vs yours “here is who did them and they found x number of flaws that were remediated.” You can increase the chances of them reading it by calling all the thumbnail documentation and audits you want them to read a reviewers guild and press kit, make it public to everyone. Heck, you can even do the unthinkable and snail mail it too so they know it exists.... lol, not in marketing, but I hate seeming missed opportunities,

  • Thanks for the feedback! There are definitely ways to improve the marketing on the security side of things. We'll keep them in mind. :)

  • gl0tzk0wski
    gl0tzk0wski
    Community Member
    edited December 2017

    As a german i would not overvalue "Stiftung Warentest". Their (technical) journalists made some bad decisions in past. Of course "Stiftung Warentest" is very famous in Germany, but e.g. the case/article about "Ritter Sport" (chocolate) shows, that they make mistakes and are not perfect. The trouble is only that the reputation of such companies suffers and not all companies like "Ritter Sport" are fighting for their rights. After reading the article, i was shaking my head, because in my opinion they made some mistakes, because they not dig enough about 2FA and 1Password. They are lot of article around about this subject. Writing "they are not experts" is not an excuse for me, because they should research well before they judge, especially "Stiftung Warentest".

  • :) :)

  • At the end of the day we can't expect everyone who looks at 1Password to fully educate themselves about everything. It's on us to communicate how we're better. I think there's a lot of room for improvement there.

    Rick

  • telephoneman
    telephoneman
    Community Member

    @gl0tzk0wski yes that’s why I said it’s not fair how they rated. And now there is a result announced to other non expert that this product isn’t that good. So an expert should tell them that they are not right ...

  • AGAlumB
    AGAlumB
    1Password Alumni

    Fair enough. :)

This discussion has been closed.