Data rot... Scan and identify defunct URLs

Ocean West
Ocean West
Community Member

There seems to be a lot of long gone sites that had been part of the internet bubble that has been acquired or shuddered, it would be nice if there was some tool that could scan urls and - perhaps compare that with other DNS services to determine if the site is truly defunct.


1Password Version: Not Provided
Extension Version: Not Provided
OS Version: Not Provided
Sync Type: Not Provided

Comments

  • AGAlumB
    AGAlumB
    1Password Alumni

    @Ocean West: Hmm. That's a good point. I'm not sure that's something 1Password should do though. I understand the impulse of course. But there are two approaches I can see, and both are not good:

    1. Have the app itself go through and try connecting to websites. This is bad because that can take a lot of time and waste resources. We've done something similar, connecting to websites to try to get favicons in the past. That was good because it was less work for us, and offered privacy to users. But it really bogged down the app and system when this was happening, and overall is a pretty inefficient way of solving this problem,
    2. Have our own server handle this. It could certainly do this more efficiently, since it could aggregate — e.g. contact someoldbrokenwebsite.com once for all users...but then we'd be in a position to know which websites our users have logins for. Some other password managers collect and sell that kind of data, but even though we can easily say we don't want to do that, we also don't want to be in a position where we could if we turned evil or transferred ownership.

    So while I think it's an interesting idea, I think it is probably best that it's not something 1Password does, so we can continue focusing on security and privacy. I hope this helps. Be sure to let me know if you have any other questions! :)

  • wkleem
    wkleem
    Community Member
    edited March 2018

    I emphasise. I have been hunting for broken logins, some sites of which the server is now non existent and which I have forgotten about for years.

  • AGAlumB
    AGAlumB
    1Password Alumni

    Thanks for letting us know! :)

  • wkleem
    wkleem
    Community Member

    @brenty,

    Did not realise the wrong word was used until after you replied. Emphatise, not emphasise.

  • AGAlumB
    AGAlumB
    1Password Alumni

    No worries. I know what you mean. :)

  • Ocean West
    Ocean West
    Community Member

    why not leverage the same logic as https://blog.agilebits.com/2018/02/22/finding-pwned-passwords-with-1password/ and have an aggregate table of known defunct domains against ones that resolve you could do it server side efficently and not know the actual data.

  • AGAlumB
    AGAlumB
    1Password Alumni

    @Ocean West: I was thinking in that direction, but again, the question is how does this fit with password management? We'd have to do that work and maintain the service, but I'm not seeing the security benefit. And we have a lot of stuff that does to work on.

This discussion has been closed.