Apple's AI Can Now Change Your Passwords. What Could Possibly Go Wrong?

drob518

Yea, I saw that during the WWDC keynote and physically cringed. As the article says, what could go wrong?

dewey

There's this standard that is being worked on by the people working on the Passwords app at Apple (They are active on Mastodon, and often talking about that) which will probably be helpful for this feature too: https://www.w3.org/TR/change-password-url/

ThejaCH

I mean isn't it either to complex to implement or not a good implementation kind off thing? A good chunk of people do use devices other than apple eco system one's and if they try to login and then suddenly, you can't!

TechRemarker

Yes, also immediately thought of all the endless ways this could go wrong and end with someone losing access to their account, which depending on their account could be trivial or life altering, especially if their loss ends up being someone else's gain. Apple takes baby steps so I'm sure this will be limited in nature and most likely will get delayed until fully tested, but I'd definitely avoid testing during betas (with any real accounts that is).

dotcoma

Can it be turned off ?

throwaway85825

People already have a hard time remembering passwords without them being automatically changed.

AshamedCaptain

Call me when it can _delete the account_ from all those websites, which is likely the primary reason the user has not updated the password yet.

Animats

Back in 1984, I wrote the original "obvious password detector".[1] It just checks whether a password has English language trigram stats. This prevents dictionary attacks. Everything is so much more complicated now. [1] https://www.animats.com/source/obvious/obvious.c

vablings

This could have nuclear level consequences. Imagine somehow your keychain is compromised. Using a change password URL means an attacker could literally lock you out of every account at the same time

micromacrofoot

I already let 1Password generate all my passwords, so as long as they're just invoking tools with AI rather than having it attempt manually, it doesn't seem like such a big deal?

pokstad

I’ve had the iOS password app think that it changed my password, when it did not, and then lose my old password.

hmokiguess

https://xkcd.com/2044/

zerobees

This article appears to be 100% AI. I guess there's some irony that a company ships an AI feature and someone else uses AI to come up with criticisms of that feature. But the article... doesn't actually say anything? It's just full of weird, generic short-sentence LLMisms ("Detection is observation.", "Changing the password is authority.", "The security benefit is real.", "That is a meaningful improvement.", "This is not just text generation. It is an agent taking action with a sensitive credential.", ...). It doesn't offer any insights into the actual architecture that Apple came up with, whatever it might be. It doesn't propose a better design, other than a bunch of super-generic things that apply to every single software project ever ("The system should verify the exact website and account before filling or changing anything.", "This feature deserves focused adversarial testing during the beta period."). So... it's upvoted just because the title mentions Apple and AI?

doodlebugging

I wonder whether the AI generated password that you allow to be created on your iPhone in the Passwords app can be recovered and added to whatever password manager you might be using on Windows or Linux desktop. It seems like this is a great way to lock oneself out of access to an account on some of the devices that they own that do not have access to the Passwords data storage. I can see where this can be a benefit in helping users secure their accounts with stronger passwords but I think that there is a lot of potential for this to become a real problem.

nikisweeting

Very curious if they're implementing browser driving themselves or using an off-the-shelf library like stagehand, browser-use, etc. to drive the DOM. Hopefully they open source it if it's in Swift. A11y-tree alone is not enough for many sites because lots of auth stuff happens in OOPIF frames that need special handling/stitching/interactive element filtering. There's also the issues of many captchas around auth stuff being implemented using canvas elements (that are hard to instrument for browser agents without relying on CUA). Can their on-device 3B model really handle accurate CUA driving? I guess we'll see...

flyingshelf

This is a great article except the "That can happen for plenty of boring reasons" list. Almost each of those reasons is completely unrelated to AI and can happen even if you attempt the change 100% manually with or without a password manager.