An AI Vibe Coding Horror Story

direwolf20

Some people only care about actual consequences. Download all the data and send it, in the post on a flash drive, to the GDPR regulator's office and another copy to the medical licensing board because why not.

websap

Do you think if the agency hired a consultant to build this , a consultant couldn’t have made the same mistakes? Lack of security theater is a good thing for most businesses

delis-thumbs-7e

Meanwhile on Linkedin… Every sales bozo with zero technical understanding is screaming top of their virtual lungs that evrything must be done with AI and it is solution to every layoff, economic problem, everything. It is just a matter of time when something really really bad happens.

andai

Archived version: https://archive.ph/GsLvt https://web.archive.org/web/20260331184500/https://www.tobru...

mikojan

Hard to believe... This activity should certainly land you in a German prison?!

spaniard89277

I did something similar to a local company here in Spain. Not medical, but a small insurance company. Believe it or not, yes, they vibecoded their CRM. I sent them an email and they threatened to sue me. I was a bit in shock from such dumb response, but I guess some people only learn the hard way, so I filed a report to the AEPD (Data protection agency in Spain) for starters, known to be brutal. I've also sent them a burofax demanding the removal of my data on their systems just last friday.

peyton

Kinda crazy but hopefully the future holds a Clippy-esque thing for people who don’t know to set up CI, checkpoints, reviews, environments, etc. that just takes care of all that. It sorta should do this anyway given that the user intent probably wasn’t to dump everyone’s data into Firebase or whatever. I personally would like this as well since it gets tiring specifying all the guardrails and double-checking myself. Using this stuff feels too much like developing a skill I shouldn’t need while not focusing on real user problems.

consumer451

What would a responsible on-boarding flow for all of these tools look like? > Welcome to VibeToolX. > By pressing Confirm you accept all responsibility for user data stewardship as regulated in every country where your users reside. Would that be scary enough to nudge some risk analysis on the user's part? I am sure that would drop adoption by a lot, so I don't see it happening voluntarily.

repeekad

A perfect example of why a product like Medplum exists, as opposed to completely reinventing the wheel from scratch

BrissyCoder

This reads like internet fiction to me. Very vague and short.

agos

I really hope OP also contacted their relevant national privacy authority, this is a giant violation

zoobab

Avoid javascript like plague, it can be overwritten at the client side.

krater23

The only thing what helps is deleting the database. Every day. Until the thing goes down because the 'developer' thinks he has a bug that he can't find.

EdNutting

Software engineering is looking more and more like it needs a professional body in each country, and accreditation and standards. Ie it needs to grow up and become like every other strand of engineering. Gone should be the days of “I taught myself so now I can [design software in a professional setting / design a bridge in a professional setting].” I’m not advocating gatekeeping - if you want to build a small bridge at the end of your garden for personal use, go for it. If you want to build a bridge in your local town over a river, you’re gonna need professional accreditation. Same should be true for software engineering now.

ionwake

Anyone else read the title on HN and shudder not wanting to actually click it?

GistNoesis

Who should get jailed ? Does the company which willingly sells the polymorphic virus editor bear any responsibility, or should the unaware vibe coder be incumbent ?

faangguyindia

It's nothing new, dunning kruger existing long before AI entered coding realm. Several years ago ran into one american company which consulted with me. They had 4000 paying customers and they rolled out their billing solution which accept crypto, paypal and stripe. They had problem with payment going missing, i migrated them to WHMCs with hardening and they never had any issues after. Now people may laugh at whmcs but use the right tool for job U need battle tested billing solution then whmcs does count it can support VAT, taxes, reporting/accounting and pretty all which you'll error while you try to do it all yourself. Too bad there aren't battle tested opensource solution for this

jillesvangurp

I think the issue here is less about AI misbehaving and more about people doing things they should not be doing without thinking too hard about the consequences. There are going to be a lot of accidents like this because it's just really easy to do. And some people are inevitably going to do silly things. But it's not that different from people doing stupid things with Visual Basic back in the day. Or responding to friendly worded emails with the subject "I love you". Putting CDs/USB drives in work PCs with viruses, worms, etc. That's what people do when you give the useful tools with sharp edges.

aledevv

> All "access control" logic lived in the JavaScript on the client side, meaning the data was literally one command away from anyone who looked This is the top! This is a typical example of someone using Coding Agents without being a developer: AI that isn't used knowingly can be a huge risk if you don't know what you're doing. AI used for professional purposes (not experiments) should NOT be used haphazardly. And this also opens up a serious liability issue: the developer has the perception of being exempt from responsibility and this also leads to enormous risks for the business.

sjamaan

So much is missing from this story. Did they report it to the relevant data authority? Did the fix they said they applied actually fix anything? Etc.