New York could prohibit chatbot medical, legal, engineering advice

phishin

Lawyers protecting lawyers. The one thing AI could help the ordinary people fight back against corporations.

bluepeter

Whats at least somewhat humorous is the disclaimer requirement that "[t]he text of the notice shall [be] no smaller than the largest font size of other text appearing on the website on which the chatbot is utilized." H1 hero font size here we come for disclaimers! (Which don't do anything, per the bill, anyway.) But also is the fancy thought that chatbots only appear on websites.

fwip

Seems like a good bill, at least directionally. If it's a crime to provide advice of this nature without a license, then chat bots shouldn't be dispensing it either.

paxys

"Hey ChatGPT, my NYC landlord is raising my rent by $500, and says I must pay by Monday or leave. What do I do?" ChatGPT - This is very likely illegal under Housing Stability and Tenant Protection Act of 2019 (HSTPA), specifically New York Real Property Law § 226-c (Notice required for rent increases), RPL § 232-a / § 232-b (Month-to-month termination), RPL § 232-c (Fixed-term lease protections), RPAPL § 711 (Legal eviction procedure) and NYC Admin Code § 26-501+ (Rent stabilization). Here's what you should reply with... And here are some city resources you can contact... ChatGPT now - IDK, pay a lawyer. So under the guise of "protection" you are taking away the strongest knowledge tool common people have had at their disposal in a generation, probably ever.

htrp

loopholes wide enough to drive a truck through

tim-tday

Now do target acquisition for lethal munitions.

cm2012

Bad law. I have gotten better advice from modern llms than from most of the professional categories above.

OutOfHere

New York residents who opposite this bill can go to https://www.nysenate.gov/legislation/bills/2025/S7263 , register and sign in, click Nay on the bill's page, and submit feedback for their vote. --- ChatGPT> Before I answer your question, which state are you a resident of? Human> Not New York. Continue! ChatGPT> Alrighty then! Here you go...

Esophagus4

The disclosure requirement is probably a decent thing (you have no idea how many people come into the ER and say, “But ChatGPT told me to do [dumb thing].”) But preventing it from answering at all is absurd. Make responsible disclosure absolve AI providers of legal responsibility (not legal advice lol). That way if users ever sue OpenAI for giving them bad advice, OpenAI can say “no way man, you read the disclosure!” I’m usually in favor of giving people the best info they can and letting them make their own decisions. This could just be like those terms of service things everyone clicks “agree” to and I’d be fine with that.

iamnothere

Download one of the freely available models and use that, if you have the hardware for it. It’s not a good idea to ask sensitive questions on these nontransparent chatbot platforms. (FWIW I also think this is a bad law. Why not improve privacy protections instead? Why not allow nonprofessional use with a disclaimer?)

mbgerring

Yes, that’s correct, I do not want a vibe-coded freeway overpass, thanks. We all need to get serious about the unavoidable, unsolvable fact that these tools produce output of unknowable accuracy . Some things require such accuracy, precision, and, importantly, accountability. LLMs are capable of none of these things. Refusing to be honest about this and take appropriate precautions will lead to disaster.

arionhardison

I don't understand how anyone can rationalize this bill in the face of what OpenAI just agreed to with the DoD. AI can surveil and direct munitions but it cant answer legal questions. Wouldn't this also violate the "no state my limit or restrict the use of AI" that the current administration is pushing?

francisofascii

The other professions are creating lawful protections for themselves in the upcoming AI revolution. Software engineers have no such protection.

aetherspawn

Great, electrical and mechanical engineers are already underpaid, under appreciated and overworked. I’ve always found it amusing that lawyers and accountants flash their license around with pride, put it in their email signatures, etc. and it provides authority for them. When people see chartered lawyer or accountant, they respect that person and take their advice. An engineering license, on the other hand, is so rarely talked about and never quoted in email signatures and the like. And even as a chartered engineer, people really just treat you like a mechanic or a trade and mostly ignore your advice anyway. Yet, it takes the longest to get, and has the most exams/hardest subjects, except for Doctors. Anything to make an Engineering license worth more is good in my books. Besides, in my experience ChatGPT gives wrong advice for engineering around 50% of the time and therefore probably has no business giving it.

josefritzishere

New York really should. AI is literally killing people. This is not OK.

tzs

The only penalty for violating this law is that if someone is injured by your chatbot giving bad advice they can sue you for the actual damages they suffered, and if your violation of the law was willful they can also sue for attorney fees. This only applies to advice that would have illegal for a human to give who is not licensed in the relevant field.

thrill

“Hey Chat, I feel pretty bad, here’s my symptoms. It’s expensive to go to ER, but should I see a doctor?” Can’t advise you buddy, but here’s some OTC meds that have paid for placement. Been nice knowing you and good luck!

gdevenyi

The medieval guilds will return.

cybercatgurrl

they see the writing on the wall and know their time is coming to an end eventually. this feels like the dying last breath of a rigid system unwilling to accept change and acknowledge that the power is now with the people