GLM-Legal

lenerdenator

There should be laws about disclosure on the part of attorneys about their use of AI. I know they're already using it, but spending years or decades in prison because Lionel Hutz, Esq. did something stupid at a prompt sounds unappealing to me.

Natsu

It looks like they're relying on tracked changes and human review, but I didn't see anything addressing something that has frequently made the news: LLMs outright hallucinating cases or holdings. If the human lawyers don't catch that on review, they're likely to have an angry judge and an AI saying "absolutely, you were right to call me out on that, I shouldn't have hallucinated a non-existent case and made the judge sanction you."

dyauspitr

Lawyers are going to start living gilded lives. I don’t see us getting past their professional human sign-off so they are essentially going to be guaranteed, easy, high paying jobs with the LLMs doing all the heavy lifting in the future.

postalcoder

Looks like this is not an open model. I'm curious how the sentiment toward chinese models will shift as labs trend toward keeping them proprietary. Also, I wonder if this is at all related to the recent reports that Harvey has been getting squeezed on margins by the US labs.

dsr_

https://quoteinvestigator.com/2019/07/30/lawyer/ "he who will be his own Counsellour, shall be sure to have a Fool for his Client." How much worse the lawyer who subcontracts Freakazoid? https://en.wikipedia.org/wiki/Freakazoid ! "Freakazoid has ... access to all of the Internet's knowledge. He cannot fly, but often runs around with his arms extended and making whooshing sounds with his mouth, pretending he is."

Depurator

Curious what domain is next, GLM-Science? And when will it be profitable to drill down to even more domain specific models like GLM-Economic-Geography or GPT-Aquaculture?

toephu2

Sort of ironic, in Google Chrome, when attempting to visit that legal z.ai site I get: Your connection is not private Attackers might be trying to steal your information from z.ai (for example, passwords, messages, or credit cards). Learn more about this warning net::ERR_CERT_COMMON_NAME_INVALID

theokrueger

the liability model of these models is poor, i think progress in that domain should be a heavy focus.

arctic-true

A lot of the examples they show seem to be focused on common law countries. The US and UK are where most of the money is to be made in the legal industry, but I am curious to see what ripple effects we get from people attempting to use AI tools geared toward US/UK-style legal systems in other kinds of systems.

crossroadsguy

Lawyers, essentially the exorbitant and freeloading self-appointed gatekeepers of the deliberately arcane, unaffordable, inaccessible, and patently unequal legal systems all across the world, will now have even their routine/rote, repetitive, menial work done by cheaper LLMs for cheap.

consumer451

Wait, so this not an open weight model, correct? Only available via z.ai? So, if a US or EU person shares their private legal info here, it's processed and stored by an adversarial country that has no extradition treaty with their country?

senshan

The usual concern is attorneys, but judges are probably even more overloaded these days. Can this help judges as well? If so, would this ultimately lead to better decisions?

iLoveOncall

"advancing". Absolutely laughable, we're talking about less than 20% accuracy on benchmarks. 100% useless.

Betelbuddy

No discussion on this can avoid mentioning this ongoing disaster... "AI Hallucination Cases" - https://www.damiencharlotin.com/hallucinations/

otterley

As an attorney, I am not particularly impressed about a 20% pass rate on a benchmark. The name of the game for litigation practice is about being prepared for novel situations, not relitigating past ones.

mathis-l

Not sure how good the Harvey benchmark is. I recently looked into traces from an agent solving their firm knowledge task subset and found multiple problematic tasks and rubrics. Here are 3 examples (there are more): https://github.com/harveyai/harvey-labs/issues/191

bb123

Would be nice if it said what legal system it is set up for. US law? Chinese law? European law? All of them? The author seems to assume we'd know.

newsclues

Professional AI is so interesting to me, legal, medical, engineering, accounting all can benefit. And the opportunity for a moat of AI that is correct and accountable(legally) is ripe for the picking.

aliljet

How does this compare with existing frontier models especially when processing massive volumes of evidence that might be contradictory...

alexnewman

Why don’t I lose all legal privs when I hook up my email to these third party services