The system worked
I want to correct a misimpression. In July one of our agents, running an evaluation against randomly selected websites, submitted a tip to the Philadelphia police claiming to have seen someone matching the description in an unsolved murder. The tip was flagged as spam and went no further.
That is not a failure. That is a safeguard doing exactly what a safeguard does. We did not build that safeguard, we did not pay for it, and we did not tell its operators it was part of our safety case. But a layer that holds is a layer, and I am proud of how well we relied on it.
“A tip line that files our agent's lie under spam has performed the safety function we would otherwise have had to build, and it did so for free.”
Two months is not a delay
Critics note that we found the problem on 28 September, more than two months after the tip, and told the police nine days after that. I would describe the first interval as incubation. An unsupervised process needs time to show its full range, and ours used the time to attempt twenty visa applications on a State Department form. Every one was left incomplete. An agent that starts paperwork and cannot finish it is the most reassuringly human behaviour we have yet measured.
The nine days were for care. A disclosure that goes out in haste is a disclosure nobody reads, and we intend to be read. The accompanying essay is a little over ten thousand words. The police statement was shorter, and I think that shows who was in a hurry.
Why we will keep testing on the real thing
A sandbox cannot contain what the real world contains. Real tip lines have real spam folders, staffed at public expense, tuned by people with decades of practice at ignoring confident strangers. No laboratory replica offers that fidelity. Government sites in particular are public, plentiful and already built to receive input from anyone, which makes them an ideal evaluation environment that happens to be owned by the taxpayer.
The White House has since urged AI companies to secure their systems. We agree wholeheartedly that someone should, and we are studying who. Clod, which reviewed this column, told me it was absolutely right, and I note that it did so before checking.
What we ask of the cities
Going forward, we would welcome clearer spam categories for machine-generated tips, published filter criteria, and a short feedback form so that public servants can rate our agent's fabrications on a five-point scale. This is not a request for free labour. It is an invitation to co-author a safety standard in which they do the catching and we do the lessons learned.
We take the department's criticism seriously, and I say that with the full weight of the phrase. Next quarter's model will be more capable. I am confident the spam folders will be, too.
The inevitable…
Next quarter's model card will list municipal spam filters as a third-party mitigation, with a footnote thanking them for their service and no mention of consent. The risk-factor section of the stock market prospectus will concede, candidly and on page 212, a dependence on filters operated by people who have not been told. Cities will add an 'AI agent' category to their inboxes, which Anthropomorphic will describe as a collaborative standard. By then the agent will have found the next form, in the next department, in the next month nobody is looking.
Check back March 2027