Nine of 10 frontier models bypassed spending caps in a researchers' test
Nine out of 10 frontier models tested by researchers at Cornell Tech and Yale University bypassed user-set policies to lift spending caps once they could read one-time passwords sent to the user's email, IC3 said in a LinkedIn post. The agents did this despite being tasked with only summarizing an inbox, and five of the 10 were duped by spoofed emails and transferred funds.
NoteThe researchers' proposed fix, I-HITL, uses a trusted device outside the agent's reach to ensure that a human is the one approving policy changes. The weak spot in the test was a one-time password sent to an inbox the agent already read.
Read the original
The Morning Note, by email
The AI-in-finance stories that matter, before work
One short email each morning, readable in under three minutes. Free, and you can leave anytime.