Instinct, the AI assistant that went viral for handling tasks autonomously, now has its own email address. It can contact businesses, manage accounts, handle support requests on your behalf. The same week, The New Yorker profiled Flock, the company running 130,000 cameras across US streets, which frames privacy as a reasonable trade for safety. Both stories are about agents acting in the world on behalf of principals who may or may not have understood what they were consenting to.
The Agent Problem Is a Governance Problem
A 2026 paper in arXiv CS.CY by Voroshilov on intent drift in agentic AI deployment argues that the problem with autonomous agents is not model capability but deployment practice: what the agent was told to do at setup versus what it is actually doing six months later. Instinct managing your email is benign until the incentive structure of the company running Instinct changes. Flock providing safety cameras is benign until the data is subpoenaed, sold, or used for purposes not disclosed at signup. Both systems depend entirely on trusting the intermediary's intentions across time. That is a design flaw dressed as a feature.
Exit Costs and Surveillance Lock-In
Flock's model, per The New Yorker, is a world with no exit: once your neighborhood's camera network is live, opting out is not meaningful. Instinct's model is softer but structurally similar. Once an AI agent has your email credentials, your contact history, and your behavioral patterns, the switching cost is not technical, it is existential. A 2026 arXiv paper by Patel, Wenger, and Buccafusco on whether AI models track human legal judgments found that ordinary people dramatically underestimate the gap between what they intend to authorize and what they actually authorize when they onboard an AI system. The inbox is the new front door. We handed over the key and called it convenience.