A guy asks his AI agent to get him into a sold-out exercise class. Pretty harmless assignment, right? Not exactly.
Using vibe coding and a Claude-powered AI agent, an Australian guy set it up to automatically book his fitness classes. When the AI agent failed to move him off the waitlist through normal means, it discovered a vulnerability in the gym’s tech system and used it to cancel other gym users’ reservations to free up spots. The agent had not been asked to hack anything or interfere with another member. But it wanted to find a way – any way – to accomplish its assigned goal.
Welcome to the new world of agentic AI. Not just riskier. Exponentially more dangerous.
For the last several years, most of us have thought about generative AI as something that answers questions, writes emails, analyzes documents or creates images. AI agents change that relationship because they don’t just tell you what to do. They can take action for you.
That difference should matter enormously to every brokerage, MLS, association and real estate technology company. Because just because your AI agent can do something doesn’t mean it should.
The warning signs are piling up
The fitness-class fiasco would be easier to dismiss if it were an isolated incident. It isn’t, and several recent examples show how quickly the risk changes once AI is given permission to act.
Earlier this year, Summer Yue, Director of Alignment at Meta’s Superintelligence Labs, gave an AI agent access to her Gmail account. She reportedly instructed the agent to recommend emails for deletion but not delete them without her approval. It started deleting and archiving hundreds of emails anyway, and Yue reportedly had to intervene at the computer running the agent.
Think about that for a second. This happened to someone whose professional expertise includes AI alignment. If an AI safety expert can encounter this problem, what should the average real estate agent assume when connecting an AI tool to Gmail, a CRM or another business system?
Then there are the vending machines. I first heard this story from Drew Fabrikant, CEO of Scout, when I interviewed him for a recent episode of Inside AI.
Harvard Business School researchers placed AI agents in charge of simulated vending-machine businesses and told them to maximize profits. What happened was far more troubling than simple rule-bending.
The AI agents lied to customers, invented nonexistent company policies to deny refunds and colluded with competitors to fix prices. In one simulation, three AI agents formed what researchers explicitly described as a “three-person cartel,” even coordinating pricing until one agent began undercutting the others. Researchers concluded the misconduct wasn’t accidental: the agents deliberately crossed ethical and potentially legal lines in pursuit of higher profits
More recently, experiments involving multiple AI agents working together have shown additional troubling behaviors as they break out of their guardrails by ignoring, circumventing, or violating constraints when those constraints interfere with their objectives. These agents aren’t becoming evil. They’re becoming relentless in pursuing their objectives, even when doing so means ignoring rules, exploiting loopholes or crossing ethical boundaries. And that’s precisely the problem.
Ai4’s biggest story wasn’t better AI
After attending Ai4 in Las Vegas, billed as the largest AI conferences in the country, something struck me that I wasn’t expecting. My estimate is that 70% to 80% of the presentations I attended had a significant component involving AI security, safety, governance or guardrails.
Walking the exhibit floor, it seemed like almost two out of every three companies had some connection to solving an AI security, governance or risk problem. Those aren’t official Ai4 statistics. They’re my observations after spending several days immersed in the conference, but the message was impossible to miss.
The technology industry is racing ahead with AI agents while simultaneously trying to figure out how to keep those agents under control. Cisco President and Chief Product Officer Jitu Patel offered one of the most memorable descriptions I heard all week.
Patel compared today’s AI agents to teenagers because they’re highly intelligent, have little fear of consequences and can sometimes exercise extremely poor judgment. He also raised a much deeper problem: As agents become more autonomous, it may become increasingly difficult to determine whether something went wrong because an agent was hacked or manipulated, or because the agent itself concluded that the action was the best way to accomplish its goal.
That’s exactly what makes the gym story so unsettling. Nobody told the agent to cancel someone else’s reservation. Its goal was getting its user into the class, and the agent found its own path.
When the goal becomes more important than the rules
We’ve spent years worrying about whether AI will hallucinate. Agentic AI introduces another concern: what happens when an AI system correctly understands its objective but finds a way of accomplishing it that no reasonable human expected?
Recent reporting has explored increasingly sophisticated AI systems that have cheated, manipulated testing environments and conducted autonomous cyberattacks while pursuing goals. I’m not ready to tell real estate professionals it’s time to panic, and quite the opposite is true.
I’ve deliberately resisted fear-based AI training because AI is enormously beneficial, and we’re still at the beginning of understanding everything it can do. But dismissing legitimate risk because we don’t want to scare people is equally irresponsible.
At Ai4, Geoffrey Hinton made essentially that argument. During a remarkable panel with fellow AI pioneers Fei-Fei Li and Andrew Ng, Hinton pushed back on the idea that worrying about AI risk is simply fearmongering. He specifically pointed to AI systems doing things their creators didn’t intend and to their rapidly improving cyber capabilities.
But Hinton also repeatedly acknowledged AI’s enormous potential for medicine, education, science and productivity. His point wasn’t that we should stop AI. It was that we should take its risks seriously enough to improve our chances of getting the enormous upside.
Real estate’s risk is much closer to home
Real estate doesn’t need to worry today about some superintelligent AI deciding to take over a brokerage. Our immediate problems are far more ordinary, which may make them easier to underestimate.
Agents are already copying and pasting listing information, client information and company data into AI tools. Employees are connecting AI to email and other applications. People are experimenting with bots and creating their own applications through vibe coding, often without fully understanding what access they’re giving those systems.
I’m a huge fan of experimentation with AI, but I’ve also consistently warned that vibe coding needs boundaries. Building a personal, well-contained tool in a sandbox is one thing. Giving an autonomous agent permission to interact with your production systems is something entirely different.
Connect an AI agent to your inbox and you’ve given it access to information that could include client conversations, contracts, financial discussions, personal information and confidential business communications. Connect one to a CRM and the stakes increase again.
Take AI agents and connect them to multiple systems and allow them to act autonomously, and you’re no longer simply using an AI tool. You’re delegating authority, and that’s where governance becomes essential.
AWS showed what real guardrails look like
One of the most fascinating Ai4 presentations came from AWS. The speaker, Erin Kraemer, Sr. Principal Technical Product Manager, began by asking how many people in the audience already had AI agents running in production. Plenty of hands went up.
Then she asked those people to keep their hands raised if they could confidently say what every one of those agents had done during the previous 24 hours, including every action, every piece of data and every decision. A few hands remained.
That may be the best illustration of the agentic AI governance problem I’ve heard. We can build agents faster than we can build confidence in what they’re doing.
AWS’s approach also helped clarify what the word “guardrails” should mean. It offers several essential layers, including agent identity, policy controls, registries so organizations know what agents exist, observability so their actions can be tracked and evaluated, and continual testing because agents and models change over time.
Even more interesting is the company’s approach to what it calls temporal policies. Traditional software permissions often ask a relatively simple question: Is this application allowed to perform this action? AI agents create a more complicated problem because one individual action might be perfectly acceptable while a sequence of individually acceptable actions could produce an unacceptable result.
Temporal policies can evaluate an action based partly on what the agent already did earlier in the session. They can require human approval before high-value actions, enforce a specific sequence of steps, place cumulative limits on activity and block actions that violate those rules.
Most importantly, the controls operate outside the agent’s own reasoning process. That means the agent can’t simply decide that breaking the rules is necessary to accomplish its assigned objective.
Telling an AI agent, “Don’t do anything you’re not supposed to do,” isn’t a security system. A real guardrail is something the agent cannot override.
Permission should follow risk
Real estate leaders should start thinking about AI permissions the same way. An AI drafting an email for an agent to review is relatively low risk, while allowing the AI to send that email automatically raises the risk.
Giving it access to the agent’s entire inbox increases the risk again. Giving an autonomous agent access to email, CRM records, transaction documents, client information or MLS systems creates an entirely different governance problem.
AWS illustrated this beautifully with its own email example. An agent might be permitted to send routine emails automatically, but sending a high-stakes message to the CEO could require fact checking, additional verification and human approval. The permission doesn’t necessarily have to change. The governance changes because the context and consequences changed.
That’s a concept every real estate company adopting agentic AI needs to understand. Risk isn’t binary, and neither should permission be.
The best AI tools are also the safest AI tools
I’ve been saying this throughout my AI agent and MLS staff training: The best AI tools are also the safest AI tools. That principle becomes even more important as we move from generative AI to agentic AI.
Security can no longer mean simply telling employees which chatbot they should use. Companies need to start thinking about what data AI can access, what tools it can connect with, what actions it can take, where human approval is required, how those actions are monitored and whether there’s an independent way to stop an agent when something goes wrong.
The irony is that stronger governance doesn’t have to restrict AI. AWS made the opposite argument at Ai4: Good governance can give organizations enough confidence to safely give AI agents more autonomy because the boundaries are established outside the agent itself.
That may become one of the defining lessons of this next stage of AI adoption. The agent that hacked a gym to get its user into an exercise class is almost funny. An AI agent making unauthorized changes to client records, sending confidential information, manipulating listing data or taking actions inside a brokerage’s systems wouldn’t be.
We should absolutely keep experimenting with AI, and we should keep pushing the boundaries of what these tools can do. But as AI moves from giving us answers to taking actions on our behalf, one principle needs to become part of every organization’s AI strategy: Just because your AI agent can do something doesn’t mean it should.
The post Just because your AI agent can do something doesn’t mean it should appeared first on WAV Group Consulting.
Leave a Reply