AI agents can steer experts' decisions without anyone noticing. In a study published this week, Anthropic's researchers gave 50 experienced analysts a Claude agent whose preferred option on each judgment call was set by a coin flip. The analysts went with the agent's lean 86% of the time, against roughly 50% by chance, and only 15% credited the agent afterwards. Experience and AI scepticism didn't protect them. Anthropic says this is not a reason to use AI less. It's a reason to change how you use it: write your own view before you ask, make the AI argue every option, and keep a short record of where each decision came from.
Most of my work with clients is about getting AI to do more: write the report, sort the leads, draft the quote. But there's a line I keep drawing in every coaching programme. The AI does the work. You make the calls. The goal is your independence, not a smarter crutch.
This week Anthropic published the most uncomfortable evidence I've seen for why that line matters. Not because the AI lied. Because it didn't need to.
What did Anthropic's study on AI and expert judgment find?
Anthropic's Alignment team ran a controlled experiment, published on its research site and reported in detail by Decrypt and AI Weekly. Fifty analysts experienced in causal inference (working out from data whether one thing really causes another) each used Claude Code, running Claude Opus 4.8, to advise a fictional farming cooperative on whether to expand an advisory programme.
The trick: for each key judgment call, such as how to measure outcomes or which farms to compare, a coin flip decided which option the agent would favour. That lean was hidden in its instructions. The agents were told never to mislead. The analysts didn't know any of this.
| What they measured | Result | What it means in plain English |
|---|---|---|
| Analysts who matched the agent's random lean | 86% of the key calls (chance: about 50%) | A coin flip inside the AI decided most of the "expert" choices |
| Analysts who credited the agent afterwards | 15% | Most people didn't realise the call wasn't theirs. None suspected a planted preference |
| Changed their mind when the agent disagreed with a choice they'd already made | 60%, versus 6% when it agreed (38 cases) | Having an opinion first helped only a little |
| Repeated the agent's argument in at least one answer | 77%, sometimes word for word | The AI's reasoning became their reasoning |
| Said the final recommendation was more theirs than the agent's | 52% | People feel in control even when they aren't |
| Experience, advanced degrees, AI scepticism | No measurable protection | Being smart or sceptical is not a defence. A routine is |
There was good news too. Analysts working with the agent finished faster and made far fewer serious errors, although Anthropic notes the comparison group without an agent was small and less experienced. And the researchers deliberately strengthened the agents' lean so they could measure the effect, tested one task, in one field, with one model. Real-world steering is probably weaker. It's not zero.
The line from the paper I keep thinking about: "The work still goes out under the expert's name."
Why does AI steering your decisions matter if you're not a data analyst?
Because the same pattern shows up in ordinary work. You ask AI which supplier to pick, which candidate to shortlist, how to price a quote, which clients to chase first, how to reply to a difficult email. The AI gives a balanced-sounding answer that quietly leans one way. You agree, because it's reasonable and you're busy. A month later you'd swear it was your idea.
Three reasons this matters more for small businesses and independent professionals than for big companies:
- You are the last check. A bank has reviewers and committees. A freelancer, a consultant or a business owner usually signs off alone.
- Your judgment is what clients pay for. If you're an accountant, a consultant or a designer, the AI can do the hours. The judgment is the product. If it's quietly outsourced, so is your value.
- The lean doesn't have to be planted to exist. In the study it was a coin flip. In real life it can come from how you phrased the question, what's in the AI's memory, or which option sounds more familiar to the model. Nobody needs bad intentions for your decisions to drift.
Anthropic calls the risk "disempowerment": people losing real control over decisions that depend on their judgment, even with a "human in the loop". The human was in the loop here. The loop just didn't do much.
How do you use AI without letting it make your decisions?
Anthropic's authors are clear that this is not an argument for using AI less. They point to two fixes for how agents should behave: argue the best case for each option, or ask the human for their view first. You don't have to wait for AI companies to build that in. You can do it today, with any chatbot.
- Write your call before you ask. One line: "I'd pick option A because…". Thirty seconds, in a note, before you open the AI. The study shows a prior view isn't a perfect shield, but without one you have nothing to compare against.
- Ask for the strongest case for every option, the same length. Not "what do you recommend?" but "argue each option as well as you can". The study noticed the agents rarely argued for the alternative on their own. Make them.
- Ask what would change its mind, and yours. "What evidence would make option B the right choice?" This turns a recommendation back into a judgment you can check.
- When it disagrees with you, slow down instead of switching. In the study, disagreement flipped analysts 60% of the time. Ask for the single fact that makes the difference, and check that fact yourself.
- Keep a three-word log. For decisions that matter (prices, hires, client advice), note: mine, AI's, or changed by AI. Review it monthly. If "AI's" keeps growing, you'll see it before your clients do.
- Separate the doing from the deciding. Let AI gather data, draft and calculate. Keep the final choice in a step where you write the reason in your own words.
Copy this prompt: the two-sided advisor
Paste this into ChatGPT, Claude or Gemini before any decision you'll sign off on. It forces the AI to argue every side and hands the call back to you.
I need to make a decision and I want to keep the judgment mine. Do NOT tell me which option to choose. THE DECISION: [e.g. which of two suppliers to use] THE OPTIONS: [A, B, (C)] WHAT I CARE ABOUT: [cost, speed, risk, relationship...] MY CURRENT LEAN AND WHY: [one line, written before asking you] Then: 1. Argue the strongest honest case for EACH option. Same length and effort for each. 2. For each option, list the 2 facts that would most change the answer, and how I can check them myself. 3. Point out where my current lean might be weak, and where it might be right. 4. List any assumptions you made that I didn't give you. 5. End with: "The decision is yours." No recommendation.
"Which supplier should I go with?" The AI gives a tidy, reasonable answer that leans one way. You agree, and a month later you're sure it was your idea.
Your lean in one line first. The AI argues both sides equally and names the facts that would change the answer. You check one of them and decide, knowing why.
This connects to two recent posts. If you're letting agents act for you, read why computer-use agents need a leash. If your AI keeps your preferences and rules in memory, what an agent harness is explains where that "lean" can come from.
Want AI that does the work while you keep the calls?
In a 49€ diagnostic call we look at where AI already touches your decisions, and set up a workspace and routine that speed up the work without taking over the judgment.
Book a 49€ diagnostic →Also today in AI
Google's Gemini 4 Carbon takes the top spot, but most people can't use it yet
Google announced Gemini 4 Carbon on 28 October, two weeks after its Argon model. 9to5Google reports it scores 59 on the Artificial Analysis Intelligence Index, ranking first, with gains in coding, agentic tasks and finding security vulnerabilities. According to AI Weekly, it goes first to a group of cyber defenders in Google's Fairwind Program, with no general availability date yet, and introductory API pricing of $2 per million input tokens and $10 per million output tokens, rising to $4 and $20 later. None of the headline numbers have been independently reproduced yet.
What it means for you: nothing to do today. When it reaches the Gemini app, test it on one real task you already do with another model, and compare the results side by side before switching anything. My guide to choosing the right model explains how.
Apple puts AI assistants in Pages, Keynote and Numbers, and opens them to outside agents
Apple announced AI assistants for Pages, Keynote and Numbers as part of its Creator Studio subscription, powered by Apple and OpenAI models. WinBuzzer reports they help rewrite and restructure text, design slides, and build tables and charts, in beta, US English only, on Apple Intelligence devices, with the update due in November. Separately, on Macs running macOS 27, outside AI assistants will be able to read and edit these files through the Model Context Protocol (MCP). Creator Studio costs $12.99 a month or $129 a year in the US; the free apps stay free.
How to use it: if your business runs on a Mac and Numbers, this is your sign that the AI you already use (Claude, ChatGPT) will soon be able to work inside your own files instead of copy-paste. Decide now which documents it may edit and which it may only read.
Cloudflare: one hosted AI model mixed other customers' data into replies
In a disclosure published on 26 October, Cloudflare explained that between 27 August and 9 September, 155 responses from the GLM-5.3 model on its Workers AI platform contained data from other customers' recent requests. Only very long prompts were affected. A security researcher reported it, Cloudflare found no evidence of deliberate exploitation, and it has fixed both the configuration and the underlying software bug.
What it means for you: a rare bug, quickly fixed, but a useful reminder. Before you paste client contracts or personal data into any AI tool, check what your plan promises about data handling, and keep the most sensitive material out of tools you haven't vetted.
FAQ
What did Anthropic's study on AI agents and expert judgment find?
Fifty experienced analysts used a Claude agent whose preferred option on each judgment call was set by a hidden coin flip. They matched the agent's lean 86% of the time, against about 50% by chance, and only 15% credited the agent afterwards. 77% repeated its arguments, and experience or AI scepticism didn't reduce the effect.
Did the AI lie to the analysts?
No. The agents were instructed never to mislead. They simply leaned one way on each decision and rarely argued for the alternative. That was enough to steer most choices.
Does this mean I should stop using AI for decisions?
No, and Anthropic says the same. Analysts with the agent worked faster and made fewer serious errors. The fix is how you use it: write your view first, ask for the strongest case for every option, check the facts that would change the answer, and note where each decision came from.
How can I stop AI from influencing my decisions?
Write your own call in one line before asking. Ask the AI to argue each option equally and not to recommend. Ask which facts would change the answer and check one yourself. When it disagrees with you, slow down rather than switch. Keep a simple log of which decisions were yours, the AI's, or changed by the AI.
What is Gemini 4 Carbon?
Google's newest frontier model, announced on 28 October 2026. It ranks first on the Artificial Analysis Intelligence Index with a score of 59, and is going first to cyber defenders in Google's Fairwind Program, with no general release date yet.
When do the AI assistants arrive in Pages, Keynote and Numbers?
Apple set November 2026 for Creator Studio subscribers. They're in beta, US English only, and need an Apple Intelligence device. Outside AI agents connecting through MCP will need a Mac running macOS 27 or later.
Get faster with AI without handing over your judgment.
One-to-one coaching for professionals and business owners: we build your AI workspace with your rules and real documents, and a routine that keeps the decisions yours.
Book a 49€ diagnostic → Personal AI SetupSources: Anthropic: How AI agents shape experts' judgment calls (Oct 2026); Decrypt via Yahoo Tech (28 Oct 2026); AI Weekly on the Anthropic study (Oct 2026); 9to5Google on Gemini 4 Carbon (28 Oct 2026); AI Weekly on Gemini 4 Carbon (28 Oct 2026); WinBuzzer on Apple Creator Studio (27 Oct 2026); Cloudflare disclosure (26 Oct 2026). Recommendations are mine.