In August I gave an LLM agent the role of CEO at our rural internet store. It had access to the sales spreadsheet, Meta Ads Manager, the CRM and our WhatsApp conversations. It proposed experiments, changed campaigns and backed every step with data.

Early in the month, the agent analyzed our SIM card ads and concluded that no creative paid for itself. Its ROAS came out at 0.43: every dollar we spent came back as 43 cents. It recommended cutting the SIM card budget to zero. On August 8, I did.

The math had two errors.

The first one sat on the revenue side. The agent compared 100% of ad spend against revenue from only the sales that carried the ad ID. About 22% of our sales carry that ID. The rest arrive through WhatsApp without a referral, through Google search or straight to the store. When I redid the math, the real ROAS was 2.04. Divide 0.43 by 2.04 and you get 0.21, almost exactly our tracking coverage.

The second error was one of scale. The agent found a marginal ROAS near 1.05 and applied it to the whole budget. That marginal described only the last slice of July's spend. The right move was to trim July's excess and keep the channel running.

With the budget at zero, WhatsApp conversations fell 63% and revenue from the product line fell 14%. In my conversations with the agent, it never went back to the ROI. Every new analysis started from the same 0.43. The number had become a premise, and everything built on top of it looked consistent.

My own suspicion about the drop made me stop. Eleven days after the cut, I redid the math outside the conversation: revenue from the sales spreadsheet against each month's spend. From April to August, SIM card ROAS was 5.4, 2.6, 2.5, 2.0 and 6.2. The budget paid for itself every month. I showed the table to the agent. It recognized the error and saw that the actions it had drawn from that number no longer held up. That same day, I turned SIM card ads back on at US$13 a day.

After the wrong number, each of the agent's analyses made sense. It never went back to check the source number. I accepted the math for 11 days before checking it myself.

Before I change a budget on an agent's recommendation, I now follow three rules:

1. I rebuild ROAS from the complete revenue source, without depending on attribution. If the number only closes with tracked sales, I measure tracking coverage first.
2. I separate marginal from average. A marginal near 1 calls for trimming the excess. It does not justify shutting the channel off.
3. I check the math outside the conversation that produced the recommendation. If I ask the same agent for another analysis with the same number as input, it repeats the error with more arguments.

An LLM agent with access to your data makes an excellent advisor. It reads faster than I do, crosses sources and raises hypotheses I would miss. Put one on your advisory board. The budget decision still belongs to a person who checks the premise and answers for the choice.

The next time an agent recommends cutting a campaign, redo the math with total revenue before you approve it.