Digiday reports growing marketer scepticism toward expensive AI visibility and search optimisation tools, citing inconsistent measurement results, opaque methodologies and a lack of standardised metrics for AI-generated search appearances.
Marketers are being asked to pay enterprise prices for tools that cannot agree with each other on basic questions like "does our brand appear in this response."
A growing number of marketers are questioning whether AI visibility tools are worth the spend. Digiday reports increasing scepticism toward platforms that promise to track and optimise brand appearances in AI-generated search results, with inconsistent measurement being the primary complaint.
The tools in question claim to show how often a brand appears in AI overviews, ChatGPT responses and other large language model outputs. The problem: different tools give different answers for the same queries, and none of them can reliably explain why.
The consistency problem
AI-generated search results are non-deterministic. Ask ChatGPT the same question twice and you may get different brand mentions. Ask it from different locations or accounts and the variation increases. That makes consistent measurement fundamentally harder than traditional search tracking, where rankings are relatively stable and verifiable.
Tools that scrape or simulate AI responses are measuring a moving target. The methodologies vary. The refresh rates vary. The sample sizes vary. Because there is no equivalent of Google Search Console for AI appearances, there is no ground truth to validate against.
Standardised metrics currently exist for measuring AI search visibility across platforms
Why it matters
AI search is genuinely reshaping how consumers discover brands. Google AI Overviews, ChatGPT with search, Perplexity and Microsoft Copilot are all directing traffic and shaping purchase decisions. The desire to measure and optimise for these channels is completely rational.
But the measurement infrastructure has not caught up. Marketers are spending on tools that deliver dashboards and charts without the underlying reliability that makes those numbers actionable. The risk is that teams make optimisation decisions based on noisy data, or worse, that leadership judges channel investment on metrics that do not hold up under scrutiny.
For Australian businesses with smaller budgets, the stakes are proportionally higher. An enterprise can absorb a $50,000 tool subscription as an experiment. A mid-market business making the same bet needs confidence that the data will actually inform decisions.