AI Brand Recommendations: Visibility Over Rankings

After diving into this research, my first reaction is a mix of relief and caution. The core argument here is crystal clear: AI tools like ChatGPT, Claude, and Google AI produce wildly inconsistent lists when asked for brand or product recommendations. The chance of getting the exact same list twice is less than one in a hundred, and the same order is even rarer. This shatters the illusion that AI-generated rankings are stable or reliable metrics for brand visibility. Commercially, this is huge. Companies are pouring millions into AI tracking products that promise to measure brand presence or ranking in AI-generated recommendations, but without understanding the fundamental randomness baked into these models, they are essentially chasing ghosts. The research smartly pivots from ranking positions to a more nuanced metric: visibility percentage across many repeated prompts. By running the same queries 60 to 100 times, a pattern emerges showing which brands consistently appear in AI responses, even if their order fluctuates. This visibility metric, while imperfect, offers a statistically meaningful proxy for brand prominence in AI's 'consideration set.' For leadership, this means a shift in how we think about AI-driven brand tracking. We should stop buying into single-shot ranking reports from AI tools and instead demand transparency about methodology and sample sizes. The tension lies in the fact that AI's probabilistic nature conflicts with traditional expectations of search or recommendation stability. Yet, there is an opportunity here to develop smarter AI tracking tools that embrace this randomness and focus on aggregate visibility rather than rankings. The article also highlights a critical blind spot: prompt diversity. Users rarely craft similar prompts, meaning the variation in AI responses in the wild is likely far greater than controlled experiments suggest. This adds complexity to any tracking effort but also points to the need for large-scale, diverse prompt sampling. From an operator's perspective, the takeaway is clear: treat AI recommendations as probabilistic signals, not gospel truths. Invest in repeated sampling and statistical aggregation to understand your brand's true AI visibility. Demand that AI tracking vendors publish their data and methods openly. And be wary of tools selling simplistic ranking metrics that ignore the inherent chaos of AI outputs. This research is a much-needed reality check for marketers and executives eager to harness AI insights but prone to overtrusting the technology without scrutiny. The future of AI brand measurement will require patience, rigour, and a mindset shift away from fixed rankings toward probabilistic visibility.