LLM outputs (AI answers from chats in Claude, ChatGPT, Perplexity, etc…) are customized based on what the system knows about the user which is why two people can search the same thing and get very different recommendations for the same product, service, or piece of advice. If it knows person A has a cat and person B has a dog, and both ask “which toy should I buy for my pet under $20”, one person will get a dog toy and the other get a cat toy.
The current output comes down to the LLM knowing:
- Which data sources have products that are relevant to the person asking the question
- The specific sections of a website, forum, or community that match the interests and needs of the person researching
- Weighing the “trusted” sources of each platform differently based on the buying, consuming, and informational trust habits of the individual user on the LLM
- Credibility could become a signal as well (i.e. pay to play or actual editorial content)
In the future I think this will customize even more without needing detailed prompts like showing “treats for training a dog that has a chicken allergy, seizures, and is a rescue with trust issues, or with disabilities like being deaf.” If what I am seeing continues to progress, the person won’t need to type “deaf” or “rescue” into the prompt. More on this later in the post.
When the users ask for sources, both users could be shown identical sources including affiliate shopping lists on blogs and media sites, and both could be shown the same social media creators and reddit forums. This does not mean each of these were used in the recommendation, or that the specific content is weighed as heavily. Some of the sources may be shown only because they built trust for the brand whose products make it to the output via RAG validation. The products in the recommendation may not even be on the lists.
The amount of weight given could be divided up multiple ways:
- The list may be used to verify they are pet toys, but not given weight on which toys or treats to recommend.
- If there are social media forums and influencers (marked with authorship) talking about them with no sponsorships or affiliate links, this may hold extra credibility in the future as it is authentic vs. money being exchanged.
- If there are no affiliate links (including javascripts that monetize on the outbound click) on the list, it less likely to be pay-to-play and the listcicle may be more likely to have journalistic integrity.
- A mix of big brands with new companies and small businesses shows actual research and that payment is not required to be included.
- Lists with only big brands or affiliate links mean the editor did not research outside of approved vendors. They simply choose from a list and published.
- Yes, some publications require testing, but to be included you still have to pay so it isn’t actual journalistic integrity where they have a say on including unpaid vendors, it is still pay-to-play algorithm wise.
- To be unique and have something of value (i.e. helpful content), unknown brands that are included on the list but not others, and that are also tested like the big brands, show research had to actually be done rather than choosing from pre-approved brands on a list.
- The reviewer or editor may have special licenses and certifications that matter for one user, but not the other.
- If the author is certified in providing healthcare for dogs with disabilities vs. someone that only has general certifications, this could be weighed as more relevant for the specific user.
The algorithms used by AI, and AI’s ability to crawl are both still basic at best. They’re not rendering javascript, they’re likely still grounding in search vs. their own trust factors, AI has limited data resources for generating a result, etc… As they improve it is likely their sources for generating a trustworthy and user-relevant response will as well. That’s how I see this changing, and it is very different than SEO.
SEO meets the needs of a larger audience and a general query, AIO/GEO from an LLM output is a customized and personalized experience for the specific user only, so that listcicle brands are spending money on will likely not be an effective way to get featured in the future.
You could have someone like me that loves running and lives in an urban setting here in Washington DC, and a similar person demographically that also loves running, but lives in Denver Colorado or Salt Lake City Utah. We both need running shoes for trail running, but my trails are flat, we have heavy humidity, and a strong amount of rainfall.
Denver and Salt Lake City have inclines and elevation, are more likely to have snow for more of the year, and be dryer climate wise. The LLM could have seven or more publications or resources it thinks are sources of quality information that will benefit both of us. All seven could be shown as sources of information, but not all will be used for both of us in the recommendation.
- Runners world magazine
- A reddit subforum for running
- Washingtonian magazine
- The Denver Post
- Popville/Prince of Petworth
- Visit Denver
- Men’s Health
- Social media creators that focus on running
Right now AI would likely source the major fitness and running publications for advice as well as retail stores and blogger lists that are in their indexes. From there they cross reference with YouTube, reddit, and social media to try and gauge if something is factual or a good output (i.e. grounding). Next they generate a response based on what they have from these sources and what they know about me personally. In the future as AI gets more advanced and knows more about me and the other person, this can and will change.
If the LLMs are grounding in Google, you should expect their outputs to start changing soon as well because their trust signals shift based on Google’s algorithm shifts.
They will know the climate I run in is flat, humid, and gets slippery from rain and ice, so a stronger grip in rain and waterproofing on the shoe may matter to me. The other person will need something that works for mountains or hills with elevation runs vs. flat, and has to work for snow and ice (like mine), but is designed for dry heat vs. moisture.
Both of us will get citations from the mass media companies and listcicle, but mine may come from authors in the media company relevant to my needs that write specifically about running in situations I come across.
- If they mention why certain shoes failed and the ones they changed to naturally in the guide (not in a shopping list), this will likely carry substantial weight via RAG validation in the future.
- The shopping list says its a brand that sells product which can build trust the brand is legit, but the validation of quality and for a specific use or purpose comes from natural inclusions in actual expert content.
The other person’s sources will be from the same sources, but based on author’s that write for their needs. This is similar to authorship in SEO and AI understanding who the authors write for.
Being featured in the publication on a specific page or two is not going to work for the “long run” (sorry, couldn’t resist). I feel very confident in saying that. Google is already warning not to do this for AI Overviews, so it is a safe bet that the LLMs will follow suit and the pay-to-play model for listcicles will start working against you.
As the algorithms and RAG validation gets more advanced, being featured on a sourced list doesn’t actually mean you get any credit either. If the author and reviewer are not relevant to the individual user asking a question, the list may not count for much because the authors aren’t specialized in meeting their needs. But it may still show up as a source when you ask for it.
Articles written by someone that runs in DC, NOLA, or Miami will have similar experiences to me (minus the snow and ice), so as they talk about specific trails and mention the shoes, socks, and gear that work for them, this adds trust and credibility for the output shown to me. It does not build it for the person in Denver.
Then there is the pay-to-play affiliate listcicles and how they’re different from SEO. The main shopping list could be used to determine which stores and brands are legit as the brands are paying to be listed and featured, but the products recommended are for the masses which works for SEO, but not personalized LLM and AI outputs.
This is because SEO meets the needs of a larger audience and a query from a search engine. AIO/GEO from an LLM output is a customized and personalized experience for the user only, so the list brands are spending money on will likely not be effective in the future as it is not customized to user needs, just large search queries.
If I was a brand that creates shoes perfect for elevation and snow running, sites like Washingtonian won’t matter and neither will urban running guides in Florida as there are no mountains or large hills. They may be sourced, indexed, trusted, and cited for brand validation, but the output won’t be relevant for the other person in an LLM, so these get a lower priority in trust or ignored completely for the query. The brand in this example should make these lists and creators a secondary priority as that is not where their customer base is.
Just like a running guide in Washingtonian magazine or in a local blog like Popville will be relevant for me, they may show up as sources for the person in Colorado or possibly Phoenix, but the content may not be used to make a decision on which brands to feature in the output as the authors do not write for someone in Colorado. Spending time on these lists because they’re sourced will not be a good use of time or resources if the shift I think I’m seeing continues.
As a brand, think about where your actual buyer base is. Right now I’m seeing people flock to any site or community being cited, and this will likely be seen as spamming for citations and outputs. It is too easy to game and the lists will become too similar. From there the algorithms need to find new resources with unique inputs for validation vs. the pay-to-play.
The same goes for paying influencers and creators who are showing up in AI Overviews or getting cited as sources. Just because they show up and get sourced does not mean they’re relevant to your buyer base, and that will likely matter more as LLMs customize for individual users.
We’re focusing on where the actual customer base is and the content that focuses on their needs. We’re then encouraging creators and editors to talk about the benefits that meet our customers needs. By focusing on communities where our customers are there’s a better chance that in turn can generate revenue vs. only trying to get a citation. This is a more natural approach rather than chasing citations because they’re sources.
Getting this information is easy as well. Your customers will tell you if you ask them post checkout, via surveys, and by scanning through your live chat and support ticket databases.
By building the trust with the sites, communities, and influencers that cater to our client’s audience needs vs. the a huge audience and SEO listcicle, the LLM knows how and when to feature specific products for each specific LLM user. This is the opposite of SEO where it is for a larger audience.
If I sell shoes for both elevation and flat running, I can work with authors at the same publication that meet both types of runners’ needs, and to be featured in the shopping listcicles. This will in theory meet the future needs of both SEO and LLMs while building trust signals for RAG validation.
I have a strong feeling the experts that match our client’s audience base vs. being an expert in general will pass trust to their brands and product lines, and the algorithms will use what they know about the content (sizes, grip, comfort, pronation, etc…) with schema to show products without having to be on a shopping list. Because they already know about the person asking for a recommendation, they can now match it based on the person’s demographics and needs via the information they get from actual content and RAG validation.
Right now LLMs are easy to game. You can pay-to-play, spam sites, and win all day, but this will come to an end. We’re already seeing it with Google’s products. If the LLMs are grounding in Google, you should expect their outputs to start changing as Google changes because their trust signals shift based on Google’s shifts. When it drops from Google, it may drop in the LLMs too once the LLM updates. Instead of chasing silver bullets, focus on where your customers or users go for information and research. Track the authors they trust, the sections of the websites or communities they engage with, and work to get featured there. This is where I see more long term success coming from.
