The question of “Will AI kill SEO” is probably the most common question I’ve received while developing the thesis for daydream.

The argument goes something like this: what if in the future, most search queries go to services like ChatGPT that provide citation-less paragraphs of results? If that happens, publishers are completely disintermediated and thus, SEO becomes irrelevant. Despite how prevalent this view is, I think it’s severely misguided and unlikely to materialize.

The survival of LLMs is contingent on a balanced relationship with publishers

There are three key reasons why the survival of LLMs is contingent on citing sources.

1. Chatbots must sufficiently incentivize publishers in order to access to their content

The common input for both search engines and chatbots is content, produced by millions of individuals and organizations around the world. Without that content, chatbots would be unable to render answers to the ever-growing list of questions people have about the world.

In the event that chatbots start disintermediating publishers, publishers will quickly lose the incentive to allow chatbots to crawl and index their content. Why would you allow someone to crawl your website if they’ll never credit you for it? Already, solutions like Sphere have emerged to address this concern by denying access to LLM crawlers.

There will be a minimum threshold of value that LLMs will need to offer publishers, in order to maintain access to their content. If the LLM creator wants to have the broadest and deepest access to data sources, it’s in their own best interest to build a positive relationship with content publishers.

2. Sources are an important factor in delivering quality results

One can rationally assume that providing the best result for the user is in the best interest of the chatbot. If that is the case, providing link-outs to publishers is important for maintaining their access to a broad swath of content, as well as providing high quality results to the end user.

• Citing sources is important for trust and verifiability. Not including sources doesn’t increase the quality of the result — it decreases it. If the chatbot creator is interested in providing the best possible result for the user, they should eventually start providing source data as well. Given that it’s possible to provide these sources without compromising on the user experience in other ways (as we’ll see in the next section), we should expect attribution to become more important over time.

• Some search queries require that the user be re-directed to another party that can fulfill their request. For example, consider the search for “[document] template”. Canva famously attracted millions of users by programmatically generating pages to serve search queries like this. When a user makes a search like this one, they’re looking for a website that they can download that document template from. If no hyperlinked source was provided to actually download that template from, the quality of the result would be mediocre at best.

  Another example would be a search like “vacation rentals near [location]”. The user making the search in this case is likely looking for vacation rentals that they can book directly. Without linking out to a destination website where this can be done (ex. Airbnb, VRBO, etc.) the search result would be mediocre.

3. Chatbots cannot be monetized effectively without promoting publishers

Removing the ability for content publishers to appear in answers altogether would drastically handicap the ability for chatbot creators to monetize their users. Google is able to generate $200B+ from search through its ads business, AdWords.

If chatbot creators like OpenAI choose to only rely on charging end users for premium access like they do now with ChatGPT Plus, those businesses will not be nearly as lucrative as the machine Google has built. This is because in general, advertising revenues generally exceed the revenue that can be generated by charging the user directly. This principle has held true across a diverse array of advertising businesses from Meta, to Google, to Netflix and Disney.

Cutting out content publishers completely in chatbot search results is likely to lead to worse monetization possibilities. Only charging end users will not generate nearly the amount of revenue created by offering paid placements for content publishers. It’s likely that the leading Google rivals of the future will, in some way, offer paid promotion opportunities for publishers.

Will the pessimistic case come to fruition?

For the reasons mentioned above, we can reasonably come to the following conclusion. It’s unlikely that chatbots using LLMs will be able to dis-intermediate publishers. The closer they get to doing this, the more they risk a counter-reaction from the publishers. It’s in the best interest of the chatbots to credit publishers both to maintain their access to that content, as well as for producing high-quality answers to search queries. Cutting out publishers completely would also render the chatbots incapable of generating advertising revenue, which would leave them with less-than-ideal monetization options.

The evolution of SEO

Search engine optimization vs. Chatbot optimization

In the previous section, we established that it’s unlikely that chatbots will completely dis-intermediate publishers. There are strong reasons to believe that citing, crediting, and linking out to publishers is in the best interests of the chatbot creators themselves. With that said, what does that mean for SEO?

In the same way that SEO emerged as sub-industry for optimizing the ability to appear on search engine results, a new set of optimization practices will emerge around appearing in chatbot results. If virtually 100% of online Q&A flows through search engines right now (almost all of it to Google), we can expect that to become far more fragmented over the coming decade.

While Google may prove capable of adapting, its unlikely that it’ll maintain the same level of market dominance that it’s had for the last 20+ years. It’s more likely that its market share will be shaved off by companies like Microsoft, OpenAI, chatbots build on open source LLMs, and a variety of other players.

In summary, SEO as we know it is likely slowly to decay over time. A new generation of optimization practices will take it’s place, inclusive of optimizing across both chatbots and search engines. TL;DR — in the future SEO ≠ optimizing for Google search.

Take the long view

Although new optimization practices will emerge for appearing in chatbot results, there’s one principle that’s likely to survive all the changes yet to come. That principle is the following: Content should be written for people, not machines. The content that best serves the user is likely to have staying power, whether its on Google or a chatbot. While there may be some optimization principles that differ across surfacing for Google vs. other players at the indexing layer, the most important factor in surfacing for either will be the quality of content written.

Reconciling our theory with today’s reality

It’s helpful to reconcile the conclusions mentioned above with the actual state of chatbots right now. In this section we’ll walk through the three major generative AI chatbots and examine their approach to working with publishers. What stands out is an overwhelming shift towards embracing sources and citations across the board.

OpenAI (ChatGPT)

Note: On July 3, 2023 OpenAI disabled the “Browse with Bing” feature. The primary reason cited for this step was the unintended display of content in a manner that OpenAI deemed undesirable.

OpenAI hasn’t released any public statements on how they think about working with publishers.

By default, ChatGPT doesn’t provide sources or attribution for any questions you ask it. This is likely to be a temporary state, and not a representation of how ChatGPT will look over the next few years. If

ChatGPT’s internet connected version provides some signal into how ChatGPT will evolve over the next few years, providing sources for every search result. In this particular dimension though, ChatGPT’s results are still inferior to Google’s by a long shot. When I asked ChatGPT “What are the best white sneakers for men?” it essentially summarized one GQ article which it retrieved via the Bing web search API.

This search demonstrates to me the difference in the types of search queries that are best handled by a search engine vs. a chatbot. When it comes to shopping, receiving a single, brief answer based on 1-2 sources isn’t ideal. In this particular case, it’s more useful to see the ~10 results Google produces, make a snap decision about which one I trust the most, and then click through. For example, if I could choose, I’d rather get my answer to this question from The Wirecutter over GQ.com.

The impact of not properly citing sources on user experience is clearly evident when using ChatGPT. Without expanding the way they providing citations, it’ll remain subpar for almost all search queries with purchase intent. These search queries are the most valuable ones, so it seems like a non-starter for ChatGPT to remain in it’s current state. If it does, it won’t be able to compete effectively with Google or Bing.

Microsoft (Bing)

Microsoft’s new chat-based Bing is powered by GPT-4, as part of a strategic partnership with OpenAI. On March 29, 2023 Microsoft’s Yusuf Mehdi published a blog post outlining its stance on how they work with publishers. In short, they want to drive more traffic to publishers, not less, it appears to be working.

“We want to drive more traffic to publishers in this new world of search.  It is a top goal for us, and we measure success in part by how much traffic we are sending from the new Bing/Edge.”

Yusuf also outlines the importance of balancing incentives across the search ecosystem.

“The Internet works because of an important ecosystem which starts with publishers creating great content which drives traffic and interest by people to consume that content, and then ultimately advertisers that want to reach people in a high quality and targeted environment.”

This approach is reflected in the way they surface results on Bing. When I asked Bing “Where can I find a job offer letter template?” it provided me with 3-5 different recommendations, along with links to each source.

Note: The new Bing search is only available on Mac via the Microsoft Edge application, which doesn’t seem to work at all on my M2 Mac. Because of that, I decided to use the Bing application on my iPhone instead.

Compared to OpenAI’s responses, Bing’s struck me as vastly more useful. They balanced a concise response with citations to a variety of useful sources, all while managing to integrate advertising in a subtle but effective manner.

Google (Bard + GenAI Snippet)

Google’s unveiled two interesting projects thus far. The first is Bard, their ChatGPT competitor. The second is their Search Generative Experience (SGE), which is a new feature in Google search. Google describes Bard as “a complement to search.” Both of them use the same foundational LLMs that Google has been developing for some time now. Let’s start with Bard, and then move onto SGE.

When I first used Bard to ask for the “best hiking trails near San Francisco” in early June, I received a list of recommendations without any citations.

Similar to OpenAI, this created a subpar user experience for me. Where do I go if I want to learn more about one of the trails they’re recommending?

Since then, Google shipped an update on May 15, 2023 that added citations.

If you search the same query now, you see a very different user experience. Each and every recommendation is accompanied by multiple citations.

Turning to SGE now (which you can join here), I searched “what is the best travel size hair dryer” and received the following result. This snippet provides citations to two types of pages. The first are publishers that created comparison articles on this particular topic (on the right), while the second are retailers that actually sell the products being mentioned (on the left).

If you click one of the products, a new screen opens up with a list of places to buy that item.

One can infer based both on Google’s core business model (advertising), and the evolution of both Bard and SGE that they have a similar goal to Bing. They want to send more clicks to publishers by leveraging generative AI, not less. They haven’t published anything directly to this effect yet, but I suspect an announcement like that will follow shortly after (or before) Google releases SGE to the public.

Shifting in a common direction

Of the three major search players worth monitoring (Google, Bing, and OpenAI), it’s clear that both Google and Bing are employing generative AI to drive more clicks to publishers, not less. There’s early indications that OpenAI is also moving in this direction, but their position is not as clear as either Bing’s or Google’s. That said, I expect them to follow suit for two primary reasons.

The first is that without citations, the quality of ChatGPT’s answers for most queries are subpar and in many cases, worse than Google or Bing’s. Integrating citations will be an essential part of producing better results than its competitor.

The second is that producing an answer to every question is not cheap, and there’s a small amount of revenue OpenAI can generate by charging for ChatGPT Plus relative to what it could generate through advertising. Sam Altman has already stated that they’ll need to “monetize [ChatGPT]” somehow at some point.” That “somehow” is likely to be advertising.

Source: Twitter

Looking into the future

There’s a lot that will change for the future of SEO. In 5 years, we should imagine a world where Google has significantly less market share than it does now. Microsoft, OpenAI, and other competitors (many of which probably don’t exist yet) at the indexing layer will carve away market share and become valuable sources of traffic.

The ability to produce high quality content at scale will also improve dramatically, as LLMs and the workflows for using them in content production continue to get better. This is the piece that daydream is trying to accelerate.

What won’t change is the mutually dependent nature of the relationships between content publishers, indexing layer like Google, OpenAI, and Microsoft.

Contrary to popular opinion, it seems like generative AI will drive more clicks towards publishers, not less. There’s initial data from Bing to support this, and I expect Google to follow suit soon given how similar their approach is to Bing’s.

OpenAI is a wild card and has less to lose as compared to incumbents with existing search-based advertising businesses, but without citations its answers lack the trust, verifiability, and depth people are accustomed to.

If you have thoughts on this topic, I’d love to hear from you directly. Join our waitlist or email me at [email protected].