← AI Agents & Vibe Coding
Episode 12··33:53

Google answers by itself, ChatGPT barely sends visitors: is SEO still worth it

You built a website, and visitors from search come less and less often. Everyone around says Google will soon be useless, and it is unclear whether to keep spending time and money on promotion. In this episode you will understand why Google more and more often answers questions by itself and which pages lose visitors the most. You will learn why people do not swap Google for ChatGPT but use both, what time frame makes sense for your plans, and how many people AI models send to websites today. I explain how an AI model looks for sources for its answer and why it does not simply retell Google's top results. I cover which trendy tricks "for AI" do not deliver, why in both search and AI the authority of a site decides everything, and how to earn it without buying links. I speak from experience: in one year I took my own project from zero to about half a million visitors without ads. At the end there are three tasks for your agent: check whether AI bots can see your site's text, make sure they are not blocked from your site, and add answers to your clients' neighboring questions on your important pages.

Transcript

Tap a paragraph to listen from there.

Good evening, startup founders and vibe coders! Today I am going to tell you what is really happening with search on the internet: whether Google is actually dying, how many people already come to websites from ChatGPT, and whether it is worth spending money and effort on promoting your site in search right now. People talk about this a lot based on gut feeling. I will try to lay it all out with facts.

Here is what you will take away from this episode. You will understand why fewer visitors come from search even though people search more than ever. You will learn what time frame makes sense to plan for, and why the turning point is coming more slowly than it seems. I will explain how an AI model chooses which site to cite in its answer, and why no checklist will work without the authority of your site. And there will be a few things you can hand to your agent today.

I will start with a feeling that many people have right now. It seems that search engines are living out their final years. Young people ask an AI model and do not google. A little more time, and typing a query into a search bar will feel as odd as writing a letter on paper and taking it to the post office. We will check whether that is true. But first, let us look at what is happening right now inside Google itself.

You have almost certainly seen this. You ask Google a question, and at the top there is a ready answer written by an AI model. This is called AI Overviews. There is also AI Mode, where instead of a list of links a model simply talks with you. And this is where something begins that the industry calls cannibalization. Google takes other people's texts to give the answer itself, and the person no longer needs to go to the site the answer came from. According to the company Similarweb, this summer these overviews appeared on forty-three percent of queries. A year earlier, on fifteen percent.

How hard does this hit websites? The most honest measurement we have is an experiment by two economists. They took about a thousand ordinary people in the United States and installed a small program in their browsers. It split people into groups at random: some saw normal Google, while for others the AI overviews were hidden. The groups were alike, and the only difference between them was whether a person was shown a ready answer or not.

Here is the result. Where an overview appeared, people clicked through to websites almost forty percent less often. Searches after which a person went nowhere at all rose to almost three quarters, while without an overview it was about half. And here is the curious part. People were equally satisfied with search, with the overview and without it. So websites lose visitors, and the person hardly even notices a difference.

That is where the figure of minus forty percent comes from, the one people argued about this week on Hacker News, a large forum for programmers. But it is important to understand what exactly was measured here. It is the loss of clicks where an overview appeared. It does not mean that every site lost forty percent of its visitors. Other measurements give a similar picture. Last year the Pew Research Center looked at almost seventy thousand searches. When there was an AI answer at the top, people clicked a regular link in eight cases out of a hundred, and without such an answer, in fifteen. And a link inside the AI answer itself got a click in one case out of a hundred.

For those who live off this traffic, this is no longer research, it is bookkeeping. People Inc., a large American media publisher, reported in early November that it had "lost nearly 70% of our Google search traffic over the last few years". Inside the company they have a name for it: "Google Zero".

What does Google itself say? Its head of search wrote last year that total click volume to websites has been, in her words, "relatively stable". But Google does not publish its own figures, and all the independent measurements show a decline. Meanwhile, Google itself is doing fine: at the end of October the company said that the number of queries is growing faster than at any time in the last five years. There is no contradiction here. More people are searching. Questions have become longer. But people go to websites less often, because the answer is already on the screen. There is one more honest detail: Google's chief business officer admitted himself that ads in AI Mode bring in less money for now than in regular search. So Google is going down this road even while losing some money on every query.

So here is the first thing I want you to remember. The search engine is not dying, it is changing. It used to be a directory of links, and now more and more often it answers by itself. This hits hardest on reference questions like "what is" and "how does it work". According to the service DataForSEO, on such queries in the United States AI overviews appear in roughly seven cases out of ten. If your project lives on articles that explain common knowledge, that traffic will keep leaking away. But where a person needs to do something, buy something, book an appointment or find a specific company, overviews appear less often, and the loss, by all signs, is smaller.

Now to the question that worries people most. When will people stop googling altogether? There is a popular picture: young people will move to AI, and older people will stay in Google simply out of habit. Let us check it against the numbers.

In February of this year, Pew surveyed more than five thousand adult Americans. Among people aged eighteen to twenty-nine, sixty-one percent had tried ChatGPT. Among those over sixty-five, only nineteen percent. That is a huge gap, so the generational picture holds up here. But look at how it was three years ago: thirty-three percent among the young and four among the older group. Both groups are growing, and fast. And the people who use chatbots every day most often are not the youngest at all, but people aged thirty to forty-nine.

And the most interesting part is what happens to a person once they start using ChatGPT. Semrush, a company that sells tools for promoting websites, looked at a huge body of data on how people move around the internet. It turned out that after their first encounter with ChatGPT, a person goes to Google not less often but more often: it was ten and a half times a week, and it became a little over twelve. According to Similarweb, about ninety-four percent of ChatGPT visitors keep using Google. An analyst at eMarketer put it very precisely: "people also tell us they haven't stopped using search engines – they've simply added AI as a new behavior on top of search."

So the idea about habit is right, but it works more broadly than it seems. Habit does not hold only the older people. It holds everyone. A person does not choose between Google and AI, they use both. A short question still gets typed into Google, and there it is already Google's own AI that answers. A long conversation happens in ChatGPT.

And this is where the image of the paper letter becomes really interesting. Letters did not disappear because people got tired of writing. They were replaced by the phone that is always in your pocket. With search, things are trickier. The replacement arrives in the very same search bar. You type a query the way you always have, and it is already an AI that answers you. So googling the old way, with a list of ten links, really may become as old-fashioned as a paper letter. Only people will not leave Google, it is Google that will change around them.

Forecasts about timing have already been wrong here. In early twenty twenty-four, the research firm Gartner predicted that by this year the volume of traditional search would fall by a quarter. In terms of the number of queries, that did not happen; there are more of them now. Later a Gartner analyst admitted that this figure was born in an internal debate, not from calculations. But clicks to websites in the United States, by the count of SparkToro and Similarweb, fell by about a quarter per thousand searches. People search, they just go to websites less often.

So why is everything so slow, when in other areas agents turned everything upside down in a matter of months? Here is an example I watch myself. Not long ago it seemed that editing short videos for social media would be automated in about two years. Roughly half a year has passed, and many people already make videos entirely through an agent: it edits the video by itself from the source footage. Why so fast? Because here only one person has to change a habit, the one who makes the video. The viewer does not care how the video was edited. The creator tried it, saw that it works, and the next day they have a new process.

With Facebook ads it is already harder. Meta has long been moving toward having a machine set up the ads. Last year The Wall Street Journal wrote that Meta wants to fully automate both the creation of ads and the choice of audience by the end of this year. More than nine million small businesses already use at least one of its tools that makes ads with artificial intelligence. The capability is there. But many agencies still split the budget: part goes to automation, part is set up by hand. And from what I see, many businesses still hire a Facebook ads specialist, just as before. Here it is the business owner who has to change a habit, and they have to trust a machine with their own money. That is slower.

And search is the hardest case. Here the habit has to change not for one creator and not for one business owner, but for billions of people who have been typing queries into the same bar for twenty years. The technology is already here, but the turning point has not come. And not because the features are missing, but because the habit is the same. That is why predicting the timing here is even harder than with videos or ads.

So what time frame should you plan for? Nobody can name the year Google dies; there are simply no forecasts with a sound method. But the direction is visible. eMarketer, for example, expects that as soon as twenty twenty-seven, AI Mode inside Google will overtake ChatGPT in monthly users. So the main competitor of traditional search is not ChatGPT, but Google itself in AI mode. This is how I would plan: for the next couple of years, Google will remain the main source of people for most sites. But every month it will answer by itself more and more often. You need to prepare not for people leaving Google, but for Google letting them through to your site only when you have something its answer does not.

All right, and what about ChatGPT? How many people does it bring in already? There is a loud claim here. In early November, the head of consumer ChatGPT said that ChatGPT's search queries are already about one fifth of Google's. But he counts as a search any question after which ChatGPT goes to the internet, and he did not say where he got the figure for Google. Other estimates are much more modest, down to the point that on desktop computers in the United States, ChatGPT accounts for less than three percent of all searches. It all comes down to what you count as a search.

But for you a different question matters: how many people come to your site specifically. And here the numbers are fairly clear. Ahrefs, another company that sells tools for promoting websites, recently counted visits to more than a hundred thousand sites for October. All the AI assistants together brought these sites just under half a percent of visits. Google, almost a quarter. That is a difference of roughly fifty-four times. ChatGPT itself gave about a third of a percent, while Gemini, Claude and Perplexity gave hundredths of a percent.

Why so little, if ChatGPT, according to the company itself, has more than a billion users a week? Because ChatGPT sends people almost nowhere. According to Similarweb, in May links appeared in roughly seven answers out of a hundred. In the rest, not a single one. The person got an answer and left. And when a click does happen, about six out of ten clicks now lead to the site's home page. So a person learned a name and went to take a look at the company, not to read a specific article.

And now an idea I hear often: ChatGPT gives old sites little, but seems to give new ones something quickly. In Google a new domain takes a long time to build trust, while in ChatGPT, people say, you can get in within a few days. There is one clean experiment on this. Semrush published eighty-one new pages and watched for a month who would start citing them and when. In the first day, Google in AI Mode cited more than a third of the pages, and ChatGPT one in ten. By the end of the month ChatGPT was actually citing even more than Google. But Google was definitely faster. Semrush itself honestly noted that its blog has a very strong domain, and this experiment does not show what would happen with a brand new site.

So where does the feeling come from that ChatGPT gives new sites something quickly? I think the point is that mentions come quickly, not people. Glenn Gabe, a well-known search specialist, described a site stuffed with AI-written texts that is not visible in Google at all. Yet in ChatGPT's answers it came up on more than ninety thousand queries. A mention is when an AI model refers to a site somewhere inside its answer. It does not mean that anyone actually visited. You see a nice number in a report, while there may be no visitors at all.

There is also a myth about the quality of these visitors. You may have heard that people from ChatGPT buy twenty-three times better. Ahrefs got this figure on one single site, its own. Meanwhile an independent scientific study of almost a thousand online stores showed that visitors from ChatGPT bought not better, but even slightly worse than visitors from regular search, although this difference does not show up in every check.

What to do about this in practice. ChatGPT itself adds a service tag to the address in its links; it is called a UTM tag, and the source in it is ChatGPT. Ask your agent to find visits with this tag in your analytics and show them separately. But remember that the picture will be incomplete. A person who saw you in an AI answer and then typed your name by hand will never be recorded as coming from ChatGPT. So growth in direct visits and in searches for your company's name may also be a trace of AI.

Now the most important part. How search works in general, and how search through AI works. Without this understanding, all advice rests on faith.

With regular search things are more or less clear, and the formula has long been known. You make content and earn backlinks, meaning other sites link to yours. The more strong links from authoritative sites, the more the search engine trusts you, and the more traffic it gives you. Google does say that links have come to matter less. A large study on a million queries confirmed it: they matter less, but they still matter. There is also the second half of the formula, how people behave. In twenty twenty-four, Google's internal documents leaked online, about two and a half thousand pages. They revealed a system that takes clicks into account: it has markers for good and bad clicks and for the page where a person stayed the longest. How much weight it carries, the leak does not show. But the idea is clear: Google looks at whether a person found on the page what they were looking for.

And now AI models. There is a theory going around that they simply take the top Google results for your question and retell them. That is not quite so. The process looks roughly like three steps.

Step one. The model breaks your question into many ordinary search queries. Google calls this "query fan-out". When Google launched AI Mode, it wrote exactly that: the system issues several related searches across subtopics at the same time, and then brings the results together. By one agency's estimate, the Gemini model produces about eleven subqueries per question, and deep search mode can make hundreds. OpenAI writes the same about ChatGPT.

Imagine you ask an AI model which client booking system to choose for a small salon. The model does not search for that whole phrase. It separately searches for comparisons of such systems, prices, reviews, integration with messengers. And your site can get into the answer not because it ranks first for the original question. It gets in if it answers one of those small subquestions well.

Step two. Each subquery goes to a search index. An index is a huge database of pages that a search engine has downloaded and processed in advance. And here is an important thing: different AI models have different indexes. Overviews and AI Mode in Google search Google's own index. ChatGPT, according to OpenAI, uses third-party search providers, and which ones exactly it does not officially say. Bing, Microsoft's search engine, is widely known, and the press reported that OpenAI also used a service that collects Google's search results. Perplexity, by its own account, has its own index. And Claude, judging by strong indirect signs, searches through Brave Search, a small search engine from the makers of the browser of the same name. Anthropic has not officially confirmed this.

Step three. The model takes pieces of text from the pages it found and writes an answer from them. The pages that get cited are the ones that come up most often for those subqueries. Microsoft explained separately that AI cuts a page into chunks. So it advises: "Avoid long walls of text". Do not hide answers in tabs and expandable blocks, because AI may simply not see them. And do not keep the main content only in PDF files or in images.

Now let us go back to the theory that they take Google's top results. Ahrefs measured what share of the links in Google's overviews come from the top ten results. In the summer of last year it was about three quarters. In March of this year, already less than forty percent. Almost a third of citations come from pages that are not even in the top hundred. And for ChatGPT, only about a tenth of pages match Google's top ten.

What follows from this? AI does not copy Google's first line. It takes whatever comes up for its own subqueries, and not only from Google. So it matters more for you to be visible on many neighboring questions than to take a single first place. But you cannot do without regular search either. If you are not in the index, AI has nowhere to get you from. Google states plainly in its official guide that "optimizing for generative AI search is optimizing for the search experience, and thus still SEO."

And one more detail of the mechanism that concerns you directly, because your site was most likely built by an agent. The bots that collect pages for ChatGPT, Claude and Perplexity, according to a measurement by Vercel, do not run JavaScript. That is the language that makes buttons work and loads content in the browser. Many modern sites are built like this: the server sends an almost empty page, and the browser fills in the text. A person sees everything as normal. Google sees it too, because its bot does run JavaScript. But ChatGPT's bot gets an empty box. I should note that Vercel itself sells technology that solves this, and the measurement was made at the end of twenty twenty-four. But it is worth checking your site, and I will tell you how at the end.

Now about what works and what does not. And here I will say something that books on promotion rarely say. If you do everything by the book, by a checklist from some article, with the right headings, descriptions and keywords, there will be no traffic. Because next to you there are people who buy links, and they will take all the search demand for themselves. This is not theory, it is a fact.

I have seen such a case myself. A casino bought the domain of a former website of the Embassy of Ecuador. Wikipedia and other authoritative resources once linked to that site. And now the casino sits in the first line of Google's results, and Google does nothing about it.

And this is not a one-off story, it is a whole market. A company that itself trades in links counted these deals. From January twenty twenty-four to May of this year, almost three thousand seven hundred expired domains of government bodies, nonprofits and political campaigns were sold. Four hundred forty-eight of them turned into casino and betting sites. The buyers paid more than twenty-two million dollars for these domains. And the authors explained the mechanism like this: "When a charity or campaign domain expires, the links that other websites gave it do not expire with it. Those links are what these buyers pay for."

Google explicitly forbids this. Since March twenty twenty-four its rules have had a separate point on expired domain abuse, and the example given there is, believe it or not, "Casino-related content on a former elementary school site". So Google knows perfectly well about this trick. Penalties come with spam updates, and this year there were four of them. But how often they actually hit anyone, Google does not publish. Whether to buy links is up to each person, and it is a decision about risk for the whole project. But the main conclusion does not change: in search, authority wins. A checklist without authority will not pull you up.

With AI models the story is similar, only right now they are even easier to fool. In October, researchers showed that eight out of ten AI search engines started citing, within a week, a concept that the scientists had simply made up. One of the platforms cited publications bought for fourteen dollars in less than an hour. But notice how it worked. The planted content went not through a new site, but through already authoritative platforms like Reddit. So even deception works through someone else's authority. And defenses are being built quickly: this year Google wrote a ban on attempts to manipulate AI answers into its rules. And listicles where a brand names itself the best gave unstable mentions in a test by Ahrefs, and hardly helped well-known brands at all.

Now about what gets sold as magic pills. The first is the llms.txt file, a special file "for AI" that supposedly helps models understand your site. Google's John Mueller said: "To me, it's comparable to the keywords meta tag." That is an old service line with a list of keywords that search engines stopped reading long ago. A check of server logs showed that more than a quarter of one hundred thirty-seven thousand sites have such a file, but ninety-seven percent of these files were not requested by anyone at all in May.

The second is structured data markup. This is special code that describes a page for machines: this is a product, this is a price, this is a review. It has no noticeable effect on how often AI models cite you: Ahrefs added it to almost two thousand pages and saw no difference. But do not conclude from this that you should remove it. It helps regular Google show your site more clearly in the results, and regular Google, as we have already found, is still the main source of people.

The third is the famous "plus forty percent visibility" from optimizing for AI. This figure comes from a research paper from twenty twenty-three. Only it was measured in a lab system where five sources were already laid out in front of the model, and that is the best result, not the average. And a review of forty-five research papers, published this summer, concluded that not a single such technique has yet shown a lasting effect across different AI models and over a long period.

So what does work? The same foundation as in Google. The site has to be in the index. The text has to sit right in the page's code. Headings have to be close to the questions people ask: in a large Ahrefs study on ChatGPT, the closeness of a heading to the model's internal subqueries turned out to be the main factor. And you need content of your own. Google's guide directly asks for what others do not have: your own data, your own measurements, your own experience. A retelling of common knowledge, AI can take from anyone, but your own numbers it can only take from you.

And once again we come back to authority. A study by SE Ranking on one hundred twenty-nine thousand sites showed that citation in ChatGPT is best predicted by the number of sites that link to you. And whether a brand is mentioned in AI answers is most strongly linked to mentions on YouTube and brand mentions across the internet in general. This is a correlation, not a proven cause: big brands simply have more of everything at once. But look at whom AI models cite most of all. In Google's overviews, roughly a quarter of all links lead to YouTube, and almost a fifth to Reddit. In ChatGPT, Reddit is in first place altogether. So the formula has not broken, it has expanded: content, links, and also what people say about you on other platforms.

How do you build this authority without buying it? Here are two things you can hand to your agent. The first. Ask it to take ten to twenty questions that your clients actually ask, run them through ChatGPT and through AI Mode in Google, and write down which sites the AI models link to in their answers. That is your map: YouTube, certain sections of Reddit, industry sites, directories. That is where you need to show up, honestly, with useful answers in your own name, not with bought reviews. The second. Ask your agent to gather from your own work whatever can be counted: prices, timelines, common client mistakes, results, and to make a page with those numbers. Other sites link to pages like that more willingly, because this data is not available anywhere else.

One important caveat to everything I have told you today. Almost all the large datasets on search and AI are released by companies that sell promotion tools: Ahrefs, Semrush, Similarweb and others. It is in their interest for the topic to stay hot. And the promotion market really is rebuilding itself around AI: Adobe bought Semrush for almost two billion dollars precisely to help brands be visible in AI answers. That is why I tried to rely first of all on independent experiments.

Here I will say a few words about myself, because for me this is not theory. My project now has about half a million in organic traffic. Organic means people came on their own, not through ads. And of those, about three hundred thousand come from search engines. It is a medical niche, including the American market. And I built this in a year, from scratch, on an empty domain. That is why I speak so confidently: traffic comes not from a checklist in an article, but from understanding how a search engine makes decisions, and from authority, without which that understanding does not work. Those who know how search promotion works are getting traffic now. Those who understand how search through AI works will keep getting it. And the good news is that in many ways it is the same knowledge.

So is it worth spending money on search promotion right now? My answer is yes. Google remains the main source of people, and AI models take their answers from the same index and from the same results for their subqueries. Investment in search works for both channels at once. But it is worth spending it not on reference articles that Google now retells itself, and not on a checklist without authority. And it is better to measure the result in leads and money, not in rankings and visit counts.

And now three technical tasks worth handing to your agent right now. Each of them follows from the mechanism I have described.

The first. Does an AI bot see the text of your site? Ask your agent to download your home page and a couple of important pages the way ChatGPT's search bot does, that is, without running JavaScript, and to show whether the main text is in the code it gets back. If you do not know what your site is built on, that is fine. Just ask the agent whether the server sends the page text right away, or whether the browser fills it in. If it is an empty box, ask the agent to rework the site so that the server sends ready text. This is the most common problem with sites built by an agent, and few people know about it.

The second. Does your site let in the search bots of AI models? There is a trap here. OpenAI has two different bots. One, GPTBot, collects texts for training models. The other, OAI-SearchBot, is for search in ChatGPT. If you block the second one, your site will not be shown in ChatGPT's search answers; this is stated directly in OpenAI's documentation. But you can block training and keep search open. Perplexity and Claude have their own search bots too. Ask your agent to open the robots.txt file, the file with rules for bots in the root of your site, and check the rules for each of them. If you ever asked your agent to protect your site from bots, it may have blocked these as well.

If your site runs through Cloudflare, ask your agent to check its bot settings too. Since July of last year, Cloudflare blocked AI bots by default for new sites, and since mid-September of this year, by all accounts, it lets search bots in again. If your site was set up in that window, this check is a must, because that kind of protection kicks in before a bot even reads robots.txt. And the most reliable check is the server logs. These are the records where the server notes every visit, including visits by bots. Ask your agent to see whether OAI-SearchBot has ever come to you even once. If it has not, ChatGPT simply has nothing of yours to cite. Some hosting providers keep logs only for a short time or give only partial access; your agent will tell you what can be done in your case.

The third. Rebuild your important pages around the way the model breaks up a question. Start by finding out which subqueries it actually asks. Ask your agent to connect your site to Bing Webmaster Tools, a free site owner dashboard in the Bing search engine. Since February of this year it has had a report on how many times your site was cited by Microsoft's AI, and which phrases it used to search for your content. There are no clicks in this report, only mentions, but the phrases themselves are a direct hint. Then have your agent, for each important page, make a list of the neighboring questions a client asks around that topic, and add short direct answers to them on the page, each under its own clear heading. And have it move answers out of tabs, expandable blocks, PDF files and images into the regular text of the page.

Let me sum up. Googling the old way, with a list of ten links, really will over time start to look like a letter on paper. But the search engine is not dying; it is becoming an AI itself. Clicks are leaving first of all from reference questions, and habit is slowing down people's move to ChatGPT, and it will for a long time. AI models do not copy Google's first line, they search by their own subqueries, so the winner is the site that answers neighboring questions and can be read by a bot. In both Google and AI, authority decides: a by-the-book checklist without links, mentions and your own data will not bring traffic. So it is worth spending money on search promotion, but spending it on authority and on what is not in any ready-made answer.

That is all for today. You have been listening to Evgenii Arsentev, creator of Arsminal, a terminal for managing AI agents where your whole agent team works in one window, like chats in a messenger. Founder of ArsentevAI, Dad4Kids and ReelsZavod. Try Arsminal for free at arsminal.com. Thanks for listening.

0:00−33:53 left