EnglishSvenska
AI searchSwedenAI crawlers
Written by · Lead Developer
GEO · Nordic markets

Swedish is 0.61% of the training data. Then Swedish media blocked the retrieval bots too.

Four in ten Swedes use AI tools, and more than half of them would rather ask a chatbot than Google. What they rarely ask is where the answer came from. The Swedish evidence base is thin to begin with, and the Swedish sites that could fill the gap have largely shut the door. The result is that the AI will happily answer you in Swedish while building that answer out of something else.

George, the Baseline Labs mascot

Nought point six one percent

CulturaX is one of the cleaned multilingual text mountains used to train language models: 6.3 trillion tokens across 167 languages. English accounts for 2,846,970,578,793 tokens, or 45.13% of the whole thing. Swedish accounts for 38,486,181,494. That is 0.61%.

The ratio is roughly 74 to 1. For every Swedish sentence the model has seen, it has seen about seventy-four English ones. That does not make Swedish a low-resource language in the way Faroese is, but it decides what the model has read most about when it comes to answer a question on mortgages, insurance, or which robot vacuum tested best.

45.13%
of CulturaX is English
0.61%
of CulturaX is Swedish
74×
more English text than Swedish

"But Swedish Wikipedia is enormous"

True, and it is the best argument against the whole thesis. Swedish Wikipedia has 2,626,776 articles and is the fifth largest edition in the world, ahead of French and far beyond what ten million people should reasonably produce.

The reason is a bot. Lsjbot, built by Sverker Johansson, mass-produced articles about species and geographic features. When Swedish Wikipedia passed one million articles in 2013, the bot had written 454,000 of them, close to half the entire corpus. The same bot is why Cebuano, a language of the Philippines, now has the second largest Wikipedia at 6.1 million articles.

Volume is not the same thing as depth. A stub about a beetle is an article in the statistics but not much to ground an answer on. Sweden has quantity without necessarily having the kind of content a model leans on when someone asks about an industry.

2.6M
articles on Swedish Wikipedia
5th
largest Wikipedia edition
454,000
articles written by a single bot by 2013

14 of 23 Swedish news sites block OpenAI's retrieval bot

A thin evidence base could be offset by Swedish media showing up in the answers. So we read the robots.txt of 24 Swedish news and business sites on 5 August 2026. One could not be fetched. Of the remaining 23, fourteen block OAI-SearchBot outright.

That is the number that matters, because OAI-SearchBot is not the training crawler. It is the bot that fetches pages when ChatGPT is answering a question right now. Block it and you do not exist as a source in the answer, however good your content is.

SiteOwnerGPTBot (training)OAI-SearchBot (answers)PerplexityBot
svt.seSVTBlockedAllowedAllowed
aftonbladet.seSchibstedAllowedAllowedBlocked
svd.seSchibstedAllowedAllowedBlocked
dn.seBonnier NewsBlockedBlockedBlocked
expressen.seBonnier NewsBlockedBlockedBlocked
di.seBonnier NewsBlockedBlockedBlocked
sydsvenskan.seBonnier NewsBlockedBlockedBlocked
nyteknik.seNy Teknik NewsBlockedBlockedBlocked
omni.seSchibstedAllowedAllowedAllowed

One detail worth noting: Aftonbladet and SvD explicitly block Google-Extended, CCBot, ClaudeBot and PerplexityBot, but have no rule for GPTBot at all. It gets in by default, through the * rule. That looks less like a policy and more like a list somebody filled in once.

The pattern follows ownership. Bonnier's titles close everything: DN, Expressen, Di, Sydsvenskan and HD block both training and retrieval bots, and Ny Teknik does the same outside that group. Schibsted's titles let OpenAI in while shutting Perplexity out. And SVT makes the distinction that is actually considered: training bots refused, answer bots welcomed. SVT was the only site in the sample that explicitly allowed all three retrieval bots.

14 of 23
block OpenAI's retrieval bot entirely
18 of 23
block the training bot GPTBot
1 of 23
explicitly allows every answer bot

Blocking training and disappearing from answers are two different things

This is the most expensive misunderstanding in the industry right now, and an easy one to make. GPTBot, CCBot and Google-Extended collect text for training. Closing them is a reasonable copyright negotiating position and costs you nothing in visibility.

OAI-SearchBot, ChatGPT-User and PerplexityBot do something else entirely. They fetch the page the moment a user asks a question, in order to cite it. Block those and you have not protected your material, you have only ensured somebody else answers in your place. We have written more about why "block all AI bots" now costs you visibility.

Most of the fourteen sites above have in all likelihood made a deliberate decision about training data and taken the retrieval bots along with it. The effect is the same either way: when somebody asks ChatGPT about Swedish business, Dagens industri is not in the room.

We built our own model, and the size tells the story

Sweden has not ignored the problem. AI Sweden, with RISE and WASP, built GPT-SW3 in six sizes from 126 million to 40 billion parameters, explicitly to generate text in Swedish and the Nordic languages.

The interesting part is the scale. The entire training set is 320 billion tokens, and it has to cover Swedish, Norwegian, Danish, Icelandic, English and program code together. The English portion of CulturaX alone is 2.85 trillion tokens, close to nine times as much text for a single language. The number says less about GPT-SW3 than about what a small language has to work with.

What it means if you run a Swedish company

The conclusion is not to give up. It is that the Swedish gap is an opening, because few people are filling it.

  1. Check your own robots.txt first. Open it and look for OAI-SearchBot, ChatGPT-User and PerplexityBot. If any of them carries Disallow: /, you have made yourself invisible in AI answers without necessarily meaning to. It takes five minutes to correct.
  2. Write the Swedish material that is missing. When 0.61% of the text is in your language, every substantial Swedish page about your industry is disproportionately valuable. Concrete numbers, prices, comparisons and dates, not marketing prose.
  3. Expect English sources to compete for Swedish questions. Your competitor in the AI answer is not always another Swedish company. It can just as easily be an international site that happened to write about the subject in more detail.
  4. Start with ChatGPT. As we have shown before, ChatGPT dominates Swedish AI search by a wide margin. That is where the effect shows up first.

The first step is knowing how you are mentioned today, and what language the sources behind that answer are written in. Most Swedish companies have never checked.

See how AI describes your brand

Questions we get about the Swedish source picture

Does 0.61% mean the AI is bad at Swedish?
No. Modern models transfer knowledge across languages, so the Swedish itself is usually fine. What it affects is which sources and which facts the model has seen most of. The language in the answer can be flawless while the content rests on an English page about a different market.
You measured robots.txt, but have you measured how often .se sites are actually cited?
Not yet, and we would rather say so plainly than imply something we have not measured. Our measurement here covers which bots 23 Swedish sites let in, not the final citation share in answers. Citation share for Swedish-language queries is the next study we want to run, and as far as we know it is not published anywhere today.
Should I let the training bots in as well?
That is a business decision, not a technical one. Blocking GPTBot and Google-Extended costs you no visibility in today's AI answers. Blocking OAI-SearchBot does. If you want to hold the line on copyright you can do it without disappearing, exactly as SVT does.
Can your robots measurement be reproduced?
Yes, and that is the intention. We fetched /robots.txt from 24 domains on 5 August 2026 and parsed grouped User-agent blocks, taking the exact agent name ahead of the * wildcard. Sveriges Radio did not respond at fetch time and is therefore excluded. Robots files change, so the figures apply to that date.

Sources: CulturaX (Hugging Face), List of Wikipedias (Wikimedia Meta), Lsjbot, WASP, AI Sweden, GPT-SW3 40B model card and Svenskarna och internet 2025 (Internetstiftelsen). The robots.txt measurement is Baseline Labs' own, carried out on 2026-08-05.

George
Online
0%