OpenAI

OpenAI launched SearchGPT, a search prototype that answers queries directly, on 25 July 2024^[5]. The feature shipped as ChatGPT search on 31 October 2024 for Plus and Team accounts, covering news, weather, stocks, sports and maps, and citing its sources through inline citations and a sources button beneath the answer^[1]. Those citations are what visibility measurement counts, because they are the only place a site appears in the output.

Four user agents carry OpenAI traffic, and they have different jobs. GPTBot collects content that may be used to train the foundation models, and disallowing it in robots.txt signals that a site should not be used for that training^[6]. OAI-SearchBot is the separate crawler that surfaces sites inside ChatGPT's search features, and publishes its address ranges at openai.com/searchbot.json^[2]. ChatGPT-User fetches a page when a person asks ChatGPT or a Custom GPT a question, or through a GPT Action, and OpenAI states it does no automatic crawling^[7]. OAI-AdsBot visits pages submitted as ads on ChatGPT to check they meet OpenAI policy^[3]. Blocking the training crawler therefore does not remove a site from ChatGPT search, a distinction that separates the two robots.txt settings most rules written for AI crawlers treat as one.

OpenAI has bought access to archives rather than relying on open crawling alone. The Axel Springer agreement of 13 December 2023 covers Politico, Business Insider, Bild and Welt, paid content included, and was reported by the Financial Times at tens of millions of euros a year; ChatGPT shows summaries of that content with references and links to the full articles^[8]. Under the Financial Times agreement announced on 29 April 2024, FT articles surface in ChatGPT with summaries, quotes and links, all attributed to the publication^[9]. The News Corp deal announced on 23 May 2024, covering the Wall Street Journal, the New York Post, the Times and the Sunday Times, was reported by the Wall Street Journal at $250 million over five years in cash and credits^[10].

The Times sued OpenAI and Microsoft for copyright infringement on 27 December 2023 over the use of its articles to train and run ChatGPT^[4]. A preservation order of May 2025 required OpenAI to keep every ChatGPT output log it would otherwise have deleted; that blanket obligation was lifted on 9 October 2025, with retention surviving only for accounts the Times has flagged^[11]. On 7 November 2025 the magistrate judge ordered OpenAI to produce 20 million de-identified consumer conversations to the news plaintiffs^[12]. Summary judgment briefing was scheduled to conclude on 2 April 2026, with a ruling expected in the third quarter of that year and a trial possible in 2027 if any claims survive^[13]. The United States Department of Justice has sided with OpenAI and Microsoft, arguing that training on the newspaper's content does not violate copyright law and that the success of the American AI industry is an important national security interest^[14].