Author: Metehan Yeşilyurt
Profile: https://www.linkedin.com/in/metehanyesilyurt/
ChatGPT retrieval system LEAK alert 💥
I found "something huge" in ChatGPT's server-sent events over the weekend.
Inside the stream is a detailed debug view of ChatGPT's web retrieval system. Which queries it wrote, which engines it called, what came back, scoring objects,what it fetched, how it split each page, and what finally made it into the answer.
I have been pulling this data for a few days. On Saturday I started getting rate limits, so we were (maybe) the most active users of that stream this weekend:)
A few things I can share today:
One question is never one search. Yes, we know there are fanouts. But actually more fanouts behind the scenes!
For a single "best AI visibility tools" prompt, ChatGPT ran 5 search rounds, wrote 18 different queries(hidden queries), made 50 engine calls, pulled 228 results, fetched 223 URLs with selected chunks, and cited 16. You see 16 links.
✍ Let's start today with renderer. How actually ChatGPT uses your page for retrieval.
Your meta tags travel with every result.
There is a separate og_data object in the payload and it is empty on every single result. The raw meta_tags list is what is actually kept.
The page body is not HTML. It is a markdown-like text render. I compared its fingerprints against the common HTML to text parsers. Two-space "* bullet", "* * " for horizontal rules, "# heading" and "--- | ---" table separators with no outer pipes all match the Python html2text library. Turndown, markdownify and Trafilatura each match only one or two of those. So html2text is the strongest candidate, but it is not a stock build.
That render is what gets cut into blocks of roughly 170 words and scored. The model does not see your full page. It sees one to three of those blocks per source.
The images below is not a mockup. Every code block is copied from the stream as is, including our own Peec AI product page.
What I showed above is a small slice. One prompt produces a 150,000-line JSON dump, and the two fields in the image are maybe 2 percent of it.
The same stream also carries:
Every rewritten query, and which of roughly ten internal engines each one was sent to (web, news, Wikipedia, Reddit, arXiv, YouTube, PDF and more)
📍 A per-result score, plus a score object that breaks that score into its components
📍 A per-chunk score for every block of every fetched page, and which blocks were kept for the prompt
📍 A should_fetch decision on each result, with crawl date and publication date
📍 A second ranking pass done in the model's reasoning, where domains are re-ordered before the answer is written
📍 The exact prompt the model receives, with the word budget it is given per source
📍Separate result types for shopping and local queries, with their own fields
If you work on GEO or AI visibility, this is the closest look at ChatGPT's retrieval pipeline I have seen. MORE TO COME.
Follow David Konitzny, Jan Ehrlinspiel, Tomek Rudzki, Malte Landwehr there is a lot more coming THIS WEEK.
I found "something huge" in ChatGPT's server-sent events over the weekend.
Inside the stream is a detailed debug view of ChatGPT's web retrieval system. Which queries it wrote, which engines it called, what came back, scoring objects,what it fetched, how it split each page, and what finally made it into the answer.
I have been pulling this data for a few days. On Saturday I started getting rate limits, so we were (maybe) the most active users of that stream this weekend:)
A few things I can share today:
One question is never one search. Yes, we know there are fanouts. But actually more fanouts behind the scenes!
For a single "best AI visibility tools" prompt, ChatGPT ran 5 search rounds, wrote 18 different queries(hidden queries), made 50 engine calls, pulled 228 results, fetched 223 URLs with selected chunks, and cited 16. You see 16 links.
✍ Let's start today with renderer. How actually ChatGPT uses your page for retrieval.
Your meta tags travel with every result.
There is a separate og_data object in the payload and it is empty on every single result. The raw meta_tags list is what is actually kept.
The page body is not HTML. It is a markdown-like text render. I compared its fingerprints against the common HTML to text parsers. Two-space "* bullet", "* * " for horizontal rules, "# heading" and "--- | ---" table separators with no outer pipes all match the Python html2text library. Turndown, markdownify and Trafilatura each match only one or two of those. So html2text is the strongest candidate, but it is not a stock build.
That render is what gets cut into blocks of roughly 170 words and scored. The model does not see your full page. It sees one to three of those blocks per source.
The images below is not a mockup. Every code block is copied from the stream as is, including our own Peec AI product page.
What I showed above is a small slice. One prompt produces a 150,000-line JSON dump, and the two fields in the image are maybe 2 percent of it.
The same stream also carries:
Every rewritten query, and which of roughly ten internal engines each one was sent to (web, news, Wikipedia, Reddit, arXiv, YouTube, PDF and more)
📍 A per-result score, plus a score object that breaks that score into its components
📍 A per-chunk score for every block of every fetched page, and which blocks were kept for the prompt
📍 A should_fetch decision on each result, with crawl date and publication date
📍 A second ranking pass done in the model's reasoning, where domains are re-ordered before the answer is written
📍 The exact prompt the model receives, with the word budget it is given per source
📍Separate result types for shopping and local queries, with their own fields
If you work on GEO or AI visibility, this is the closest look at ChatGPT's retrieval pipeline I have seen. MORE TO COME.
Follow David Konitzny, Jan Ehrlinspiel, Tomek Rudzki, Malte Landwehr there is a lot more coming THIS WEEK.
Images:
