Beating Google at Search with Neural PageRank and $5M of H200s — with Will Bryk of Exa.ai

Summary

• Exa is not pitching an answer layer on top of incumbent search; it is rebuilding crawling, neural indexing, retrieval, and serving as a vertically integrated “Google 2.0.” Will Bryk says Google’s latency, cost, and web-scale constraints trap it in an architecture rooted in keywords, while Exa is “neural all the way.” The company now calls itself “the OpenAI of search”: fundamental research underneath, APIs and end-user products above.
• The core technical bet is “neural PageRank”: train on the text surrounding links, hide the destination document, and predict what was cited. This combines relevance with canonicity—a widely referenced Paul Graham essay becomes easier to retrieve under the many different phrases people use to describe it. Unlike conventional PageRank, the model learns both what a document means and how important it is.
• Exa’s new list product turns search from a 400-millisecond lookup into a variable-compute job targeting near-perfect recall. A query such as “startups working on hardware in San Francisco” might preview results, accept feedback on the boundary, then run for minutes, hours, or a day to find all 325 qualifying companies. swyx likens the idea to “o1 for search”: different questions deserve different compute budgets.
• Exa’s proposed moat against ChatGPT Search and Perplexity is ownership of retrieval itself, not merely a web cache plus Bing or Google results. Mapping perhaps 100 billion URLs to stored documents is materially different from building an algorithm that retrieves the right pages across that corpus. Bryk’s quality warning is equally direct: “If slop comes in, slop will go out.”
• The larger thesis is that “superintelligence” remains bottlenecked without “superknowledge.” Even a GPT-5-level system cannot produce a comprehensive list of New York PhDs who have thought about search if Google cannot surface them. Bryk expects a smaller, efficient “intelligence module” to call specialized search rather than inefficiently memorizing the web. As swyx recalls Bryk putting it, even AGI will need search.
• The clearest near-term monetization is B2B list-making across venture sourcing, sales, recruiting, finance, and research. One VC reportedly used Exa for eight-hour stretches over multiple days; Exa itself finds customers and search researchers with its own engine. That threatens information-arbitrage workflows while rewarding public, high-quality work.
• The economic gamble is that falling model costs can outrun neural search’s compute intensity. Exa purchased a $5 million H200 cluster, mixes owned hardware with AWS, preprocesses the corpus, and reranks only perhaps 10, 30, or 100 candidates rather than running LLMs over billions of pages at query time. Bryk argues that a roughly 200× cost decline over a couple of years makes rebuilding search newly rational, though closed platforms and data access remain constraints.

Deep dive

1. GPT-3 made “Google 2.0” look newly possible

• Will traces Exa’s founding to a stark 2021 contrast: GPT-3 could understand subtle paragraphs, while Google might answer “shirts without stripes” with striped shirts. The YC pitch was deliberately audacious—“we are better than Google” and “Google 2.0”—but the founders also brought AI experience and a working demo.

• Metaphor began as a research endeavor around “perfect search,” without committing to a downstream application. Will compares its evolution to OpenAI releasing ChatGPT as a research preview and becoming more product-oriented: Exa conducts fundamental work toward something like “AGI for search,” then commercializes the resulting capabilities.

• The renamed company’s name, Exa, means 10^18, against Google’s namesake 10^100. The provocation—“10 to the 18th is greater than 10 to the 100th”—encodes “smaller is better”: a real query needs the complete 325 matching companies, not a theatrical “30 million results found.”

2. Link prediction turns web citations into neural PageRank

• Exa’s original self-supervised objective took text surrounding a hyperlink, hid the linked document, and asked the model to predict it. Given “check out this really cool aerospace startup” followed by SpaceX, repeated over billions of examples, the system learns to retrieve SpaceX for semantically equivalent queries.

• Will clarifies that Exa does not memorize or generate URL strings; “document prediction” would be the more accurate name. The architecture is Transformer-inspired but different, and the initial base model has since gained synthetic data and supervised fine-tuning for greater control and robustness.

• The same objective functions as “neural PageRank.” If many people cite a Paul Graham fundraising essay, Exa learns its canonicity; because those references can describe it in perhaps a thousand ways, Exa simultaneously learns its meaning. Will’s claim is that this is “PageRank on steroids,” not merely domain authority or backlink counting.

• Training distribution remains decisive: high-quality sources produce better predictions, while poor inputs reproduce poor retrieval. Exa can shape its data toward valuable material, but the categorical lesson applies to answer engines downstream too: “If slop comes in, slop will go out.”

3. Exa owns the full stack to return finite, exact lists

• Building search means discovering URLs, crawling them, processing documents into an embedding or more complex index, and serving that index at high throughput and low latency. Will notes that each layer might occupy hundreds or thousands at Google; Exa commonly assigns “one or two people each.”

• His critique of incumbents combines architecture and economics: Google must cover the web, answer in roughly 400 milliseconds, and keep per-query costs low. Built from a 1998 keyword framework, its enormous system is difficult to reconstruct around LLMs and embeddings; Exa entered without that legacy and found “a lot of low-hanging fruit.”

• The launch promises near-perfect lists for complex requests—every qualifying hardware startup in San Francisco, or every past-year blog post discussing swyx’s podcast. If the true answer contains 325 startups, Exa’s goal is to return all 325, not merely rank a familiar first page.

• Exa is retaining both distribution paths: search infrastructure that developers embed in applications and a direct interface where users can buy credits and run searches immediately. Will again invokes OpenAI’s pairing of an API with ChatGPT.

4. Comprehensiveness becomes a variable-compute product

• Will separates two problems that search must treat differently. Subjectivity requires a threshold and feedback loop—show examples, let the user reject boundary cases, and effectively “create a classifier on the fly.” Comprehensiveness requires continuing to search until that agreed boundary has been exhaustively covered.

• In principle, sufficient compute solves recall: GPT-4 could inspect the entire web and classify whether each page discusses swyx’s podcast. Will says it might take a year and cost a million dollars or more, but the thought experiment establishes a continuum between cheap approximation and exhaustive retrieval.

• Exa therefore exposes a compute budget and previews results before allowing a larger job, although Will says the upper and lower bounds are still being worked out. swyx likens this to “o1 for search”: allocate more test-time work to harder questions rather than giving every query the same fixed latency.

• The resulting interaction breaks a deeply conditioned expectation. Search might take one minute, 10 minutes, or a whole day; for sourcing and research, “writing a search, going to get a cup of coffee, coming back and you have a perfect list” can be an acceptable trade.

5. Superknowledge is the missing tool layer for superintelligence

• The hosts connect comprehensive retrieval to businesses built on information arbitrage: if everyone can request the same high-quality facts through an API, privileged list-building becomes less defensible. Will frames the deeper question as what happens “if knowledge is kind of solved,” distinguishing that world from intelligence alone being solved.

• His canonical example is a GPT-5-level system asked for every New York City PhD who has thought about search. If Google cannot retrieve those people, the model is blocked regardless of its reasoning ability: “We’re literally going to have super intelligences that are using Google.”

• Bryk’s preferred architecture pairs “superknowledge” with superintelligence. A smaller, economical LLM acts as an intelligence module and calls an efficient search tool, rather than scaling until one model memorizes the web; GPT-4-level systems could also call search, but inference economics favor specialization.

6. Owning retrieval—not merely a web cache—is the competitive claim

• Will distinguishes Exa from systems that combine traditional retrieval with an LLM-written answer. ChatGPT Search or Perplexity may crawl pages and maintain their own content stores, but possessing a URL-to-document cache is not equivalent to searching the underlying corpus without relying on Bing, Google, or another retrieval API.

• His scale example makes the distinction concrete: storing perhaps 100 billion crawled URLs in a key-value system is straightforward compared with instantly retrieving the semantically correct subset from 100 billion documents. Only a small number of companies, including Google, Bing, and Yandex, have built a full search engine from scratch.

• Customers also need the pages themselves, so Exa returns cleaned, parsed content—Markdown or chunks—for 10, 100, or 1,000 URLs. The capability began accidentally: the first system retained each page’s first 1,024 tokens for debugging, and customer demand turned that residue into a major product.

• Will concedes that “a good scraper” is easy compared with a great or perfect one, and Exa does not yet have a perfect scraper. Twitter, Reddit, Quora, and Stack Overflow increasingly restrict bots; open long-tail sites still provide much of the value, while larger scale should make paid data partnerships easier.

7. Lists monetize across research, sales, recruiting, and investing

• Venture firms can search for every company in AI-for-law, AI-for-construction, or a target’s competitive set. One investor reportedly ran Exa queries for eight hours at a time over many consecutive days; sales teams use the same machinery for prospecting, and Exa found many of its own customers that way.

• Recruiting follows the same pattern: Exa can surface people who have written about search across blogs or indexed tweets. Will’s practical implication is that candidates must make themselves web-discoverable; if someone does not publish or otherwise make their work discoverable, the engine has less evidence from which to infer expertise.

• In education, one customer retrieves roughly 20 relevant research papers, then feeds them to an LLM writing assistant. A host jokes that it is “literally just doing their homework,” while Will keeps the distinction: research is hard, and high-quality source material is the necessary input to whatever assistance follows.

• swyx frames the broader effect as: “Whatever the search engine optimizes for is what the internet looks like.” Will argues that Google’s keyword incentives encouraged keyword stuffing, whereas semantic retrieval could reward genuinely useful publication.

8. LLMs become the interface, but users still need steering

• A live search for “people writing about learning in public” exposes Exa’s remaining ambiguity: results drift toward public schools rather than swyx’s movement. Google also struggles, but Will’s answer is revealing—the correct interface is an LLM that knows the user and translates shorthand into the detailed paragraph the search engine actually needs.

• Humans are too lazy to specify every nuance; “an LLM is not lazy.” Google and Bing were designed for short human keywords and completely fail on paragraphs, while Exa is being built for precise, machine-generated queries in a world where LLMs mediate most interfaces.

• Will distinguishes objective filtering—does this page satisfy the request?—from subjective ranking—what does “best” mean? A list of all qualifying companies can be verified against criteria; “best NBA players” should ideally become an explicit metric such as shots scored per game, employee count, or site traffic.

• swyx’s pushback is that fully autonomous agents often fail because users want visibility and steering, as in coding copilots versus level-five autonomy. Will responds with previews and iteration: first confirm what the request means, then launch the long batch job. The discussion identifies failures both from weak agents and from missing human context that no system prompt fully captures.

9. o1-style training and falling inference costs widen the bet

• Will calls o1 a major step because verifiable rewards let a model bootstrap: generate many programming or math tasks, have a grader identify successful solutions, then train on those successes. o1 combines a smarter self-taught base model with inference-time thinking that may last minutes or hours; he believes the training paradigm will spread, though many model sizes and vendors will remain.

• When swyx proposes collecting graduate students’ real search trajectories as reasoning data, Will pushes back. Human labeling of niche biological or technical results proved hard even for Exa’s engineers; human labelers tend to fall back to keyword matching, while GPT-4 was better at judging meaning. Search asks “simple questions about billions of things,” unlike a GPT-5 solving one PhD-level problem.

• Exa has purchased a $5 million H200 cluster and updated its existing A100 capacity with it, while mixing owned compute with AWS. The clusters support training and research; query-time economics depend on preprocessing the corpus, then applying more expensive reranking only to perhaps 10, 30, or 100 candidates.

• swyx frames ad revenue as a ceiling on consumer-search cost, floating roughly one cent per thousand views while explicitly disclaiming the exact figure. Will’s counterweight is the pace of model economics: if a core tool becomes roughly 200× cheaper within a couple of years, failing to “rethink search from scratch” leaves a large new design space untouched.