Parallel’s Parag Agrawal: Building a New Web for AI Agents

Summary

On Training Data, Parag Agrawal explains why Parallel Web Systems is designing search for agentic workloads instead of human browsing habits. He argues that click data is the wrong signal, that search is a massive matching problem across hundreds of billions of pages and queries, and that Parallel’s system relies on multiple indexes, query rewriting, retrieval, ranking, and incremental crawling. The company launched with a search agent before a full search engine, using longer deep-research workflows to compensate for incomplete index coverage, and later pushed latency down dramatically, including a Turbo product and a 200 ms workflow. Agrawal also says the business model of the web is breaking as agents consume content on behalf of people, and he proposes incentive-aligned, differential pricing informed by Shapley values so publishers can be paid for value delivered to agents.

Notes

• Parallel’s core thesis is that human click data should not train search for agents; agent feedback should.
• The company treats search as a model-compression problem, not a classic consumer-search problem.
• Parag frames web search as a billion-to-billion matching problem.
• The scale is hundreds of billions of pages and hundreds of billions of queries over time.
• Parallel launched a search agent before launching a conventional search product.
• Query-time crawling let the agent compensate for incomplete day-one index coverage.
• Parallel treats index gaps as a latency tradeoff rather than a hard blocker.
• The alternative to agentic search was often outsourced human curation.
• Parallel’s architecture uses multiple indexes over different web subsets.
• The first stage rewrites queries for specialized retrieval systems.
• Parallel uses a fresh index plus other structured retrieval layers.
• The retrieval stack includes big-index, knowledge-graph-like, and structured-index-like components.
• Parallel’s job is to compress a huge corpus into a small ranking output.
• He describes each query as a request for about 1,000 relevant tokens from a trillion-page web.
• Parallel positions itself as a complement to models, not a model lab.
• The company crawls content that is less useful for pretraining but useful for fresh agent work.
• Parallel says better grounding can cut token usage in agent loops.
• The company claims it is faster and more accurate end to end than repeatedly calling Google.
• The agentic search stack is explicitly optimized across quality, cost, and latency.
• Parallel says it improved one product from a three-second read-time budget to 200 milliseconds.
• The new Turbo product is part of that latency push.
• Parag says web search is a forever optimization problem with diminishing returns.
• He sees no single completion metric for the product; the challenge is deciding when more optimization stops paying off.
• Search agents already support long-running research workflows for enterprise-like tasks.
• Initial customer use cases include insurance underwriting and claims processing.
• Other early uses include sales enrichment and financial workflows.
• Parallel says some customers use it to replace overnight human curation with agentic collection.
• The company’s value proposition includes replacing overnight human curation.
• Parallel says agents may make five to 20 searches even without deep research.
• Parag says some prompts can trigger hundreds or even thousands of searches.
• He estimates his own agents may do 100 to 1,000-plus Google-search-equivalent actions per day.
• Agentic search is framed as a compute-allocation problem under quality and latency constraints.
• Parallel’s broader bet is that agents will query the web far more than humans.
• The company’s business thesis is that ad-based web monetization breaks when agents replace human attention.
• Parag says the economics of the web already show stress as agents become the readers.
• He argues the solution is efficient differential pricing for content accessed by agents.
• The pricing model should pay more for extraordinary content and high-value work.
• Parag says content payments may cover inference-time freshness, training data, and liability protection.
• Shapley values are the formal framework Parallel uses for attribution and incentive alignment.
• Parag uses Shapley values to explain how to divide value created by collaboration.
• Exact Shapley computation is too expensive relative to the dollars being allocated.
• Parallel therefore relies on estimation using data and models.
• Parag argues the macro math works if a small slice of LLM inference spend flows to web data.
• He thinks 2% to 10% of inference spend on web data would exceed current standalone web-data business models.
• Parag gives a 12 to 24 month timeline for meaningful payments to many content owners.
• The company’s original name was Shapley Inc. before it became Parallel.
• Parallel’s name reflects the idea of publishing for two audiences: humans and agents.
• Andrew notes that earnings transcripts are already heavily consumed by agents.
• Parag says API docs are already primarily read by customer agents.
• Parallel tests documentation for agent readability, not just human usability.
• Parag describes three stages of the agentic web.
• Stage one is simple tool-using agents.
• Stage two is multi-agent orchestration with sub-agents waking each other up.
• Stage three is a push-based web that notifies agents when something changes.
• The long-term vision is a continuously computed, push-oriented web.
• Parallel wants to allocate compute across the web continuously on behalf of many customers.
• The Google Cloud partnership makes Parallel a grounding option for Enterprise Agent APIs.
• Gemini and other GCP-hosted models can use either Google search or Parallel search for grounding.