For a few years the default answer to "how should my agent find information?" has been: embed everything, stuff a vector database, retrieve top-k. It became furniture in every architecture diagram.
Then a May 2026 paper asked a rude question: what if grep is enough once the model can drive the search loop?
The paper, Is Grep All You Need? How Agent Harnesses Reshape Agentic Search, compares lexical search and vector retrieval inside real agent harnesses. Spoiler: on their long-memory chat QA tasks, grep often wins. And the choice of harness swings scores about as hard as the retriever itself.
What they actually tested
Authors from PwC ran two experiments on a 116-question slice of LongMemEval (long-context conversational memory).
Experiment 1: grep-only vs vector-only across:
- Chronos (their custom harness)
- Claude Code
- Codex CLI
- Gemini CLI
They also varied how tool results were delivered: inline in the context window vs written to files the model reads separately (programmatic delivery).
Experiment 2: same retrieval modes while progressively adding unrelated conversation history as distraction.
This is not a classic IR bake-off with a fixed retrieve-then-read pipeline. The agent gets to search iteratively. That changes the game.
The headline numbers
With inline tool delivery, lexical search beat dense retrieval for every harness-model pair they report. Some gaps were huge. On Chronos with Gemini 3.1 Flash-Lite, inline grep hit 86.2% vs 62.9% for inline vector. On Claude Code with Claude Opus 4.6 the gap was tiny (76.7% vs 75.0%).
Harness effects were wild. The same Claude Opus 4.6 backbone scored 93.1% under Chronos and 76.7% under Claude Code. Swapping the shell moved the ceiling as much as swapping the retriever.
Programmatic (file-based) delivery scrambled the ranking. On several pairs, vector pulled ahead once results were no longer inlined. Codex with GPT-5.4 dropped from 93.1% inline grep to 55.2% programmatic grep. Delivery path is not a footnote.
| Harness + model | Inline grep | Inline vector |
|---|---|---|
| Chronos + Gemini 3.1 Flash-Lite | 86.2% | 62.9% |
| Claude Code + Claude Opus 4.6 | 76.7% | 75.0% |
| Chronos + Claude Opus 4.6 | 93.1% | (harness lift vs Claude Code) |
Delivery path matters too. Codex with GPT-5.4 dropped from 93.1% inline grep to 55.2% programmatic grep in their report.
What this does not mean
Please do not screenshot the title and delete Pinecone.
The authors say this explicitly. Their corpus is multi-session chat with personal facts and time expressions. Answers often hang on verbatim spans. Lexical tools love that. Scientific synthesis over paraphrased abstracts, visual docs, or fuzzy semantic matches can still favor dense retrieval or hybrids.
Also, "grep" in a CLI agent usually means grep plus shell plus prompting culture. Vendor defaults matter. In the distraction scaling grids, Claude Code favored grep for Opus/Haiku across configurations, while Gemini CLI with Gemini 3.1 Pro stayed vector-ahead. Same disk corpus, different inductive bias.
The real lesson for people shipping RAG
If you build applied AI for businesses, three design choices are one system:
| Piece | Question it answers |
|---|---|
| Retriever | Lexical, dense, or hybrid? |
| Harness | How does the agent plan searches and retries? |
| Delivery | Inline snippets, files, or summaries? |
Most roadmaps obsess over the retriever because vendors sell retrievers. This paper says harness and delivery can dominate end-to-end accuracy.
That matches what I see in production agent work. Teams spend weeks on chunking strategy, then wrap the index in a thin tool schema and wonder why Claude Code finds the bug in the repo while their custom agent loops forever. The difference is often the search loop policy, not the embedding model.
What I would do on a new project
Before signing a vector DB contract for an agent that mostly reads tickets, CRM notes, or chat logs:
- Baseline with ripgrep (or BM25) inside the same agent harness you'll ship. Same tools, same prompts, same eval set.
- Measure end-to-end task accuracy, not recall@k in isolation. Agents can issue five searches. Offline retrieval metrics miss that.
- Test inline vs file delivery if your tool outputs are large. The paper shows this switch can invert winners.
- Add dense retrieval only where lexical fails: synonyms, paraphrases, multilingual fuzzy match, "find things like this."
- Budget harness engineering. Prompt packs, retry rules, and result formatting may beat another embedding upgrade.
A practical hybrid default
My current default for text-heavy ops agents:
- Lexical search first for IDs, error strings, names, exact phrases
- Light BM25 or ripgrep over the working corpus for iterative loops
- Dense retrieval as a fallback tool the agent can call when lexical returns junk
- Keep retrieved spans short and inline when possible
- Log every query the agent issues so you can see whether it's thrashing
Hybrid is not "run both always." Hybrid is "give the agent both tools and teach it when each pays off."
Why this paper is timely
Multi-agent desktops and coding CLIs made agentic search mainstream. At the same time, vector infrastructure became an assumed line item. If simple lexical tools win on common memory tasks inside strong harnesses, some of that spend is optional. The higher-leverage investment is the control loop.
That is slightly annoying if you already committed to a complex RAG platform. It is great news if you are a small team trying to ship a reliable internal agent without a research org.
Takeaway
Grep did not kill vector search. Lazy architecture diagrams just got a reality check. Evaluate retrieval inside the harness you will actually run, on the corpus you actually have. Then spend money where the eval says you're weak.
If you want help designing a lean agentic retrieval stack (and an eval that survives demos), book a free discovery call.