LlamaIndex Search Tool Integration Patterns
Using a search API as a retriever or query-engine data source in LlamaIndex.
How it works
LlamaIndex is built around the retriever abstraction, and a search API can be wrapped as a custom retriever that implements the same interface as a vector-store retriever — given a query string, it returns a list of `NodeWithScore` objects. That lets a live search API sit interchangeably alongside, or downstream of, a static document index in the same query engine pipeline, rather than requiring separate glue code.
Example
A custom retriever's _retrieve method calls the search API, then wraps each result as NodeWithScore(node=TextNode(text=snippet, metadata={"url": url}), score=rank_position), letting the rest of the LlamaIndex query engine (response synthesis, citation formatting) treat live search results exactly like nodes pulled from a local vector index.
Pitfalls
- Assigning an arbitrary or uniform score to every search result defeats any downstream reranking step that expects scores to reflect actual relevance.
- Mixing a live search retriever with a static vector retriever in the same `RouterQueryEngine` without a clear routing rule means the system can't explain why it chose one source over the other for a given query.
- LlamaIndex's citation formatting assumes a node has stable metadata; a search API that returns a slightly different snippet on a retried call can produce inconsistent citations across runs of the same query.