AI search vs keyword search: when each one wins
Semantic search is not a strict upgrade. Hybrid retrieval beats both, and the reason is more interesting than the benchmark.
Vector search understands that 'time off policy' and 'annual leave' are the same idea. Keyword search understands that 'INV-2291' is exactly one thing and approximate matches are useless. Real questions contain both kinds of term.
Where embeddings fail
- Identifiers: invoice numbers, SKUs, error codes, ticket IDs.
- Rare proper nouns that were not in the training distribution.
- Negation, which embeddings famously smooth over.
- Numeric ranges and dates.
Where keywords fail
- Paraphrase, which is how humans actually ask questions.
- Cross-language documents.
- Conceptual queries like 'what could block this renewal?'
Run both, fuse the ranked lists with reciprocal rank fusion, then re-rank the top 30 with a cross-encoder. It is a boring architecture and it wins consistently.