Reranking Bi-Encoder vs Cross-Encoder | Aspect | Bi-encoder (retrieval) | Cross-encoder (reranking) | |---|---|---| | Encoding | Query and doc encoded separately | Query + doc encoded jointly | | Index time | Precompute doc vectors | No precompute possible | | Query time | ANN search in ms | One forward pass per (query, doc) pair | | Scaling | Millions of docs | Tens to hundreds of docs per query | | Quality | Good recall | Great precision | The standard production pattern: bi-encoder retrieves 50-200 candidates, cross-encoder scores each candidate against the query, keep top 3-10. When to Re…