Vector database

Comparing a query against every stored embedding is fine for a few thousand documents and too slow past that. A vector database uses approximate nearest-neighbour indexes and returns close matches in milliseconds. You give up a little recall and get a lot of speed.

Qdrant, Weaviate, Milvus and Pinecone are dedicated options. You can also add the capability to a database you already run, with pgvector for PostgreSQL or the vector types in Elasticsearch and Redis. Start there if you can.

Most production systems pair vector search with keyword search, since exact strings such as error codes, SKUs and names are what embeddings handle worst.