Advanced · 20 minutes

How to Eliminate Hallucinations in Enterprise RAG Systems

Build production-grade Retrieval-Augmented Generation using hybrid vector search, knowledge graph validation, and citation-enforced decoding.

Step 1: Combine Dense Vector Embeddings with Sparse BM25 Keywords (Hybrid Search)

Pure vector similarity fails on exact serial numbers, legal statute codes, and part identifiers. Combine reciprocal rank fusion (RRF) between dense vectors and BM25.

function reciprocalRankFusion(
  denseRanks: Map<string, number>, 
  sparseRanks: Map<string, number>, 
  k = 60
): Map<string, number> {
  const scores = new Map<string, number>();
  const allDocIds = new Set([...denseRanks.keys(), ...sparseRanks.keys()]);

  for (const docId of allDocIds) {
    const rDense = denseRanks.get(docId) ?? 1000;
    const rSparse = sparseRanks.get(docId) ?? 1000;
    const score = (1 / (k + rDense)) + (1 / (k + rSparse));
    scores.set(docId, score);
  }

  return scores;
}

Step 2: Implement Cohere / BGE Reranking

Retrieve top-50 candidates using hybrid search, then pass them through a cross-encoder reranker to extract the top-5 most semantically grounded passages.

pip install cohere sentence-transformers

Step 3: Enforce Citation Markers in Prompt Constraints

Mandate that the model cite document IDs for every factual assertion. Post-process the output with a deterministic verifier.

const SYSTEM_PROMPT = `
You are a factual enterprise research engine.
RULE 1: Only answer using facts explicitly stated in the [CONTEXT] blocks below.
RULE 2: For every statement, include a bracketed citation tag corresponding to the document, e.g. [DOC-12].
RULE 3: If the provided documents do not contain the answer, reply verbatim: "INSUFFICIENT CONTEXT". Do not extrapolate.
`;