The Production RAG Pipeline

Run `CREATE EXTENSION IF NOT EXISTS vector;` in your Postgres instance. Create a documents table with a vector column: `ALTER TABLE documents ADD COLUMN embedding vector(1536);`. Add an HNSW index for fast similarity search: `CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops);`. If you are on Supabase, pgvector is already available — just enable the extension in the dashboard.

Install LlamaIndex: `pip install llama-index`. Use SimpleDirectoryReader to load your documents, then apply a SentenceSplitter with chunk_size=512, chunk_overlap=50. Run documents through an embedding model (OpenAIEmbedding or a local HuggingFace model). Store the resulting nodes and embeddings to pgvector using LlamaIndex's PGVectorStore integration. This gives you structured, searchable chunks with metadata.

Install LangChain: `pip install langchain langchain-community`. Create a PGVector retriever pointed at your Postgres instance and configure it to return the top-k most similar chunks. Build a RetrievalQA chain: retriever → prompt template → LLM. Use `chain.invoke({question: query})` — LangChain handles context injection automatically. Wire in Claude, GPT-4o, or Groq as your generation model.

Tag your document chunks with metadata at ingestion time: source file, date, section, document type. In LlamaIndex, pass `metadata_filters` to your VectorIndexRetriever to scope searches by source or date. This prevents cross-contamination between document sets (e.g., do not retrieve from Q3 docs when the user asks about Q4). Metadata filtering is the most underused RAG lever — add it before you tune embeddings.

Run a golden dataset of 20 question-answer pairs against your pipeline. Check two metrics: retrieval precision (did the right chunk come back?) and answer faithfulness (did the LLM stick to the retrieved context?). If precision is low, reduce chunk_size or tighten your HNSW index parameters. If faithfulness is low, strengthen your system prompt with explicit grounding instructions. Swap pgvector for Pinecone if you need managed vector search at scale.