writing / / 1 min read
Notes on a policy RAG assistant
Design notes on building a retrieval-augmented assistant for internal policy: scoping, private documents and awkward tables.
Notes from building the internal policy assistant.
The pipeline
Detect the question's language, translate if needed, search the vector store, then generate an answer grounded in what was retrieved.
Scope retrieval by department
Policy chat is scoped by department, so people retrieve from the documents they should see. Filtering at retrieval time is safer than asking the model to ignore what it was handed.
Private documents are a different mode
"My Docs" is separate: uploaded files belong to a session and are purged after a time limit, so private material never mixes with organisation-wide policy.
Tables don't survive extraction
Credit policy is full of eligibility tables with conditional logic across loan products. Automated PDF extraction mangled them, so I restructured those tables by hand before indexing. Slower, but it's the difference between a right answer and a plausible one.