A patient-centric clinical Retrieval-Augmented Generation (RAG) system for querying unstructured medical corpora with low-latency re-ranking and context verification.
Repository: https://github.com/Krish6115/Doc_Assist
Status: Live
Clinical documentation, medical reference guides, and EHR summaries are dense, unstructured, and often difficult for non-specialists to navigate. Standard semantic search over large medical datasets frequently yields low-precision results due to domain-specific jargon, fragmented context, and irrelevant passages entering the LLM context window.
Doc_Assist addresses this challenge by implementing a two-stage retrieval pipeline:
- Dense vector similarity retrieval
- Cross-encoder re-ranking
Instead of directly passing the highest vector matches to the LLM, the system first retrieves a broader candidate set and then re-ranks those candidates using a cross-encoder. This significantly reduces irrelevant context, minimizes hallucinations, improves grounding, and optimizes prompt token usage.
+-----------------------------------------+
| Next.js Client |
+-----------------------------------------+
|
HTTP POST /api/v1/query
|
v
+-----------------------------------------+
| FastAPI Backend |
+-----------------------------------------+
|
+-----------+-----------+
| |
v v
+---------------------+ +-------------------------+
| Dense Vector Search | | Cross-Encoder Re-Ranker |
| FAISS + MiniLM | | Top-k -> Top-n |
+---------------------+ +-------------------------+
| |
+-----------+-----------+
|
v
+-----------------------------------------+
| LangChain Orchestration |
| Prompt + Retrieved Context + Guardrails |
+-----------------------------------------+
|
v
+-----------------------------------------+
| Llama 3.1 Inference |
+-----------------------------------------+
|
v
+-----------------------------------------+
| Structured Clinical Response |
+-----------------------------------------+
Users submit a natural language medical question through the Next.js frontend.
- Query embedding generated using all-MiniLM-L6-v2
- Search performed against a local FAISS vector index
- Retrieves the top-20 candidate passages (k = 20)
Candidate passages are evaluated using a cross-encoder model.
The highest-quality top-4 passages (k = 4) are selected based on semantic relevance rather than embedding similarity alone.
LangChain assembles:
- User query
- Retrieved passages
- Prompt template
- System guardrails
into a grounded prompt.
Llama 3.1 synthesizes a structured, patient-friendly response using only the verified context.
- Next.js (App Router)
- TypeScript
- Tailwind CSS
- Python 3.11
- FastAPI
- Uvicorn
- LangChain
- Meta Llama 3.1
- FAISS
- Hugging Face Sentence Transformers
all-MiniLM-L6-v2
cross-encoder/ms-marco-MiniLM-L-6-v2
Evaluated:
- PGVector
- Pinecone
- FAISS
Selected FAISS for local, in-memory vector indexing.
Given the target corpus size (<50,000 chunks), FAISS delivers:
- Sub-10 ms retrieval
- Zero network latency
- No managed infrastructure dependency
- Low operational complexity
- OpenAI
text-embedding-ada-002 all-MiniLM-L6-v2bge-small-en
Selected all-MiniLM-L6-v2.
Benefits include:
- ~12 ms CPU encoding
- Zero API cost
- Strong biomedical retrieval quality
- Lightweight deployment
Direct Top-k retrieval often returns passages sharing generic medical terminology while missing the actual answer.
- Retrieve Top-20 candidates using FAISS
- Re-rank candidates
- Keep only Top-4 passages
Latency
- Reduced overall pipeline latency by approximately 25%
- Context size reduced from roughly 3,500 tokens to <1,000 tokens
Answer Quality
- Lower hallucination rate
- More relevant citations
- Better grounding on medical QA benchmarks
- Node.js 18+
- Python 3.10+
- GPU optional (supports CPU inference or Ollama/local Llama 3.1 deployment)
git clone https://github.com/Krish6115/Doc_Assist.git
cd Doc_Assistcd backend
python -m venv venv
# Linux / macOS
source venv/bin/activate
# Windows
venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .envLLM_API_KEY=your_llama_3.1_api_key_or_endpoint
EMBEDDING_MODEL_NAME=sentence-transformers/all-MiniLM-L6-v2
FAISS_INDEX_PATH=./data/faiss_indexStart the backend:
uvicorn main:app --reload --port 8000cd ../frontend
npm install
cp .env.example .env.localNEXT_PUBLIC_API_BASE_URL=http://localhost:8000/api/v1Start the frontend:
npm run devPOST
/api/v1/query
{
"query": "What are the contraindications for taking metformin alongside standard renal function impairment?",
"top_k": 4
}{
"query": "What are the contraindications for taking metformin alongside standard renal function impairment?",
"answer": "Metformin is contraindicated in patients with severe renal impairment, specifically defined as an estimated glomerular filtration rate (eGFR) below 30 mL/min/1.73 m². Starting metformin is not recommended in patients with an eGFR between 30 and 45 mL/min/1.73 m² without close monitoring due to increased risk of lactic acidosis.",
"retrieved_sources": [
{
"chunk_id": "doc_942_chunk_3",
"source": "Clinical_Pharmacology_Guidelines_2024.pdf",
"relevance_score": 0.942
},
{
"chunk_id": "doc_108_chunk_1",
"source": "EHR_Summary_Protocol.pdf",
"relevance_score": 0.887
}
],
"metrics": {
"retrieval_time_ms": 14.2,
"rerank_time_ms": 28.5,
"generation_time_ms": 612.0,
"total_latency_ms": 654.7
}
}No role-based access control (RBAC) or clinical privacy enforcement is currently implemented.
HIPAA/GDPR compliance modules are disabled in local development.
The in-memory FAISS index limits corpus size based on available system memory (approximately 500,000 passages).
Responses are generated synchronously, resulting in longer UI loading states for lengthy generations.
- Streaming responses using Server-Sent Events (SSE) or WebSockets to reduce Time-to-First-Token (TTFT)
- Distributed vector store using Qdrant or Milvus for large-scale, multi-tenant deployments
- Hybrid retrieval combining BM25 keyword search with dense embeddings for improved ICD-10 and CPT code matching
This project is distributed under the MIT License.
See the LICENSE file for more information.