📚 Document-Grounded Conversational Assistant (RAG)

This project provides a production-ready, multi-tenant platform for deploying Document-Grounded Conversational Assistants, designed for scalability, resilience, and seamless portability across any cloud environment.

Built on a robust RAG (Retrieval-Augmented Generation) architecture, the system allows multiple users to securely manage their own private document collections and interact with a context-aware AI through a modern web interface.

---

💡 Key Features & Architecture

The goal of this system is to be architected as a secure, robust, scalable, and observable pipeline designed for modern, cloud-native workloads.

1. Kubernetes-Native & Cloud-Agnostic:

   • Ready for Production: The entire application is packaged as a Helm chart, allowing for one-command, reproducible deployments on any Kubernetes cluster.
   • Seamless Cloud Migration: Deploying to a managed Kubernetes service like GKE (Google), EKS (Amazon), or AKS (Azure) is as simple as targeting a different cluster.

2. Modern Frontend & Secure API:

   • A sleek, responsive user interface built with React, TypeScript, and Tailwind CSS.
   • All backend services are exposed via a FastAPI REST API, secured with JWT for user authentication and multi-tenancy.

3. Data Isolation & Cloud-Ready Storage:

   • Multi-Tenancy: Each user's data is strictly segregated in both object storage (S3 user-specific paths) and the vector database (PostgreSQL user-specific collections).
   • Object Storage: Uses a MinIO server emulating the S3 API, enabling portability to any cloud provider's object storage solution.

4. Provider-Agnostic LLM Gateway: Powered by LiteLLM Proxy, providing a unified OpenAI-compatible interface for 100+ providers, load balancing, fallback, and Redis caching.

5. High-Performance RAG Pipeline:

   • Lightweight Vectorization: Uses FastEmbed (ONNX Runtime) for fast, CPU-friendly embedding and re-ranking.
   • Quantized Models: Uses optimized, quantized models for faster inference and lower RAM usage.
   • Advanced Retrieval: Implements Query Expansion and a Cross-Encoder Re-Ranker to ensure only the most relevant context is sent to the LLM
   • Parent Document Retriever (PDR): Combines broad context (Parent chunks in PostgreSQL) with precise retrieval (Child chunks in PGVector).
   • Smart & Dynamic Ingestion: Event-driven indexing that automatically processes only new, modified, or deleted files from the bucket.

6. Continuous Evaluation & Observability:

   • An automated Ragas evaluation pipeline periodically assesses RAG quality.
   • The system is fully instrumented with Prometheus & Grafana for real-time metrics and LangSmith for deep tracing.

---

🛠️ Tech Stack

---

🚀 Quick Start

Choose the method that fits your needs.

⚡ Option 1: The "User Experience"

Best for testing the project immediately without complex configuration.

1. Start the project:

   make
   

   That's it! The system handles security keys generation, database creation, and UI startup automatically.

2. Access the UI:

   • Access the UI at: http://localhost or with make ui.
   • You will be greeted with a login page. Go to the "Register" tab and create your first user account, then Log in with your credentials.
   • Go to the "Settings" tab, chose your LLM model(s) and add you API Key(s).
   • All set! You can now upload documents and interact with your private conversational assistant.

   You can adjust settings to modify the Assistant's behavior or switch models at any time.

Settings:

• Main LLM Model : Name of the LLM model that you will directly interact with via the chat interface.

• Secondary LLM Model : Name of the LLM model that will be used for internal tasks: Query Expansion, Evaluation generation, Data extraction...

• Strict RAG Mode: Determines whether the model can use its internal knowledge or not. If enabled, the model will only use the documents you have uploaded. If disabled, the model can use its internal knowledge (warning: this may lead to hallucinations or unexpected results.).

• Temperature: Controls the randomness of the model's responses. Lower values make the model more deterministic, while higher values make it more creative (warning: higher values may lead to hallucinations or unexpected results.).

• Reranker Threshold: Determines the minimum relevance score for documents to be included in the final response (warning: lower values may lead to unprecise results, while higher values may lead to the assistant missing relevant information.).

---

🔧 Option 2: Advanced Configuration (Dev)

For developers who want to add model providers,tune RAG parameters, change embedding models or storage locations (and more!).

1. Provider Configuration

The LiteLLM Proxy (deployments/litellm/litellm_config.yaml) manages providers. You can add OpenAI, Azure, Anthropic, Google (and other providers supported by LiteLLM) models there.

2. Environment Variables Reference

The service's behavior is controlled by the following environment variables. They must be defined in an .env file at the project root.

Main Application Configuration:

These variables control the behavior of the RAG pipeline and database connections.

---

💻 Access via API

Once the server is running, you can access the API via the /api endpoint.

🔐 Authentication

You can authenticate via the /api/auth/register endpoint.

curl -X POST "http://localhost/api/auth/token" \
	-H "Content-Type: application/x-www-form-urlencoded" \
	-d "username=email@email.com&password=password"

Get your JWT token to interact with the rest of the API via the /api/auth/token endpoint :

curl -X POST "http://localhost/api/auth/token" \
	-H "Content-Type: application/x-www-form-urlencoded" \
	-d "username=email@email.com&password=password"

👤 User Management

You can manage users via the /api/auth/users/me endpoint.

Add your LLM's model name and API Keys with:

curl -X PUT "http://localhost/api/users/me" \
	-H "Authorization: Bearer $TOKEN" \
	-H "Content-Type: application/json" \
	-d '{"llm_model": "model_name", "llm_side_model": "model_name", "api_key": "api_key", "side_api_key": "api_key"}'

View your user information with:

curl -X GET "http://localhost/api/auth/users/me" \
	-H "Authorization: Bearer $TOKEN"

🤖 Interact with your assistant

First, make sure to add documents to your knowledge base by following the document management section below.

You can then interact with your assistant via the /api/chat endpoint.

curl -X POST "http://localhost/api/chat" \
	-H "Authorization: Bearer $TOKEN" \
	-H "Content-Type: application/json" \
	-d '{"query": "Your query here", "temperature": 0.2, "strict_rag": true, "rerank_threshold": 0.0}'

Adjust the settings to your needs. query is required , the others are optional (default values are shown in the example).

---

📂 Document Management

You can manage your knowledge base (PDF, DOCX, MD) in three ways:

1. Via the UI (Recommended): Use the upload and remove features in the web interface.
2. Via API (Automation):
   • First you'll need to register and generate a JWT Token via the /register and /login endpoints : You can then use your $TOKEN with the following commands :
   • To upload a document:

     # Replace with the actual path to your local file
     curl -X POST -F "file=@/path/to/your/document.pdf" http://localhost/api/documents \
     -H "Authorization: Bearer $TOKEN"
     

   • To list all documents:

     curl -X GET http://localhost/api/documents \
     -H "Authorization: Bearer $TOKEN"
     

   • To delete a document:

     # Replace with the name of the file in the bucket
     curl -X DELETE http://localhost/api/documents/document.pdf \
     -H "Authorization: Bearer $TOKEN"
     

3. Via MinIO Console: Access http://localhost/minio (make minio) (User/Pass: minioadmin).
   • Note: If you upload via MinIO, run make ingest to trigger indexing manually.

---

📊 Observability & Evaluation

The project includes a complete monitoring stack:

1. Ragas Evaluation Runner:

   • Runs in the background to evaluate the RAG pipeline (or by using make eval).
   • Generates a synthetic testset from your documents using the LLM_GENERATOR_MODEL.
   • Computes metrics: Faithfulness, Answer Relevance, Context Precision, Context Recall and Answer Correctness.

2. Interfaces:

   • Web App: make ui or http://localhost
   • MinIO: make minio or http://localhost/minio
   • Prometheus: make prometheus or http://localhost/prometheus
   • Grafana: make grafana or http://localhost/grafana
   • PGAdmin: make pgadmin or http://localhost/pgadmin

---

🕹️ Commands (Makefile)

---

Preview Screenshots