What does RAGReader actually do?

You add a document (PDF, URL, or raw text), ask a question, and get an answer from Dense (vector) retrieval. Clicking that answer opens a deep analysis that re-runs the same question through every combination of retrieval method and LLM you select — up to nine pipelines — and scores each one against ground truth.

What document formats can I upload?

A PDF file, a web page URL (the HTML is fetched and stripped to text), or text pasted directly into the form.

Which retrieval methods and LLM models are benchmarked?

Dense retrieval (openai/text-embedding-3-small), Sparse retrieval (BM25), and Hybrid retrieval (dense + sparse candidates reranked by cross-encoder/ms-marco-MiniLM-L6-v2). Each runs against GPT-4o mini, Gemini 3 Flash, and Claude Haiku 4.5 via OpenRouter.

Which evaluation metrics are reported?

Retrieval quality: Precision@K, Recall@K, F1@K. Answer quality: ROUGE-L (precision, recall, F1), plus Faithfulness, Answer Relevance, and Answer Coverage judged by Mistral Nemo.

How is Ground Truth established for evaluation?

Via Manual Selection (ticking relevant chunks by hand) or Candidate Pooling (fusing retriever rankings with Reciprocal Rank Fusion: score = sum of 1 / (60 + rank)).

Is RAGReader open source?

Yes, RAGReader is open source under the MIT License and can be self-hosted using Docker Compose.

RAGReader — Compare Dense, Sparse & Hybrid RAG Pipelines

RAGReader — compare RAG pipelines on your own documents RAGReader needs JavaScript to run. It answers questions about a PDF, a URL, or pasted text using Dense (vector) retrieval, then re-runs the same question through Dense, Sparse and Hybrid retrieval against GPT-4o mini, Gemini 3 Flash and Claude Haiku 4.5 — up to nine pipelines — and scores every one with Precision@K, Recall@K, F1@K, ROUGE-L and LLM-judged faithfulness, answer relevance and answer coverage. The project is open source (MIT): github.com/thirza258/RagReader