ScholarRAG
Ask about 200 arXiv papers on visual anomaly detection. Every answer cites the passage it comes from, or says the papers don't contain it.
About this demo and its evaluation results
How it works
- Corpus: 200 arXiv papers on visual anomaly detection, split into 3,910 passages of about 350 words.
- Retrieval: BM25 top 10 ∪ dense top 10 (
bge-small-en-v1.5), reranked bybge-reranker-base; the best 5 passages go to the model. - Answer: Gemini (
gemini-3.5-flash-lite) answers only from those passages, with an exact quote per citation, or refuses. Every quote is checked in code (✓ verbatim).
Evaluation
| Right passage among the 5 retrieved (LLM-written / handwritten questions) | 0.91 / 0.93 |
| Right passage ranked first on paraphrased (handwritten) questions, vs BM25 alone | 0.73 vs 0.20 |
| Acceptable answers, judged by an LLM and checked on a manual sample (64 / 15 questions) | 0.92 / 0.93 |
| Incorrect answers | 0 |
| Near-miss questions (paper in corpus, detail not) refused | 20 / 20 |
| Median time per question on a laptop (Gemini speed varies) | ≈ 3 s |
Known limits
- Built for specific questions. Broad, conversational questions can be refused; naming the topic precisely helps, and "Broaden search" rewrites the question first (slower).
- The corpus is mostly industrial images; other domains such as medical imaging are covered by only a few papers.
By Moh Hasbi Alfarizi. Questions are sent to Google Gemini on its free tier, which Google may use to improve its products, so please don't enter personal information. Passages are short excerpts; follow the paper titles to read the full papers on arXiv.