HAKARI-Bench is a lightweight information retrieval benchmark for comparing dense, sparse, late-interaction, reranking, and lexical retrieval models under unified evaluation conditions.
It evaluates models across more than 35 IR benchmarks and 500 tasks using a consistent metric suite. The compact Nano-set design makes it practical to compare model quality from multiple perspectives, including languages, domains, retrieval architectures, reranking methods, quantization, truncation, and efficiency settings.
You can get started using hakari-bench.
| Overview | |
|---|---|
| 📈 Leaderboard | Interactive model leaderboard across HAKARI-Bench tasks |
| 📄 Paper | Benchmark design, validation, and evaluation methodology |
| Get Started | |
| 🏃 Quick Start | Install HAKARI-Bench, run an evaluation, build DuckDB, and open the viewer |
| 🧭 Evaluation Policy | Prompts, runtime settings, variants, reranking, and coverage requirements |
| 🛠️ Evaluation Runbook | Runnable evaluation, DuckDB, synchronization, and viewer commands |
| Explore | |
| 🌐 Benchmark Scope | Coverage across benchmarks, tasks, languages, and domains |
| 📋 Task Documentation | Public documentation for benchmark groups and individual tasks |
| 📐 Leaderboard Metrics | Metric semantics and quality-efficiency interpretation |
| 🗄️ DuckDB Schema | Leaderboard warehouse schema and query semantics |
| Contributing | |
| 🤖 Submitting Model Results | Evaluate a model and prepare a leaderboard result submission |
| 📦 Contributing Results | Result layout, validation, and Hugging Face submission workflow |
| 🧩 Custom Model Backends | Integrate non-standard dense, sparse, reranker, and late-interaction models |
