AI project by Beat Hutter
Bringing village history out of the archive
Old chronicles, annual publications and photo collections are full of stories about the local community, yet they mostly sit untouched in boxes and PDF files. This project makes the holdings of a local museum easily searchable for residents. Anyone asking about an old farmstead, a street or a historical trade receives reliable answers straight from the original sources alongside the matching photographs.
The project
Built on the real holdings of a local history museum, the system demonstrates how unstructured legacy collections can be unlocked reliably and under tight control for conversational access.
Real archival holdings
The knowledge base covers 2,543 cross-page text passages from 24 historical publications alongside 1,786 catalogued photo records and more than 8,300 indexed people, buildings and locations.
Controlled answer pipeline
An upstream query planner resolves conversational pronouns, a reranker scores candidate passages and a calibrated relevance gate blocks unsupported answers before generation.
Measured and evolving
Every change to indexing, search filters and prompts is benchmarked against a fixed evaluation set of 70 test and negative queries and refined from real usage.
Project status note: We are currently working on a live prototype. The chat runs in production on dedicated infrastructure via web interface and Telegram, but is not yet open to the general public due to copyright and data protection requirements.
RAG system architecture
Queries pass through a multi-stage pipeline of planning, entity resolution, hybrid retrieval and relevance scoring. Source lines are assembled deterministically from the catalogue.
Quality & evals
Traceability
What the system handles today
The application unites historical text research and targeted photo retrieval under a shared interface on the web and on Telegram.
Cross-page contextual chunking
Hyphenated line breaks are repaired, paragraphs are joined across page boundaries and each chunk is enriched with work, chapter and speaker context before embedding.
Dual-mode text search
Queries for people, buildings or street numbers use exact word-boundary register filters, while open thematic questions run through hybrid vector and keyword search.
Role-aware photo gallery
Image search separates depicted subjects from the photographer, supports year filters and returns structured galleries without calling a language model.
Multi-turn context resolution
The planner rewrites follow-up questions containing pronouns such as “When was it built?” or “Are there photos of it?” into standalone search queries.
Honest gap reporting
When the archive holds only partial information for a multi-part question, the system answers the documented part and explicitly states which aspect lacks coverage.
Anonymous quality monitoring
A local log records response latency, hit counts and user ratings without personal data, highlighting unanswered questions and gaps in the collection.
Governance and principles
Reliable handling of historical sources and collection data follows fixed architectural rules:
No claim without archival evidence
If retrieved passages fall below the relevance threshold after reranking, the pipeline stops before calling the language model and states transparently that no records were found.
Code builds the citation line
The language model places only internal reference markers in the text. Application code deterministically builds the visible source line with book title, issue number and page range from the catalogue.
Respect content ownership
Original texts, historical photographs and collection records remain under the museum's control and are kept strictly separate from the public source code repository.
Privacy across images and logs
Operational logs store neither IP addresses nor names. Before opening the photo archive more widely, photographs showing contemporary individuals are reviewed and anonymised for data protection.
Hardened interfaces
The internal API and vector database cannot be reached from the internet. Public access is restricted to the web frontend and the cryptographically verified Telegram webhook.
Measure before changing
Every adjustment to chunking, search weights or prompts runs through the automated evaluation across all four test suites before release.
Roadmap
What runs on the server today, what is currently being refined and what comes next.
In operation
- Hybrid search on Qdrant (2,543 text chunks, 1,786 photo entries)
- Query planner with pronoun resolution and entity register (8,331 entities)
- LLM reranker and calibrated relevance gate
- Deterministic citations for web chat and Telegram
- API protection with rate limiting and anonymous logging
Next
- Privacy measures: review of the photo collection and anonymisation of contemporary individuals in images
- Vector search fine-tuning: refining weights for complex thematic queries
- Portfolio release: staged publication of reviewed architecture and code modules
Then
- Automated OCR pipeline for newly scanned publications and historical minutes
- Incremental register updates when ingesting new archival holdings
- Curator analytics view to inspect unanswered queries and collection gaps
Research
- Local cross-encoder for faster reranking without external API calls
- Multimodal image search using visual embeddings for uncaptioned photos
- Audio indexing for spoken Swiss German oral history interviews
Technology
Operated on a dedicated European virtual server with internal container network isolation and no exposed database ports. Only language model and dense embedding calls go through the OpenAI API.
- Orchestration
- Custom Python service (FastAPI, AsyncIO, Pydantic Structured Outputs) without heavy agent frameworks
- Language model
- OpenAI gpt-4o-mini for query planning, reranking and source-grounded answer synthesis
- Embeddings
- OpenAI text-embedding-3-small (1536D dense) and FastEmbed Qdrant/bm25 (local German sparse vectors)
- Entity register
- In-memory lookup with RapidFuzz (WRatio) and rule-based normalisation for people, buildings and addresses
- Channels
- Streamlit (web interface) and Telegram Bot API via authenticated HTTPS webhook
- Vector database
- Qdrant (v1.16.1, self-hosted in Docker, hybrid retrieval via Reciprocal Rank Fusion)
- Catalogue and logging
- Curated YAML publication catalogue and local SQLite in WAL mode for anonymised operational metrics
- Platform and network
- Linux VPS (Ubuntu 24.04), Docker Compose, Traefik reverse proxy (Let's Encrypt), Nginx for media files
- Quality and release
- Custom evaluation framework (70 test cases across 4 suites) and multi-stage release gate with Gitleaks