Examples integrated into the training and evaluation environment.
02 / LLM grounding & evaluation / 2026
HALLUCINATION GUARD
An OpenEnv-compatible training and evaluation environment for factual grounding, citation discipline, calibration, and refusal behavior.
- Role
- Environment design, reward engineering, API deployment, and documentation
- Duration
- March - April 2026
- Status
- open-source
Datasets unified through the environment.
Signals combined into the reward design.
Eight hallucination categories across five severity levels.
A / EVIDENCE
REAL OUTPUTS, RELEASES, AND SOURCE.
Product captures link to the live or published source. The architecture view is generated only from documented project layers and is not presented as a product screenshot.
SOURCE ↗EVIDENCE / 01
Live Space overview with documented dataset scale, task tiers, reward components, and environment workflow.
B / PROBLEM
What needed to change
Factuality is not a single score. Training and evaluating grounded language-model behavior requires evidence, calibration, citation, refusal, and contradiction signals to be represented together.
C / SYSTEM RESPONSE
What I built
HallucinationGuard combines a large curriculum of examples with a nine-component reward system and exposes reproducible REST and MCP evaluation interfaces.
D / ARCHITECTURE
THE SYSTEM, LAYER BY LAYER.
Each layer below names its responsibility directly, preserving the distinction between repository evidence and explanatory architecture.
Response + evidence
Supplies the claim, sources, citations, and confidence context.
Signal evaluators
Scores factual, semantic, citation, NLI, and calibration signals.
Reward system
Combines nine components into a training or evaluation signal.
REST + MCP service
Provides reset, step, grading, batch, and leaderboard interfaces.
E / ENGINEERING DECISIONS
Multi-signal reward
Grounding quality cannot be represented responsibly by lexical overlap alone.
TRADE-OFFA richer reward is more computationally expensive and requires careful interpretation.
Curriculum ordering
Task families and adaptive difficulty structure how models encounter grounding behaviors.
TRADE-OFFCurriculum design adds another variable to compare across runs.
Session-isolated state
Independent episode state supports reproducible concurrent evaluation.
TRADE-OFFIsolation requires lifecycle handling instead of a single global evaluator.
F / STACK
PythonFastAPIPyTorchHugging FaceDockerOpenEnvMCP
G / LIMITATIONS
WHERE THE SYSTEM STOPS.
- The environment evaluates signals; it does not guarantee that a model response is true.
- Scores remain sensitive to dataset quality, evaluator assumptions, and domain coverage.
- The hosted Space is research infrastructure and should not replace expert review in high-stakes contexts.
H / WHAT I LEARNED
WHAT THE BUILD CLARIFIED.
- Evaluation becomes more actionable when each signal remains inspectable instead of collapsing immediately into one number.
- A reproducible service interface makes the same environment useful for training, batch analysis, and interactive inspection.