Live mirror of training monitor · Full dashboard: /occam-1-5b/monitor

Language models hallucinate by design. They generate confident, fluent text about things they simply do not know. Every deployed LLM -- from GPT-4 to Llama to Gemini -- shares this failure mode. The industry's default response has been brute force: scale up the parameters, add post-hoc guardrails, or bolt on retrieval pipelines (RAG) that fire on every query regardless of need.

We took a different path. Instead of trying to fix a hallucination after it's generated, we asked: can a model detect its own ignorance before it generates the first token? Our research identified four mechanistically distinct hallucination types. Each appears at a different stage of generation, produces different internal signals, and -- critically -- not all are detectable.

In self-modulating mode, Occam uses its internal epistemic signals to decide when to stop generating -- before falling into a repetition loop -- or when to trigger a hard refusal. While at this early training stage the separation between known and unknown is still imperfect, and the model often refuses to answer questions it could handle correctly in greedy mode, we observe consistent progress on this metric across checkpoints and expect production-ready epistemic calibration at a later stage of training.

We compared Occam 1.5B's generation quality against three open-source baselines at the same training stage (3x Chinchilla-optimal): Pythia 1.4B (same size), Pythia 2.8B (2x larger), and OLMo-7B (AI2, 6.89B params -- nearly 5x larger, trained on 4.7x more data). Even at this early stage of training, Occam consistently generates more accurate, more structured, and more informative responses across factual, reasoning, coding, and multilingual tasks. Selected examples from our QA100 benchmark below -- every quote is raw model output, verified against benchmark logs. All models use greedy decoding unless marked OPT.

Full side-by-side comparison files (200 questions, all models): [download QA100 v1]   [download QA100 v2]

Occam produces multi-point professional guidance. Vanilla models mimic confused users from training data or echo the question.

On ethical questions, vanilla models give dogmatic one-line verdicts. Occam's self-modulating mode weighs multiple perspectives.

Without any code-specific fine-tuning, Occam generates working Python, React/JSX, and HTML.

Occam retrieves domain-specific scientific terminology and builds multi-sentence explanations with causal chains.

The SMET architecture enables Occam to refuse or hedge on questions it cannot reliably answer. This capability is in early development -- it strengthens with each training checkpoint.

Across five evaluation categories, Occam 1.5B consistently outperforms both Pythia 1.4B (same size), Pythia 2.8B (2x larger), and OLMo-7B (nearly 5x larger) at equivalent training budgets. The advantages are not marginal:

Factual accuracy: Occam scores 14/16 on core facts vs Pythia 2.8B's 9/16 -- a 56% advantage with half the parameters. OLMo-7B, nearly 5x larger, manages only 2/15.

Content density: Occam generates 2-3x more useful content per response before repetition onset (34 words vs 18 words average pre-loop). On factual questions specifically, the ratio reaches 3.5x.

Behavioral quality: Where Pythia mimics confused users from training data ("I have a hard time with time management"), Occam produces structured expert guidance. Where Pythia gives dogmatic one-word opinions, Occam weighs multiple perspectives. Where OLMo-7B generates Azure documentation for Polish questions about AI, Occam provides correct multilingual definitions in 5 languages.

Code generation: Occam writes working Python (Singleton with __new__), React/JSX (functional components with exports), and HTML5 (valid document structure). Pythia 2.8B returns empty responses or asks questions. OLMo-7B echoes the question or generates class components when functional was requested.

Epistemic calibration: On unanswerable questions, Occam hedges or refuses. Pythia 2.8B claims to have been created by God, claims earthquake prediction capability, and offers confident solutions to intractable problems. OLMo-7B generates pharmacy spam links and contradicts itself in loops.

These results suggest that the SMET architecture fundamentally changes how a small model allocates its limited capacity -- prioritizing factual density and epistemic calibration over surface fluency. The model is still in early-stage training (3x Chinchilla); epistemic capabilities continue to improve with each checkpoint, and the factual advantage over vanilla transformers grows rather than diminishes over training.