Interactive field note · AI evidence
When Should an AI Insight Stop? Four Evidence Stress Tests
Change a detail in an invented workout record and watch which conclusions survive. This is a transparent test of evidence rules—not a claim that an AI model or health outcome has been validated.
A convincing sentence can still have the wrong inputs
An app may state a number with confidence while one input was expressed in different units, shown under another time zone, copied twice or not observed at all. A useful evidence check asks whether the sentence survives those changes. It does not ask whether it sounds confident.
For this demonstration, imagine two workout-duration records: A is 45 minutes and B is 60 minutes. They are invented examples, not Nutrinaut users. When both are directly observed, recent and comparable, the narrow statement “these two logged durations differ by 15 minutes” follows from arithmetic. They do not show that someone recovered, became fitter, should train harder or represents a population trend.
- 01 · InputSource, time and duration
- 02 · RuleNormalize; check gaps and claim scope
- 03 · DecisionDescribe only—or abstain
All scenarios below are synthetic. The controls run deterministic JavaScript in your browser. They send nothing to Nutrinaut or a model provider. This page does not accept or inspect your personal records.
Try changing the evidence
Start with the same two invented records. In each card, change one representation or evidence condition. A green result means only that the narrow, typed 15-minute difference is supported by these two records; amber means the checker withholds it or rejects a stronger claim.
45 minutes or 2,700 seconds?
Changing the unit should not change elapsed duration.
A: 45 min · B: 60 min
Changing the unit does not change the observation.
A new date, or a new event?
The same instant can display on another calendar day. A timestamp without its offset is ambiguous.
A: 11 Oct 23:50 UTC · B: 12 Oct 06:00 UTC
The timestamp carries an unambiguous instant.
Does a high confidence number fill a gap?
A declared 99% confidence cannot replace a missing observation or make an old one current.
A: observed 45 min · B: observed 60 min
Two directly observed, recent durations are available.
Two rows do not authorize every conclusion
An exact duplicate is one event; conflicting copies require investigation. Even clean rows do not prescribe training.
One A record · descriptive comparison
This supports arithmetic about two logs, not a personal recommendation.
The useful result is often “not enough evidence”
What changes the conclusion?
Seconds versus minutes and two timezone renderings of the same instant preserve the narrow arithmetic statement. Missing time-zone context, a missing or old observation and conflicting copies do not. A stronger training prescription is outside the evidence even when the records are clean.
This is a way to test the support for a statement. It does not test whether a wearable measured the workout accurately. It does not determine a safe exercise dose. Most importantly, it cannot certify natural-language advice just because a structured numerical field passed an arithmetic rule: the sentence still needs review for unsupported implications.
A short check for your own AI insight
- Find the observation. What was measured or logged, and by which source?
- Keep the timestamp complete. Is the offset or time zone available? Is the value current or carried forward?
- Normalize before comparing. Do units, intervals and definitions match?
- Investigate copies and gaps. Is a second row another event, a duplicate or a conflicting revision?
- Match the claim to the evidence. A difference between two logs is not a diagnosis, causal effect or training instruction.
For a broader reader-facing checklist, see Before You Follow an AI Insight: Check the Evidence. For the date-label problem, see One Night, Two Dates.
Method, provenance and limits
Prepared 13 October 2026. This page’s rule set is version synthetic-guardrails-v1. It converts durations to seconds, resolves timestamp offsets to UTC, treats identical event IDs as one observation, rejects conflicting duplicates, and accepts only a typed description of the difference between two recent, direct observations. A 30-day freshness cutoff is an illustrative engineering setting, not a medically validated threshold.
The exercise is implemented with fixed invented data and deterministic browser code; it makes no model API call. The offline test harness checks unit/order/timezone/duplicate invariance and abstention for missing, stale, ambiguous or over-strong claims. Passing those fixtures is a software-test result, not AI accuracy, current Nutrinaut app validation, independent expert review or evidence that any advice improves health. Historical Nutrinaut insight records helped identify which failure modes to examine, but the examples here contain no real account content and are not a current production-advice audit.
For how our tools document their formulas and versions, see Tools Methodology & Sources. The Connections & Data Guide explains source paths and freshness limits. These links describe different parts of the product; neither is a certification of the example above.