When an LLM generates text, it assigns a probability to every possible next word. These probabilities are the key to building AI systems that can tell you when they're uncertain.
Consider this
An LLM says "The capital of France is Paris" and also says "The best restaurant in Paris is Le Petit Cler".
Both arrive as plain text. Should you trust them equally?
Which statement should you trust more?
A log probability (logprob) is the natural logarithm of a probability. Instead of saying "90% confident", the model reports -0.105.
logprob = ln(probability)
or equivalently
probability = elogprob
Why logarithms?
Probabilities can be tiny (0.0000001). Logarithms compress this range and make the math easier.
Interactive: Probability ↔ Logprob
Probability
0.50
Logprob
-0.69
Mental Model
0 → 100% confident (never happens)
-0.1 → ~90% confident (very high)
-0.7 → ~50% confident (coin flip)
-2.3 → ~10% confident (low)
Closer to 0 = more confident
Quick Check
If logprob = -0.22, approximately how confident is it?
You're building a customer service bot. Use logprobs to auto-respond when confident, escalate when uncertain.
"Your current balance is $1,234.56"
95% confidence
"Refinancing could save ~$200/month"
72% confidence
"Next payment due January 15th"
88% confidence
"I recommend investing in index funds"
45% confidence
Production Insight
Don't guess thresholds. Collect data, plot distributions, find natural cutoffs. Different tasks have different confidence profiles.
What if the model is 52% spam, 48% not spam? That's a coin flip, not a classification.
Key Insight
It's not just the top choice's confidence. It's the margin between top two choices.
Clear Classification
Margin: 90%
✓ Auto-classify
Ambiguous
Margin: 4%
⚠ Human review
90%
Clear ✓
15%
Ambiguous ⚠
4%
Coin flip ⚠
How do you know if RAG retrieved the right documents? Compare confidence with and without context.
Query
"What is the company's parental leave policy?"
"The company offers 16 weeks paid parental leave."
WITH context
86%
WITHOUT
20%
"Standard parental leave per regulations."
WITH context
30%
WITHOUT
25%
The Pattern
grounding = conf_with_context - conf_without
High score → Context helped (good retrieval)
Low score → Wrong document or hallucinating
Negative → Context confused the model
LLMs hallucinate. But token-level logprobs often reveal the lie.
The Pattern
"Confident structure, uncertain content" — confident on "The CEO of X is" but uncertain on the actual name.
Example 1: Known entity (click tokens)
✓ All tokens have reasonable confidence
Example 2: Potential hallucination
⚠ Entity tokens have low confidence — likely hallucination
Token clicked:
Click a token to see details
Detection Rule
if is_entity(token) and logprob < -1.5: flag()
In agentic systems, logprobs become critical. An agent that confidently uses the wrong tool is worse than one that asks for clarification.
Tool Selection Confidence
User query:
"Look up the latest on Apple"
Margin: 4% — too small
Agent should ask: "Search news or check stock price?"
Cascade Routing: Cost Optimization
Cheap (Flash)
$0.001
Expensive (Pro)
$0.03
680
Cheap handled
320
Escalated
65%
Savings
Self-Correction Triggers
Monitor confidence. If it drops, stop and reconsider.
Aborted at step 5. Confidence too low.
P(trust | evidence) ∝ elogprob
When to use:
The alternative:
Treating all outputs as equally trustworthy.
That's how production AI fails.
Tutorial complete.