Token budgets with a four-character rule of thumb
Why packing context often divides length by four — what that approximation buys you, and where it quietly lies.
Token budgets with a four-character rule of thumb
When an agent packs memories into a fixed context window, every candidate needs a cheap size estimate. Exact tokenizer calls are accurate but slow and dependency-heavy for a hot path that only needs "will this fit?"
Credit: estimateTokens in Agent-Observer (internal/smartctx/packer.go).
The function
func estimateTokens(s string) int {
return len(s) / 4
}
What it does
It treats roughly four UTF-8 bytes (as Go len counts them) as one token. The packer uses that number when accumulating content under maxTokens: skip a memory if adding it would blow the budget; otherwise add it and count those tokens toward the total.
Why this shape
- O(1) and allocation-free — suitable for scoring dozens of candidates per pack.
- Matches a widely cited English/code heuristic (~4 characters per BPE-ish token) well enough for budgeting, not billing.
- Keeps the packing loop independent of any specific model tokenizer.
Gotcha
len(s) is bytes, not runes. Dense CJK, emoji, or multibyte identifiers inflate the byte length without a matching token increase in many tokenizers — so you can underfill the real window. Conversely, short English with lots of punctuation can overestimate. Use this for packing heuristics; switch to a real tokenizer when the budget must be exact.
Takeaway
Context packing is a control system for load on the prompt. A blunt divisor is fine when the cost of being slightly wrong is skipping one extra memory — not when you are metering paid tokens.