Direct answer
Use this visualizer to understand why prompt cost changes when you edit text, add JSON, paste code, or include multilingual content.
Prompt compression check
Input: A verbose support-agent instruction compared with a shorter rewrite.
Output: The output should show approximate token count, comparison delta, and visible token boundaries.
How to use this tool
- 1. Paste prompt text.
- 2. Choose a model pricing tier.
- 3. Optionally add a second prompt for diff mode.
- 4. Review approximate tokens, cost, and compression suggestions.
What tokenization is
Tokenization splits text into chunks models can process. Words, punctuation, whitespace, code, emojis, and non-English scripts can all split differently. Token count matters because context limits and billing are token-based.
This page uses a lightweight browser approximation so it stays fast and dependency-free. For billing-critical work, verify with provider-specific tokenizers.
When token counts surprise you
JSON, code, emojis, long identifiers, tables, and multilingual text often tokenize less intuitively than plain English prose. Prompt compression should remove redundancy before it removes instructions that protect quality.
Assumptions and methodology
This tool uses transparent browser-side calculations and curated assumptions rather than LLM-generated recommendations. Outputs are planning estimates. They should be validated against provider pricing, production traces, engineering quotes, or domain review before money, compliance, safety, or hiring decisions are made.
Numerical defaults are dated and surfaced on the page. The methodology favours explicit assumptions over false precision: every estimate is meant to expose the variable that drives the result, not to pretend that early planning data is exact.
Turn the result into an implementation plan
Bring the scenario to a strategy call and I will pressure-test the workflow, assumptions, failure modes, and delivery path.
Book a strategy callFrequently asked questions
- How do tokens differ across models?
- Models use different tokenizers and vocabularies. The same text can produce different counts in GPT, Claude, Gemini, Llama, or Qwen, especially for code, emojis, and non-English text.
- Why is a 100-word prompt 150 tokens?
- Tokens are not words. Punctuation, whitespace, fragments, numbers, and special characters count separately depending on the tokenizer. English prose often averages around 0.75 words per token.
- Are emojis expensive?
- Emojis can be relatively expensive because they may split into multiple byte-level tokens. One emoji can cost more tokens than one short English word.
- How do I count tokens exactly?
- Use official tokenizers or provider token-count endpoints for exact counts. This visualizer is designed for quick comparison and education, not final billing reconciliation.
- Does tokenization affect quality?
- Indirectly. Tokenization affects context budget and cost. Quality drops when important instructions or context are removed to fit a limit, not because token boundaries alone are visible to users.