Why the same PDF costs a different amount on GPT-5, Claude, and Gemini

Run the same PDF through GPT-5 and Claude and you'll usually see two different token counts. Same file, same text, different number. Then the bill lands and that's different too. It surprises people, and it reads like a bug. It isn't.

The reason is simple once you see it. Every model counts tokens with its own tokenizer, and those tokenizers chop text up differently.

A token isn't a fixed thing

A tokenizer is the small program that turns your text into the pieces a model actually reads. One tokenizer might see "tokenization" as a single token. Another splits it into "token" and "ization". Same word, different count. Do that across a 30 page PDF and the totals drift apart.

So "how many tokens is my PDF" doesn't have one answer. It has one answer per model.

What the big families actually use

• GPT-4o and GPT-5 use an encoding called o200k_base. It's the most efficient of the bunch, so it tends to report the fewest tokens for the same text.
• GPT-4 and GPT-3.5 use the older cl100k_base, which usually lands a bit higher.
• Claude uses its own tokenizer, and the newer models pack text denser, so they can report noticeably more tokens than GPT for the exact same page.
• Gemini uses a different approach again (SentencePiece), so its count sits in its own spot.

None of these are wrong. They're just different rulers measuring the same wall.

Why the cost gap is bigger than you'd think

Two things move your bill, and they stack.

First, the token count itself changes by model, as above. Second, the price per token is different per model too. A denser tokenizer plus a higher per-token price means the same PDF can cost several times more on one model than another. And the gap isn't only the sticker price on the pricing page. It's that price times a token count you didn't control.

Rough shape of it, not exact numbers: if Claude turns your PDF into 20 percent more tokens than GPT-5 and also charges more per token, the real gap is those two multiplied, not just the price difference.

How to actually compare

Two steps, both local and free.

Count the tokens with the right tokenizer first. The PDF Token Counter shows the exact count for the OpenAI encodings (o200k for GPT-4o and GPT-5, cl100k for GPT-4) and a marked estimate for Claude and Gemini, since those tokenizers aren't published for the browser.

Then turn those counts into money. The Token Cost Calculator multiplies the per-model token count by each provider's published price, so you see the real spread before you send anything. You can edit the prices when they change.

One honest caveat

Only OpenAI publishes a tokenizer you can run in a browser, so the GPT counts are exact. Claude and Gemini are estimated from those counts, close enough for planning but not for invoicing. When a number has to be exact for billing, check it against the provider's own counter. For picking a model and sizing a budget, the estimate is plenty.

The short version

There's no single token count for a PDF. Each model measures with its own tokenizer, and the cheaper sticker price can still lose to a denser count. Count with the right encoding, convert to dollars per model, then decide. If you want the context window side of this, see PDF token limits by model.