How is inference different from training?
Training is the expensive, one-off part. Over weeks or months, a model's billions of numbers are adjusted until its predictions are good, and then they are frozen. Inference uses those frozen numbers: your text goes in, the numbers are applied, and one token comes out. Then it happens again for the next token, and the next. The cookbook gets written once, but every meal still has to be cooked.
Why does the reply cost more than the question?
Reading happens all at once and writing happens one token at a time. A model can take in the whole of your input in a single parallel sweep, but it cannot write word ten until word nine exists, so each output token holds the chips for its own full pass through the model. The price lists from OpenAI, Anthropic and Google reflect that: input and output are priced separately per million tokens, and output costs several times as much.
What else makes one call dearer than another?
Length, first. A long conversation or a pasted document is re-sent and re-read on every call, which is the cost side of the context window. Then thinking: "reasoning" models work through a problem in hidden tokens before they answer, and OpenAI's documentation says those reasoning tokens are billed as output even though you never see them. Pictures and audio add tokens of their own. And the model itself matters, since a maker's biggest model costs more per token than its smaller ones.
How can a driving school's month of replies cost five times as much? (example figures)
A driving school on the Gold Coast uses an AI tool, connected straight to a model maker's API, to draft replies to 500 website enquiries a month. Each draft sends 2,000 tokens of instructions and prices (a 10-lesson pack at $650, a test-day package at $180) and gets back a 250-token reply. The rates are example figures in Australian dollars.
| Standard model | Reasoning model | |
|---|---|---|
| Input tokens | 1 million | 1 million |
| Reply tokens you see | 125,000 | 125,000 |
| Hidden thinking tokens, 3,000 a reply | None | 1.5 million |
| Input at $4.50 a million | $4.50 | $4.50 |
| Output at $22.50 a million | $2.81 | $36.56 |
| The month | $7.31 | $41.06 |
Same job, same replies, more than five times the bill, all of it in tokens the owner never sees. For short, factual replies the standard model is the sensible default. The reasoning model earns its cost on work that has to be thought through, such as untangling a messy month of campaign results.
Is Cora running inference while you talk?
Yes, every time you talk with Cora, CoraVision runs inference: your words go in and Cora's reply is produced there and then, and in a spoken conversation the speed of that step is what you hear as latency. The onboarding call, about 25 minutes long, is thousands of those steps in a row. CoraVision's pricing is not public yet. The beta opens in late 2026, places are limited, and applications are at coravision.ai/early-access.
Questions
Why are some AI apps free if every reply costs money?
Someone still pays for the computing. Free tiers are usually limited, often run smaller and cheaper models, and are paid for by subscribers on paid plans or by the company's investors. On some consumer apps your conversations may also be used to improve future models, depending on your settings.
Can I run an AI model on my own computer?
Yes, with an open-weight model such as Meta's Llama or one of Mistral's, and free programs that download and run them. Inference then happens on your machine, so there is no per-token bill, but the models that fit on a laptop are smaller and less capable than the ones behind the big apps.