What the number means
A model’s size is its parameter count — 1B is one billion, 3B is three billion, and so on up.
Roughly, it is how much the model can hold. More parameters means more capacity for knowledge and for handling complexity.
But capacity is only useful if it is pointed at your problem.
A 1B model fine-tuned entirely on quality management has all of its capacity on that subject. A 70B general model has vastly more capacity, of which quality management occupies a small share.
Which is why a small specialised model can beat a large general one at its own task.
What size costs you
Memory. Roughly, at 8-bit quantisation, a 1B model is around 1.3 GB and a 3B around 3.4 GB. Larger models scale accordingly.
Speed. Smaller models respond faster on the same hardware.
Where it can run. A 1B model runs on almost anything. A 3B runs comfortably on a normal laptop. Larger models want a GPU.
Training cost. Fine-tuning a larger model takes more time and more hardware.
FIGURE 1: WHAT SIZE COSTS
Memory
- About 1.3 GB at 1B, 3.4 GB at 3B, at 8-bit quantisation.
Speed
- Smaller responds faster on the same machine.
Where it runs
- 1B almost anywhere. 3B on a laptop. Larger wants a GPU.
Training
- Larger models cost more time and hardware to fine-tune.
A worked comparison
AboutKnowledge published two models at different sizes, for different jobs. The reasoning behind the split is instructive.
The 1B model answers questions and drafts documents. What does this clause require? Write a checklist. Suggest KPIs.
The 3B model makes structured judgements. Given a statement, produce a verdict, cite the clause, explain, and recommend.
Why the harder task got the larger model:
Judgement requires holding more at once. The model has to understand the statement, relate it to the right standard, decide a verdict, identify a clause, and produce four coherent output sections that agree with each other.
Answering a question is simpler. Retrieve and explain.
The lesson is not “3B is better”. It is that the task determines the size, and the two tasks were different enough to justify two models.
How to choose
Four questions.
How narrow is the task?
Very narrow — one subject, one kind of question — a small model can be enough.
Broad — general reasoning, varied subjects — you need a larger model or a hosted one.
How much structure does the output need?
Simple output — an answer, a classification — smaller works.
Multiple coherent sections that must agree with each other — a verdict that matches the cited clause that matches the recommendation — that is harder, and it benefits from size.
Where must it run?
On a laptop, offline, or on modest hardware — this constrains you, and it is often the deciding factor.
On a server with a GPU — more options.
How much data do you have?
A larger model needs more training data to be worth its size. Fine-tuning a large model on a small dataset wastes most of its capacity and risks it memorising rather than learning.
FIGURE 2: SMALLER OR LARGER
Smaller works when
- The subject is narrow
- Output is simple
- It must run on modest hardware
- You have less training data
Larger is needed when
- Output has several parts that must agree
- The reasoning is genuinely harder
- A GPU is available
- You have data to justify the capacity
Test rather than assume
The most practical advice in this article.
Both sizes are cheap to try. Fine-tune the small one first. If it is good enough at your task, you are finished — and you have the cheaper, faster, more deployable option.
If it is not good enough, you now know why, and you can move up with a clear reason rather than a guess.
Starting large is the expensive mistake. More training cost, more hardware, and no evidence that the extra size was needed.
What size does not fix
Bad training data. A larger model trained on flawed examples learns the flaws more thoroughly.
A poorly defined task. If you cannot describe the output you want precisely, no size produces it consistently.
Missing knowledge. A model knows what it was trained on. Size does not add facts you never gave it.
The need for review. Both of these published models say the same thing regardless of size — useful for drafting and triage, not a replacement for human review. The 3B model carries the same caveat as the 1B one.
A rough guide
1B — a narrow subject, simple output, needs to run anywhere. Question answering, classification, drafting.
3B — a narrow subject with structured multi-part output, or reasoning that needs more room. Still runs on a laptop.
7B and up — broader tasks, harder reasoning, and you have a GPU.
Hosted models — you need the strongest available capability, and the data can leave your infrastructure.
These are starting points, not rules. The only reliable method is to try the smaller option and see whether it is good enough.
FIGURE 3: HOW TO DECIDE
Define the task precisely
- What input, what output, how structured
Try the smaller model
- Cheaper, faster, easier to deploy
Test on real cases
- Not on your training data
Move up only with a reason
- You now know what was missing
The short version
Size is capacity, and capacity only helps if it is pointed at your problem.
A small fine-tuned model can beat a much larger general one at its own narrow subject — and it runs on a laptop.
Structured multi-part output is what most often justifies going larger, because the parts have to agree with each other.
Try the smaller one first. If it is good enough, you have the cheaper and more deployable answer. If not, you now know exactly what was missing.
Unsure what size model your task actually needs?
Get in touch. We start with the smallest option that could work — it is usually cheaper to test than to assume.
