A model’s size is its parameter count<\/strong> \u2014 1B is one billion, 3B is three billion, and so on up.<\/p>\n Roughly, it is how much the model can hold.<\/strong> More parameters means more capacity for knowledge and for handling complexity.<\/p>\n But capacity is only useful if it is pointed at your problem.<\/strong><\/p>\n A 1B model fine-tuned entirely on quality management has all of its capacity on that subject. A 70B general model has vastly more capacity, of which quality management occupies a small share.<\/p>\n Which is why a small specialised model can beat a large general one at its own task.<\/strong><\/p>\n Memory.<\/strong> Roughly, at 8-bit quantisation, a 1B model is around 1.3 GB and a 3B around 3.4 GB. Larger models scale accordingly.<\/p>\n Speed.<\/strong> Smaller models respond faster on the same hardware.<\/p>\n Where it can run.<\/strong> A 1B model runs on almost anything. A 3B runs comfortably on a normal laptop. Larger models want a GPU.<\/p>\n Training cost.<\/strong> Fine-tuning a larger model takes more time and more hardware.<\/p>\n FIGURE 1: WHAT SIZE COSTS<\/p>\n Memory<\/p>\n Speed<\/p>\n Where it runs<\/p>\n Training<\/p>\n AboutKnowledge published two models at different sizes, for different jobs.<\/strong> The reasoning behind the split is instructive.<\/p>\n The 1B model answers questions and drafts documents.<\/strong> What does this clause require? Write a checklist. Suggest KPIs.<\/p>\n The 3B model makes structured judgements.<\/strong> Given a statement, produce a verdict, cite the clause, explain, and recommend.<\/p>\n Why the harder task got the larger model:<\/strong><\/p>\n Judgement requires holding more at once.<\/strong> The model has to understand the statement, relate it to the right standard, decide a verdict, identify a clause, and produce four coherent output sections that agree with each other.<\/p>\n Answering a question is simpler.<\/strong> Retrieve and explain.<\/p>\n The lesson is not “3B is better”.<\/strong> It is that the task determines the size, and the two tasks were different enough to justify two models.<\/p>\n Four questions.<\/p>\n Very narrow<\/strong> \u2014 one subject, one kind of question \u2014 a small model can be enough.<\/p>\n Broad<\/strong> \u2014 general reasoning, varied subjects \u2014 you need a larger model or a hosted one.<\/p>\n Simple output<\/strong> \u2014 an answer, a classification \u2014 smaller works.<\/p>\n Multiple coherent sections that must agree with each other<\/strong> \u2014 a verdict that matches the cited clause that matches the recommendation \u2014 that is harder, and it benefits from size.<\/p>\n On a laptop, offline, or on modest hardware<\/strong> \u2014 this constrains you, and it is often the deciding factor.<\/p>\n On a server with a GPU<\/strong> \u2014 more options.<\/p>\n A larger model needs more training data to be worth its size.<\/strong> Fine-tuning a large model on a small dataset wastes most of its capacity and risks it memorising rather than learning.<\/p>\n FIGURE 2: SMALLER OR LARGER<\/p>\n Smaller works when<\/p>\n Larger is needed when<\/p>\n The most practical advice in this article.<\/strong><\/p>\n Both sizes are cheap to try.<\/strong> Fine-tune the small one first. If it is good enough at your task, you are finished \u2014 and you have the cheaper, faster, more deployable option.<\/p>\n If it is not good enough, you now know why<\/strong>, and you can move up with a clear reason rather than a guess.<\/p>\n Starting large is the expensive mistake.<\/strong> More training cost, more hardware, and no evidence that the extra size was needed.<\/p>\n Bad training data.<\/strong> A larger model trained on flawed examples learns the flaws more thoroughly.<\/p>\n A poorly defined task.<\/strong> If you cannot describe the output you want precisely, no size produces it consistently.<\/p>\n Missing knowledge.<\/strong> A model knows what it was trained on. Size does not add facts you never gave it.<\/p>\n The need for review.<\/strong> Both of these published models say the same thing regardless of size<\/strong> \u2014 useful for drafting and triage, not a replacement for human review. The 3B model carries the same caveat as the 1B one.<\/p>\n 1B<\/strong> \u2014 a narrow subject, simple output, needs to run anywhere. Question answering, classification, drafting.<\/p>\n 3B<\/strong> \u2014 a narrow subject with structured multi-part output, or reasoning that needs more room. Still runs on a laptop.<\/p>\n 7B and up<\/strong> \u2014 broader tasks, harder reasoning, and you have a GPU.<\/p>\n Hosted models<\/strong> \u2014 you need the strongest available capability, and the data can leave your infrastructure.<\/p>\n These are starting points, not rules.<\/strong> The only reliable method is to try the smaller option and see whether it is good enough.<\/p>\n FIGURE 3: HOW TO DECIDE<\/p>\n Define the task precisely<\/p>\n Try the smaller model<\/p>\n Test on real cases<\/p>\n Move up only with a reason<\/p>\n Size is capacity, and capacity only helps if it is pointed at your problem.<\/strong><\/p>\n A small fine-tuned model can beat a much larger general one at its own narrow subject<\/strong> \u2014 and it runs on a laptop.<\/p>\n Structured multi-part output is what most often justifies going larger<\/strong>, because the parts have to agree with each other.<\/p>\n Try the smaller one first.<\/strong> If it is good enough, you have the cheaper and more deployable answer. If not, you now know exactly what was missing.<\/p>\n Unsure what size model your task actually needs?<\/p>\n Get in touch. We start with the smallest option that could work \u2014 it is usually cheaper to test than to assume.<\/p>\n<\/div>","protected":false},"excerpt":{"rendered":" What the number means A model’s size is its parameter count \u2014 1B is one billion, 3B is three billion, and so on up. Roughly, it is how much the model can hold. More parameters means more capacity for knowledge and for handling complexity. But capacity is only useful if it is pointed at your […]<\/p>\n","protected":false},"author":3,"featured_media":9450,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[26],"tags":[],"class_list":["post-9449","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai"],"yoast_head":"\nWhat size costs you<\/h2>\n
\n
\n
\n
\n
A worked comparison<\/h2>\n
How to choose<\/h2>\n
How narrow is the task?<\/h3>\n
How much structure does the output need?<\/h3>\n
Where must it run?<\/h3>\n
How much data do you have?<\/h3>\n
\n
\n
Test rather than assume<\/h2>\n
What size does not fix<\/h2>\n
A rough guide<\/h2>\n
\n
\n
\n
\n
The short version<\/h2>\n