Starting from something that already works

Training a language model from nothing takes enormous data and enormous computing power. Almost nobody does it.

What most people do is start from a model somebody else trained, and adjust it.

The base model already knows language, grammar, reasoning and a great deal of general information. Fine-tuning teaches it your subject and your output format — using a fraction of the data and a fraction of the cost.

What fine-tuning actually changes

Two things, and the second is often more valuable than the first.

Knowledge. The model becomes more reliable on your subject — the terminology, the standards, the specifics.

Behaviour. It learns to answer in the shape you need. Not a paragraph of prose, but a verdict, a citation, an explanation and a list of recommendations — every time, in the same structure.

That second one is underrated. A general model can be prompted into a format. A fine-tuned one produces it consistently, without a long instruction on every call.

A real example

AboutKnowledge fine-tuned two models on quality management and ISO compliance, both published openly on Hugging Face.

The first is built for questions and drafting — quality management systems, ISO standards, auditing, CAPA, Six Sigma, Lean, FMEA, SPC and related topics.

The second is built for analysis. Given a statement about how an organisation operates, it returns a compliance verdict, the relevant clause, an explanation, and recommendations.

Two models, two jobs. One answers questions. One produces structured judgements. That split is a design decision, not an accident, and it is covered in more detail in the case studies in this series.

Why not just prompt a big model?

A fair question, and sometimes prompting is the right answer.

Where fine-tuning wins:

Consistency. A fine-tuned model produces the same structure every time. A prompted one drifts, especially on unusual input.

Cost per call. A long system prompt is sent every single time. Fine-tuning bakes that in.

Size. A fine-tuned small model can outperform a much larger general one on its specific task — which means it can run on modest hardware.

Privacy. A small fine-tuned model runs locally. Your data never leaves your infrastructure.

Where prompting wins:

Breadth. A large general model handles anything. A fine-tuned small one is good at its subject and unremarkable outside it.

Speed to start. Prompting takes minutes. Fine-tuning takes a dataset and a training run.

Change. Adjusting a prompt is instant. Retraining is not.

What you need

Three things.

A base model. Open models such as Llama, Mistral or Qwen are the usual starting point. They come in sizes — 1B, 3B, 7B and up — and the number is roughly how much the model can hold.

Training data. Examples of the input and the output you want. Thousands, not dozens. Where this comes from is a subject in itself, and one of the articles in this series.

Compute. A GPU, for hours rather than months. Techniques like LoRA reduce this substantially — enough that fine-tuning a small model is within reach of a single machine rather than a data centre.

The size question

Bigger is not automatically better for a narrow task.

A 1B model fine-tuned on quality management can answer quality management questions better than a general model many times its size — because all of its capacity is pointed at one subject.

And it runs on a laptop.

The trade-off: the small model is worse at everything else. Ask it about anything outside its training and the answer is unremarkable.

Which is fine, if you only ever ask it about its subject.

What it does not fix

It does not make the model correct. A fine-tuned model can still be confidently wrong, particularly on cases unlike anything in its training.

It does not remove the need for review. For anything with consequences — compliance, regulation, money — a person checks.

It does not stay current. The model knows what it was trained on. New standards, new rules, new products mean retraining.

Both of AboutKnowledge’s published models say this in their own documentation — useful for drafting and triage, not a replacement for human auditors or regulatory review. That is the right position, and it is worth looking for in any model you consider using.

When it is worth it

A narrow, repeated task. The same kind of question, many times.

A need for consistent structured output. Something a downstream system will parse.

A privacy requirement. The data cannot go to a third-party API.

Enough volume that per-call API costs add up.

Not worth it for one-off tasks, broad general questions, or anything where requirements are still changing week to week.

The short version

Fine-tuning takes a model that already understands language and teaches it your subject and your output format.

The format part is often the bigger win — consistent structure, without a long prompt every time.

A small fine-tuned model can beat a large general one on its own subject, and run locally on modest hardware.

It does not make the model correct, and it does not remove the person who checks.