The problem LoRA solves

Fine-tuning traditionally means adjusting the whole model.

For a model with billions of parameters, that is expensive. You need enough memory to hold all of them while training, and the result is a full copy of the model — several gigabytes — for every task you fine-tune.

Ten specialised models means ten full copies.

LoRA — Low-Rank Adaptation — changes that. Instead of adjusting the whole model, it trains a small set of additional weights that sit alongside it.

What that means practically

The base model stays frozen. Untouched.

A small adapter is trained — the LoRA — capturing the difference between the general model and the specialised one.

At use time, the two are combined.

Three consequences:

Training is far cheaper. You are adjusting a fraction of the parameters, so it fits on much smaller hardware.

The output is small. An adapter is megabytes rather than gigabytes.

One base, many adapters. Ten specialised behaviours means one base model and ten small adapters, not ten full copies.

The settings you will see

Model cards quote a few numbers. Here is what they mean, without the mathematics.

Rank (r). How much capacity the adapter has. Higher means it can learn more, and costs more to train. Both of AboutKnowledge’s published quality models use rank 16 — a common middle choice for a focused task.

Alpha. A scaling factor, usually set in relation to rank. Also 16 in those models.

Dropout. A regularisation setting. Set to 0.0 in both, which is reasonable when you have a very large training set — with a million examples, the model is unlikely to memorise its way through.

These are not exotic settings. They are the standard middle of the road, which is usually the right place to start.

Quantised training

Another term you will see, and it matters for cost.

Quantisation stores the model’s numbers at lower precision — 4-bit rather than the usual 16 or 32.

The effect: the model takes much less memory, so it fits on hardware you can actually get.

The trade-off: a small loss of precision. For most practical tasks, not enough to matter.

Both AboutKnowledge models were trained with 4-bit NF4 quantisation, using the Unsloth framework — which is designed specifically to make this kind of training fast and memory-efficient.

Which is what puts fine-tuning within reach of a single machine rather than a rented cluster.

Reading a training run

Model cards quote numbers from training. Two are worth understanding.

Steps. How many batches of examples the model was trained on. More is not automatically better — past a point, the model stops improving.

Final training loss. How well the model predicted its training data by the end. Lower is better, and the absolute number only means something in context.

AboutKnowledge’s two models illustrate the point:

The 1B quality model: 500,000 records, 10,000 steps, final loss 0.1618.

The 3B compliance analyzer: 1,000,000 records, 3,000 steps, final loss 0.1742.

Both are low, and they are not directly comparable — different model sizes, different tasks, different amounts of data per step.

A caution worth stating: low training loss means the model learned its training data. It does not prove it will do well on real input. That needs testing against cases it has not seen.

Distribution

An adapter needs its base model. Publishing a LoRA means publishing the adapter and naming what it attaches to.

Both AboutKnowledge models do this — the adapter files plus a clear statement of the base model.

They also publish a merged GGUF version. That is the adapter combined into the base and converted to a format that runs in local tools such as Ollama and llama.cpp.

Which matters for use. An adapter is for people who will load it in code. A GGUF file is for people who want to run the model on their machine in one command.

Publishing both covers both audiences.

Licensing

Worth knowing before you use any fine-tuned model.

A LoRA adapter inherits the base model’s licence. If the base is Llama, Meta’s Llama Community License applies to the merged result.

Both AboutKnowledge model cards say this explicitly and point readers to Meta’s licence before using or redistributing merged weights.

That is the correct thing to do, and its absence on a model card is a reason to be careful.

When LoRA is the right choice

Almost always, for a business fine-tuning a model for a narrow task.

Full fine-tuning is worth considering when you have very large amounts of data, a substantial hardware budget, and a task where the last few percent of quality genuinely matters.

For everything else, LoRA gets you most of the result for a fraction of the cost — which is why it has become the default.

The short version

LoRA trains a small adapter alongside a frozen base model, rather than adjusting the whole thing.

Far cheaper to train, tiny to distribute, and one base can carry many adapters.

Combined with 4-bit quantisation and a framework built for it, fine-tuning becomes something you can do on one machine in hours.

And check the licence. An adapter inherits the base model’s terms, and a good model card says so plainly.