Fine-tuning traditionally means adjusting the whole model.<\/p>\n
For a model with billions of parameters, that is expensive.<\/strong> You need enough memory to hold all of them while training, and the result is a full copy of the model \u2014 several gigabytes \u2014 for every task you fine-tune.<\/p>\n Ten specialised models means ten full copies.<\/strong><\/p>\n LoRA \u2014 Low-Rank Adaptation \u2014 changes that.<\/strong> Instead of adjusting the whole model, it trains a small set of additional weights that sit alongside it.<\/p>\n The base model stays frozen.<\/strong> Untouched.<\/p>\n A small adapter is trained<\/strong> \u2014 the LoRA \u2014 capturing the difference between the general model and the specialised one.<\/p>\n At use time, the two are combined.<\/strong><\/p>\n Three consequences:<\/strong><\/p>\n Training is far cheaper.<\/strong> You are adjusting a fraction of the parameters, so it fits on much smaller hardware.<\/p>\n The output is small.<\/strong> An adapter is megabytes rather than gigabytes.<\/p>\n One base, many adapters.<\/strong> Ten specialised behaviours means one base model and ten small adapters, not ten full copies.<\/p>\n FIGURE 1: FULL FINE-TUNING VS LORA<\/p>\n Full fine-tuning<\/p>\n LoRA<\/p>\n Model cards quote a few numbers. Here is what they mean, without the mathematics.<\/p>\n Rank (r).<\/strong> How much capacity the adapter has. Higher means it can learn more, and costs more to train. Both of AboutKnowledge’s published quality models use rank 16<\/strong> \u2014 a common middle choice for a focused task.<\/p>\n Alpha.<\/strong> A scaling factor, usually set in relation to rank. Also 16 in those models.<\/p>\n Dropout.<\/strong> A regularisation setting. Set to 0.0 in both, which is reasonable when you have a very large training set \u2014 with a million examples, the model is unlikely to memorise its way through.<\/p>\n These are not exotic settings.<\/strong> They are the standard middle of the road, which is usually the right place to start.<\/p>\n Another term you will see, and it matters for cost.<\/p>\n Quantisation stores the model’s numbers at lower precision<\/strong> \u2014 4-bit rather than the usual 16 or 32.<\/p>\n The effect:<\/strong> the model takes much less memory, so it fits on hardware you can actually get.<\/p>\n The trade-off:<\/strong> a small loss of precision. For most practical tasks, not enough to matter.<\/p>\n Both AboutKnowledge models were trained with 4-bit NF4 quantisation<\/strong>, using the Unsloth framework \u2014 which is designed specifically to make this kind of training fast and memory-efficient.<\/p>\n Which is what puts fine-tuning within reach of a single machine<\/strong> rather than a rented cluster.<\/p>\n FIGURE 2: WHAT MAKES THIS AFFORDABLE<\/p>\n LoRA<\/p>\n 4-bit quantisation<\/p>\n A framework built for it<\/p>\n Result<\/p>\n Model cards quote numbers from training. Two are worth understanding.<\/p>\n Steps.<\/strong> How many batches of examples the model was trained on. More is not automatically better \u2014 past a point, the model stops improving.<\/p>\n Final training loss.<\/strong> How well the model predicted its training data by the end. Lower is better<\/strong>, and the absolute number only means something in context.<\/p>\n AboutKnowledge’s two models illustrate the point:<\/strong><\/p>\n The 1B quality model: 500,000 records, 10,000 steps, final loss 0.1618<\/strong>.<\/p>\n The 3B compliance analyzer: 1,000,000 records, 3,000 steps, final loss 0.1742<\/strong>.<\/p>\n Both are low, and they are not directly comparable<\/strong> \u2014 different model sizes, different tasks, different amounts of data per step.<\/p>\n A caution worth stating:<\/strong> low training loss means the model learned its training data. It does not prove it will do well on real input.<\/strong> That needs testing against cases it has not seen.<\/p>\n An adapter needs its base model.<\/strong> Publishing a LoRA means publishing the adapter and naming what it attaches to.<\/p>\n Both AboutKnowledge models do this<\/strong> \u2014 the adapter files plus a clear statement of the base model.<\/p>\n They also publish a merged GGUF version.<\/strong> That is the adapter combined into the base and converted to a format that runs in local tools such as Ollama and llama.cpp.<\/p>\n Which matters for use.<\/strong> An adapter is for people who will load it in code. A GGUF file is for people who want to run the model on their machine in one command.<\/p>\n Publishing both covers both audiences.<\/strong><\/p>\n Worth knowing before you use any fine-tuned model.<\/strong><\/p>\n A LoRA adapter inherits the base model’s licence.<\/strong> If the base is Llama, Meta’s Llama Community License applies to the merged result.<\/p>\n Both AboutKnowledge model cards say this explicitly<\/strong> and point readers to Meta’s licence before using or redistributing merged weights.<\/p>\n That is the correct thing to do<\/strong>, and its absence on a model card is a reason to be careful.<\/p>\n FIGURE 3: WHAT A GOOD MODEL CARD TELLS YOU<\/p>\n Always states<\/p>\n Also states honestly<\/p>\n Almost always, for a business fine-tuning a model for a narrow task.<\/strong><\/p>\n Full fine-tuning is worth considering<\/strong> when you have very large amounts of data, a substantial hardware budget, and a task where the last few percent of quality genuinely matters.<\/p>\n For everything else, LoRA gets you most of the result for a fraction of the cost<\/strong> \u2014 which is why it has become the default.<\/p>\n LoRA trains a small adapter alongside a frozen base model<\/strong>, rather than adjusting the whole thing.<\/p>\n Far cheaper to train, tiny to distribute, and one base can carry many adapters.<\/strong><\/p>\n Combined with 4-bit quantisation and a framework built for it, fine-tuning becomes something you can do on one machine in hours.<\/p>\n And check the licence.<\/strong> An adapter inherits the base model’s terms, and a good model card says so plainly.<\/p>\n Wondering whether a fine-tuned model is affordable?<\/p>\n Get in touch. With LoRA and quantised training, a specialised model for a narrow task is a smaller project than most people expect.<\/p>\n<\/div>","protected":false},"excerpt":{"rendered":" The problem LoRA solves Fine-tuning traditionally means adjusting the whole model. For a model with billions of parameters, that is expensive. You need enough memory to hold all of them while training, and the result is a full copy of the model \u2014 several gigabytes \u2014 for every task you fine-tune. Ten specialised models means […]<\/p>\n","protected":false},"author":3,"featured_media":9435,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[26],"tags":[],"class_list":["post-9434","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai"],"yoast_head":"\nWhat that means practically<\/h2>\n
\n
\n
The settings you will see<\/h2>\n
Quantised training<\/h2>\n
\n
\n
\n
\n
Reading a training run<\/h2>\n
Distribution<\/h2>\n
Licensing<\/h2>\n
\n
\n
When LoRA is the right choice<\/h2>\n
The short version<\/h2>\n