Most writing about fine-tuning covers what it achieves.<\/p>\n
Knowing the limits matters more<\/strong>, because that is where people get hurt \u2014 relying on output that looked right and was not.<\/p>\n The best model cards state their own limitations plainly.<\/strong> AboutKnowledge’s two published quality models both say, in their own documentation, that they should not replace human auditors or regulatory review, and that compliance decisions must be verified against the official standard text.<\/p>\n That is what a professionally published model looks like.<\/strong> Its absence elsewhere is a warning.<\/p>\n A fine-tuned model is more reliable on its subject. It is not reliable.<\/strong><\/p>\n It produces the most likely output given its training. On typical cases that is very good. On unusual ones it can be confidently wrong.<\/p>\n And there is no signal.<\/strong> A wrong answer reads exactly as well as a right one \u2014 same tone, same structure, same apparent confidence.<\/p>\n Which means every use needs a review step<\/strong>, sized to the consequences. Drafting a checklist? A glance. Deciding compliance? A qualified person.<\/p>\n A model knows what it was trained on, at the time it was trained.<\/strong><\/p>\n Standards get revised.<\/strong> Regulations change. New requirements are added.<\/p>\n The model does not know.<\/strong> It will answer with what it learned, confidently, about a version that has been superseded.<\/p>\n Two implications:<\/strong><\/p>\n Retraining is a recurring cost<\/strong>, not a one-time build.<\/p>\n For anything where the current version matters, verify against the source.<\/strong> A model is a way to find your way around a standard, not a substitute for reading it.<\/p>\n FIGURE 1: THE LIMITS THAT MATTER MOST<\/p>\n Not reliably correct<\/p>\n Cannot stay current<\/p>\n Cannot explain its reasoning<\/p>\n Only as good as its training data<\/p>\n It produces an answer. It can produce text that looks like reasoning.<\/strong><\/p>\n Those are not the same thing.<\/strong><\/p>\n The explanation is generated alongside the verdict, not derived from a process you can inspect. It usually matches. It is not a proof.<\/p>\n Which matters when somebody asks how a conclusion was reached.<\/strong><\/p>\n For an operational nudge, fine.<\/strong> For anything a regulator, auditor or certification body may examine, you need a documented method a person can defend \u2014 not a model output.<\/p>\n Errors in the source get learned thoroughly.<\/strong><\/p>\n Both AboutKnowledge model cards say this directly:<\/strong> synthetic training data can contain occasional grammar artifacts, or mix standards in generic answers.<\/p>\n That is honest, and it is a general truth about fine-tuning.<\/strong> A model trained on a knowledge base inherits that knowledge base’s mistakes \u2014 and reproduces them with the same confidence as everything else.<\/p>\n Nor can it answer what it was never shown.<\/strong> Ask about a standard outside its training and you get a plausible answer assembled from adjacent knowledge. That is worse than an admission of ignorance<\/strong>, because it looks like an answer.<\/p>\n A model fine-tuned on one subject is unremarkable outside it.<\/strong><\/p>\n That is the trade you made. All the capacity on one thing means less of everything else.<\/p>\n Practically:<\/strong> do not use a quality-management model for general questions. It will answer, and the answer will be worse than a general model’s.<\/p>\n Know what your model is for, and use it for that.<\/strong><\/p>\n The line that should not move.<\/strong><\/p>\n Compliance decisions. Regulatory submissions. Anything a certification body reviews. Anything where money moves.<\/strong><\/p>\n A model can prepare, triage, draft and flag.<\/strong> All of that saves real time.<\/p>\n A person decides.<\/strong><\/p>\n Both of these published models state exactly this position<\/strong> \u2014 useful for drafting and triage, not a replacement for human auditors or regulatory review. Written by the people who built them<\/strong>, which is the most credible place for that caveat to come from.<\/p>\n FIGURE 2: WHAT IT DOES AND DOES NOT DO<\/p>\n Genuinely useful for<\/p>\n Not for<\/p>\n Some models can be prompted to express uncertainty. It is not reliable.<\/strong><\/p>\n A model does not have a calibrated sense of its own confidence.<\/strong> It will produce an answer with the same apparent certainty whether the case is typical or unlike anything it has seen.<\/p>\n Which means you cannot use its confidence as a filter.<\/strong> “Only review the ones it was unsure about” does not work.<\/p>\n The practical consequence:<\/strong> review a sample of everything, not just the cases that looked doubtful.<\/p>\n Six questions, for any fine-tuned model \u2014 yours or somebody else’s.<\/p>\n What was it trained on?<\/strong> Real data, synthetic, or a mix. From what source.<\/p>\n How much?<\/strong> Thousands or hundreds of thousands. Volume tells you something.<\/p>\n What does the model card say about limitations?<\/strong> If it says nothing, be careful.<\/strong> Every model has limits; a card that lists none is incomplete rather than exceptional.<\/p>\n What is the base model and licence?<\/strong> An adapter inherits its base model’s terms.<\/p>\n When was it trained?<\/strong> For subjects that change, age matters.<\/p>\n Where does a person review?<\/strong> If the answer is nowhere, that is the problem to fix first.<\/p>\n FIGURE 3: WHAT TO LOOK FOR IN A MODEL CARD<\/p>\n Stated limitations<\/p>\n Training data described<\/p>\n Base model and licence<\/p>\n An intended use, stated<\/p>\n A fine-tuned model is a capable assistant on its subject, whose work you check.<\/strong><\/p>\n That is a smaller claim than the marketing around AI, and a bigger one than the sceptics allow.<\/p>\n For drafting, triage and finding your way around a body of knowledge, it saves real time.<\/strong><\/p>\n For decisions with consequences, it prepares the ground and a person decides.<\/strong><\/p>\n That position is not caution for its own sake.<\/strong> It is what the people who build these models say about their own work \u2014 and they are in the best position to know.<\/p>\n A fine-tuned model cannot be relied on to be correct, cannot stay current, cannot explain itself in an auditable way, and cannot exceed its training data.<\/strong><\/p>\n It also cannot tell you when it is unsure<\/strong>, which is why sampling everything beats reviewing only what looked doubtful.<\/p>\n Use it to draft, triage and orient. Let a person decide anything with consequences.<\/strong><\/p>\n And when evaluating any published model, read the limitations section first.<\/strong> If there isn’t one, that tells you something.<\/p>\n Considering a fine-tuned model for a business task?<\/p>\n Get in touch. We will be specific about where it helps and where it should not be trusted \u2014 that conversation is worth having before the build, not after.<\/p>\n<\/div>","protected":false},"excerpt":{"rendered":" Why this article exists Most writing about fine-tuning covers what it achieves. Knowing the limits matters more, because that is where people get hurt \u2014 relying on output that looked right and was not. The best model cards state their own limitations plainly. AboutKnowledge’s two published quality models both say, in their own documentation, that […]<\/p>\n","protected":false},"author":3,"featured_media":9453,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[26],"tags":[],"class_list":["post-9452","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai"],"yoast_head":"\n1. It cannot be relied on to be correct<\/h2>\n
2. It cannot stay current<\/h2>\n
\n
\n
\n
\n
3. It cannot explain itself in a way you can audit<\/h2>\n
4. It cannot exceed its training data<\/h2>\n
5. It cannot handle what it was not built for<\/h2>\n
6. It cannot make a decision with consequences<\/h2>\n
\n
\n
7. It cannot tell you when it is unsure<\/h2>\n
What to check before relying on one<\/h2>\n
\n
\n
\n
\n
The realistic position<\/h2>\n
The short version<\/h2>\n