What fine-tuning updates
What actually gets updated when someone says "we train the model on your data"
It is the sentence that gets sold the most and explained the least. It is worth understanding before you sign, because the cost of the project and what you can expect from the result both hang on it.
The short answer: the model is not retrained. Almost all of it is frozen and one small piece is trained and added on top. And that piece teaches behaviour, not knowledge.
Why nobody retrains a large model
A language model of the kind in use today has billions of parameters, which are the numbers holding what it learnt. Training all of them again on one company's data would cost more than the company, and would need a computing centre almost nobody has.
So the industry works with a family of techniques that freeze the original model and train only a compact correction: parameter-efficient fine-tuning, PEFT. The base model is left intact; what gets saved is the correction, which takes a minute fraction of the space.
LoRA, which is what is used almost every time
The idea fits in one sentence: instead of modifying a huge matrix of weights, two thin matrices are learnt whose product is the correction.
The rank, which is the dial that matters
The width of those two matrices is called the rank. It is the one parameter that decides how much the correction can learn and how much memory it costs to train. A low rank learns little and costs little; a high one, the other way round. Whoever sells you a fine-tune should be able to tell you which rank they work with, and why.
How many parameters are touched
Between one in a thousand and one per cent of the total, depending on the rank and on how many layers it hooks into. The rest of the model does not change by a single number, which means the correction can be removed, stored or swapped for another without touching the base model.
The variants, and what each is for
Freezing one of the two matrices and training only the other halves the memory. Using fixed random matrices and training only a few scaling vectors cuts it much further, at the cost of capacity. Another variant gives the base model back part of what was learnt at each step. And another trains the two matrices at different speeds, which sounds minor and changes the result quite a bit.
What it means that this is cheap
That the bottleneck stopped being compute. Today one of these corrections is trained in hours and on a single graphics card. What costs is somewhere else, and that is what follows.
It teaches behaviour, not knowledge
This is the part that decides whether it suits you. A fine-tune changes the form: the output format, the tone, the vocabulary of the trade, the way an answer is structured. If your problem is that the model answers well but in the wrong format, or in a register that is not your company's, fine-tuning is the tool.
What it does not do reliably is put new knowledge in. Your company's facts do not end up inside that correction in any way you can query, and if they change tomorrow the whole process has to be repeated. That is why knowledge goes through retrieval, handing the model the documents at the moment of answering, and form, if it is needed, goes through tuning.
There is one more use, which is economic rather than about quality: putting a small model to imitate a large one on one specific task. The small one is cheaper and faster to run, and on that task it can match it. That is why a model is sometimes tuned even when retrieval already works.
Where the real cost is
Not in the compute hours. It is in building the set of examples: thousands of pairs of question and correct answer, written or reviewed by somebody inside your company who knows the subject. That work cannot be fully outsourced, because the correct answer is defined by whoever knows the business.
Which gives you the question worth asking in the sales meeting: how many examples are needed, who is going to write them, and how many hours of your people that costs. If a proposal talks to you about training without telling you the rank, the size of the example set or who is going to write it, the proposal is not finished. It is a fair question and the answer fits in a paragraph.
The order the engineering of the field recommends
This is not a preference of mine. It is the order proposed by AI Engineering (Chip Huyen, O'Reilly, 2025) in its chapter 7, and anyone can check it.
1. Ask properly
First you try with clear instructions to the model, versioned the way code is versioned. A surprising share of the problems is solved right here.
2. Give examples in the instruction itself
Between one and fifty, depending on the case. The model is still untouched and it still costs nothing beyond writing them.
3. Connect it to the sources
If it fails for lack of information, hand it the documents at the moment of answering. Search by keyword and by meaning at once from day one, which is what saves redoing the work, and measure every change against a set of real questions.
4. And only then, tune
If what is still wrong is the behaviour rather than the information, that is where fine-tuning comes in. The 2024 experiment by Ovadia and others measured that retrieval beats fine-tuning in almost every category, and that adding retrieval on top of an already tuned model improves the result only 43 % of the time.
If you are evaluating a proposal for artificial intelligence over your company's documents, the full conversation, with prices and with what is worth demanding from any vendor, is on this page.