|
Just Announced: The Agentic Data Summit, Dec 9 (Sponsored)
 |
AI is rewriting the rules for data infrastructure. Streaming keeps critical data moving in real time, while AI introduces new demands for how that data is accessed, governed, and acted on. The Agentic Data Summit, just announced for December 9, brings together engineers, architects, and industry leaders building mission-critical data and AI workloads in production. You’ll hear real lessons from real deployments — governing agent access, migrating off Kafka at scale, and running streaming, SQL, and AI on one unified platform. It’s free, virtual, and built for teams working on what’s next in data and AI.
Modern language models are skilled at many tasks. They can write code, summarize documents, and answer questions across many subjects. However, an application may still need something the model does not provide consistently. Its answers might omit important details, the way it classifies information might confuse similar categories, or its responses might ignore a particular expected writing style.
These things don’t mean that the model is not intelligent or doesn’t have knowledge. Often, it has the underlying capability but needs additional help applying it according to specific expectations.
That is the purpose of customizing a model. One way is to follow clearer instructions and provide better information. But when those also leave gaps between the expectation and reality of the model, further training needs to be imparted. Techniques such as LoRA and QLoRA make that training more practical by reducing the resources needed to adapt an existing model.
In this article, we are going to look at the various strategies to customize and fine-tune a model.
 |
Here are the key learning points in brief:
When instructions and information are not enough
How fine-tuning changes a model
LoRA: Learning a smaller set of changes
QLoRA: Reducing the memory footprint of the model
Tuning the techniques into a training process
Disclaimer: This post is based on publicly shared details from various sources. References at the end. Please comment if you notice any inaccuracies.
When Instructions and Information Are Not Enough
The natural starting point in customizing a model is prompting.
A prompt describes the task, identifies constraints, and specifies what a good answer should contain. For example, we can add examples of the type of answer we expect to give the model a clearer demonstration of the expected behavior. This approach is called few-shot prompting.
This simple approach can be quite effective. The task of summarizing a document becomes more useful when it specifies which details matter and how the summary should be organized. Many applications need no customization beyond well-designed instructions and prompts.
However, instructions can’t provide information that is missing. If an answer depends on an internal document or a recently updated policy, the application must provide that material explicitly to the model.
Retrieval-augmented generation (RAG) is the main technique for addressing this requirement. It finds relevant information in an external source and includes it in the model’s input. The model can then use that material when producing its answer.
Prompting and RAG both work through the information supplied during a request. They can guide behavior in a big way, but they don’t ordinarily change the model’s learned parameters.
Sometimes, recurring weaknesses remain. A model might still struggle with specialized document categories or keep producing summaries that highlight the wrong details. While adding more examples as part of every request may help, it also increases the amount of input the application must maintain and process.
This is where fine-tuning offers a different option. In fine-tuning, we can train the model on many examples so that the desired behavior becomes more firmly learned as part of the model’s default behavior.
This is not an absolute division between behavior and knowledge. Fine-tuning can also teach facts. However, retrieval remains useful when information changes frequently, or answers must be traceable to an external source.
How Fine-Tuning Changes an Existing Model
As you might be aware, a model’s behavior depends on billions of numerical values called parameters or weights. Many of these are weights that influence how information moves through the model’s calculations.
During pretraining, these values are adjusted using large amounts of training data. A text-generating model learns to predict the next token, which is basically a piece of text such as a word, part of a word, or punctuation. This process helps develop broad language capabilities within the model. Models intended for conversation usually receive additional training to follow instructions.
In contrast, fine-tuning continues from an existing model using a more focused dataset. Since fine-tuning builds on top of capabilities that are already present within the model, it reduces the amount of training required for specialization.
 |
A common approach is supervised fine-tuning (SFT). In SFT, each training example contains an input and the response the model should produce. Examples might pair documents with approved summaries, messages wi