Fine-tuning or embeddings? Choosing an approach for model customization
Machine-assisted translation of the archived article. Read the Chinese original.

The author compares managing AI to the work of an HR business partner: choosing, deploying, developing and retaining talent.
Choosing the right AI model can make a major difference to productivity and business results.
Model choice comes first in the author’s view. The user’s skill also matters.
Leadership matters both in selecting a model and in deciding how to use it.
The article argues that directing an AI system makes different demands on a manager than leading people does.
The author jokes that AI has no feelings and can take an unlimited amount of pressure.
GPT can answer many questions from its training, but what happens when an answer depends on private records or recent information that the model has never seen?
Two common approaches are fine-tuning and retrieval using embeddings.
Many document chat systems use vector search over embeddings. OpenAI illustrates this approach in its Cookbook (http://github.com/openai/openai-cookbook) ), which includes worked examples and was recently updated at the time of publication.
How do fine-tuning and embeddings differ?
Fine-tuning trains a model further on a prepared set of examples, such as prompt–completion or question–answer pairs. The resulting model learns patterns from those examples.
With retrieval, each prompt includes the relevant information found in your own data.
Wang Jianshuo explained the difference with a household analogy.
Imagine a trained housekeeper who speaks Chinese and knows how to do the job, but has never seen your home.
Fine-tuning is like spending the first hour explaining where things belong, what to leave alone and which areas need special care.
Using embeddings for retrieval is like leaving notes around the house. The housekeeper can read an instruction when it becomes relevant.
When should you use embeddings or fine-tuning?

Embeddings
The OpenAI Cookbook offers a similar analogy.
Fine-tuning is like studying for an exam. It builds long-term memory, but a week later you may forget details or remember something incorrectly.
Retrieval is like taking notes into an open-book exam. You can look up a fact when the notes contain the answer.
What is the search–ask approach to RAG?
Search your local document collection first. Then send the retrieved passages and the question to GPT so the model can use those passages when composing an answer.
Does RAG require vector search?
There are several ways to retrieve material for an answer.
Keyword search
Graph-based search
Vector search
The OpenAI Cookbook explains the implementation in more detail.
Further reading
https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb
What limits apply when using embeddings?
Retrieved text has to fit within the input budget.
The prompt must accommodate both the original question and the retrieved passages. That limits how much material can accompany each request.
You might own a shelf of textbooks but only have enough room to take a few pages of notes into the exam.
For answering questions over a large body of text, the article points to OpenAI’s search–ask approach.
Adding context consumes tokens on every request. A good retrieval step can improve accuracy by selecting just the passages needed to answer the question.
Does my data need to be in question-and-answer format for either approach?
Suppose I have 1 MB of private plain text, with no manually prepared Q&A pairs. Can I ask questions about it and use ChatGPT for anything outside those notes?
For retrieval, ordinary text is fine. Split it into chunks, search for relevant chunks and include the best matches with the question, within the model’s context window.
How can a model work with an entire PDF, book or document when its context window is limited?
The article suggests the following options.
Compress the material into a shorter context, or use Qwen’s long-text PDF API to extract the passages you need.
If the text still exceeds the context window, use a vector database and a search or ranking method. Answer each query in two steps.

1. Retrieve the documents most relevant to the user’s query.
2. Include the top n passages with the question and ask the model to answer using only those passages. The article calls this a semantic query.
