GPT model parameters explained, starting with temperature
Machine-assisted translation of the archived article. Read the Chinese original.

Common GPT generation settings include temperature, stop sequences, top_p, frequency_penalty, presence_penalty and best_of.
Stop sequences are strings that signal where generation should end. When a matching sequence occurs, the model stops producing further text.
Top P, or nucleus sampling, selects from the smallest set of likely next tokens whose cumulative probability reaches the threshold p. Top-k sampling is a separate method that selects a fixed number of candidates.
A positive frequency penalty discourages reuse of a token in proportion to how often it has already appeared. It can reduce repetitive phrasing.
A positive presence penalty discourages tokens that have already appeared at least once. This can encourage the model to introduce new material.
Best of generates multiple candidates and returns the one with the best score under the service’s selection rule.
Temperature adjusts the distribution from which the next token is sampled. The model divides its logits by the temperature before applying softmax. A higher temperature flattens the distribution. A lower temperature gives more weight to the most likely tokens.
A higher temperature can increase variety at the cost of consistency. A lower value generally produces more predictable text. Choose the setting for the task and evaluate the results.
These settings give developers several ways to adjust variety and reduce repetition.
Appropriate values depend on the application and the kind of output you need.
The following examples illustrate how the settings change the result.
Top P example. For more varied sentences, try top_p = 0.8. The model samples from the most likely next tokens whose combined probability reaches 0.8.
Frequency penalty example. When generating paragraphs about technology, try a frequency penalty of 0.2 to reduce repeated terms.
Presence penalty example. In travel writing, a penalty of 0.5 can discourage repeating destinations already mentioned, such as Paris or London. The parameter applies to previously generated tokens. It does not accept a list of banned place names.
Best of example. Generate three candidate openings for an article and return the highest-scoring one.
Higher temperature tends to produce more varied and surprising text, sometimes at the expense of coherence.
Lower temperature tends to produce more consistent, conventional text.
Consider a chatbot answering “How can I learn to program?” Different temperatures can produce different styles of advice.
At a lower temperature, the chatbot might suggest a familiar study routine,
such as reading books, writing code regularly and completing practical projects.
At a higher temperature, it might suggest making music with code or applying programming techniques to an everyday problem.
These suggestions can be more imaginative, although some may be less practical.
The effect of any one value depends on the task and model.
Try several settings and compare the results on the same examples.
