You Choose How It Picks
A model does not just hand you an answer — you can tune how it picks each next token. The gap between reliable output and creative output is partly the model, and partly your settings.
Two Kinds of Dial
At each step the model has a ranked list of possible next tokens. These settings shape which one it commits to.
- Temperature — low keeps it focused and repeatable; high makes it varied and creative.
- Top-p / top-k — limit the pool of candidate tokens the model is allowed to draw from.
- Together they trade off consistency against surprise — think in directions, not magic numbers.
Reliable or Creative
The right setting depends entirely on what the step is for. There is no single “best” value.
Reasoning Effort
Many models can “think” more before they answer — a separate lever from sampling.
- More effort helps hard, multi-step tasks — planning, tricky logic, careful analysis.
- It costs more time and tokens, so it is not free.
- For simple lookups or formatting, extra effort is usually wasted.
Settings Per Step
The same settings rarely fit every part of a pipeline. Treat each step as its own decision.
Build It
How to implement: run the same prompt twice — once at a low temperature, once at a higher one — and compare the outputs. Pick the setting that fits the task, not the one that looks impressive.
- Weekly AI Tasks tracker — parse incoming messages with LOW temperature, so the structure is consistent and checkable.
- Personal brand site — draft prose with a bit more freedom, then tighten; but keep facts grounded regardless of temperature.
What you learned
Temperature and top-p / top-k trade consistency for creativity; reasoning effort buys careful thinking at the cost of time and tokens. Match every setting to the step, and keep facts grounded no matter what.