The serving layer watches generated tokens and truncates output at the matching boundary. It helps separate records or turns, but accidental occurrences can stop a valid response early.
A stop sequence is a configured token pattern that ends generation when the model emits it.
The serving layer watches generated tokens and truncates output at the matching boundary. It helps separate records or turns, but accidental occurrences can stop a valid response early.