Becoming self-instruct
Becoming self-instruct: introducing early stopping criteria for minimal instruct tuning
- Identifier
- arXiv:2307.03692
- Published
- Jul 5, 2023
- Authors
- 7
- Type
- Paper
- Area
- Enterprise-optimized models
In one paragraph
The Instruction Following Score (IFS) measures how well a model follows instructions. Models learn to follow instructions early in fine-tuning, and IFS shows when to stop, before further tuning changes the model's underlying knowledge.
- a new metric for instruction-following ability
- IFS
- a new metric for instruction-following ability
- LLaMA models studied during supervised fine-tuning
- 7B, 13B
- LLaMA models studied during supervised fine-tuning
FIG. 02: Mechanism
How it works
Build the probe set
Instructions come from the OpenAssistant chat dataset. About half are cut off mid-sentence, giving 7,340 prompts that mix full instructions with fragments.
Label the tone
A binary classifier (BERT, 0.970 accuracy) marks each response as answer-like, or as a continuation that completes the prompt or writes the next instruction.
Score IFS
IFS is the share of answer-like responses. Base LLaMA-7B scores 0.34 with no prompt wrapper; GPT-3.5-turbo scores 0.90.
Track it during fine-tuning
LLaMA 7B and 13B are fine-tuned on about 410k gpt4all examples. IFS levels off at 0.9–0.95 after roughly 8k examples.
Stop at the plateau
A 100-question objectivity test (ObjecQA) changes most after IFS plateaus. Stopping there keeps the instruction format while limiting change to the model's underlying behavior.
FIG. 03: Interactive figure
Watch a base model learn to answer
Scrub through fine-tuning, check each response's tone, then stop at the IFS plateau to see how much semantic drift you avoid.
Loading interactive demo…
FIG. 04: Notes
Summary
The Instruction Following Score (IFS) measures the share of a model’s responses that are properly formatted answers rather than partial continuations of the prompt. IFS separates base models from instruct models, and it can track instruction tuning as it happens.
Tracking IFS during supervised fine-tuning of LLaMA 7B and 13B showed that models learn to follow instructions early in training. Continuing to fine-tune beyond that point mostly changes the model’s underlying semantics, which we measured with an auxiliary objectivity metric (ObjecQA). Semantic change is steepest exactly when IFS plateaus, so IFS makes a natural early-stopping criterion for minimal instruct tuning.
Reference
@misc{alshikh2023becoming,
title={Becoming self-instruct: introducing early stopping criteria for minimal instruct tuning},
author={Waseem AlShikh and Manhal Daaboul and Kirk Goddard and Brock Imel and Kiran Kamble and Parikshith Kulkarni and Melisa Russak},
year={2023},
eprint={2307.03692},
archivePrefix={arXiv},
primaryClass={cs.CL}
}