Becoming self-instruct

Becoming self-instruct: introducing early stopping criteria for minimal instruct tuning

Waseem AlShikh, Manhal Daaboul, Kirk Goddard, Brock Imel, Kiran Kamble, Parikshith Kulkarni, Melisa Russak

FIG. 01 Build the probe set → Label the tone → Score IFS → Track it during fine-tuning → Stop at the plateau
Identifier
arXiv:2307.03692
Published
Jul 5, 2023
Authors
7
Type
Paper
Area
Enterprise-optimized models

In one paragraph

The Instruction Following Score (IFS) measures how well a model follows instructions. Models learn to follow instructions early in fine-tuning, and IFS shows when to stop, before further tuning changes the model's underlying knowledge.

a new metric for instruction-following ability
IFS
a new metric for instruction-following ability
LLaMA models studied during supervised fine-tuning
7B, 13B
LLaMA models studied during supervised fine-tuning

FIG. 02: Mechanism

How it works

  1. Build the probe set

    Instructions come from the OpenAssistant chat dataset. About half are cut off mid-sentence, giving 7,340 prompts that mix full instructions with fragments.

  2. Label the tone

    A binary classifier (BERT, 0.970 accuracy) marks each response as answer-like, or as a continuation that completes the prompt or writes the next instruction.

  3. Score IFS

    IFS is the share of answer-like responses. Base LLaMA-7B scores 0.34 with no prompt wrapper; GPT-3.5-turbo scores 0.90.

  4. Track it during fine-tuning

    LLaMA 7B and 13B are fine-tuned on about 410k gpt4all examples. IFS levels off at 0.9–0.95 after roughly 8k examples.

  5. Stop at the plateau

    A 100-question objectivity test (ObjecQA) changes most after IFS plateaus. Stopping there keeps the instruction format while limiting change to the model's underlying behavior.

FIG. 03: Interactive figure

Watch a base model learn to answer

Scrub through fine-tuning, check each response's tone, then stop at the IFS plateau to see how much semantic drift you avoid.

Loading interactive demo…

FIG. 04: Notes

Summary

The Instruction Following Score (IFS) measures the share of a model’s responses that are properly formatted answers rather than partial continuations of the prompt. IFS separates base models from instruct models, and it can track instruction tuning as it happens.

Tracking IFS during supervised fine-tuning of LLaMA 7B and 13B showed that models learn to follow instructions early in training. Continuing to fine-tune beyond that point mostly changes the model’s underlying semantics, which we measured with an auxiliary objectivity metric (ObjecQA). Semantic change is steepest exactly when IFS plateaus, so IFS makes a natural early-stopping criterion for minimal instruct tuning.

Reference

Cite this work
@misc{alshikh2023becoming,
  title={Becoming self-instruct: introducing early stopping criteria for minimal instruct tuning},
  author={Waseem AlShikh and Manhal Daaboul and Kirk Goddard and Brock Imel and Kiran Kamble and Parikshith Kulkarni and Melisa Russak},
  year={2023},
  eprint={2307.03692},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}