Scaling instruction-finetuned language models
Finetuning language models on a collection of datasets phrased as instructions has been
shown to improve model performance and generalization to unseen tasks. In this paper we�…
shown to improve model performance and generalization to unseen tasks. In this paper we�…
How far can camels go? exploring the state of instruction tuning on open resources
In this work we explore recent advances in instruction-tuning language models on a range of
open instruction-following datasets. Despite recent claims that open models can be on par�…
open instruction-following datasets. Despite recent claims that open models can be on par�…
Maybe only 0.5% data is needed: A preliminary exploration of low training data instruction tuning
Instruction tuning for large language models (LLMs) has gained attention from researchers
due to its ability to unlock the potential of LLMs in following instructions. While instruction�…
due to its ability to unlock the potential of LLMs in following instructions. While instruction�…
Exploring the benefits of training expert language models over instruction tuning
Abstract Recently, Language Models (LMs) instruction-tuned on multiple tasks, also known
as multitask-prompted fine-tuning (MT), have shown capabilities to generalize to unseen�…
as multitask-prompted fine-tuning (MT), have shown capabilities to generalize to unseen�…
Grips: Gradient-free, edit-based instruction search for prompting large language models
Providing natural language instructions in prompts is a useful new paradigm for improving
task performance of large language models in a zero-shot setting. Recent work has aimed to�…
task performance of large language models in a zero-shot setting. Recent work has aimed to�…
Opt-iml: Scaling language model instruction meta learning through the lens of generalization
Recent work has shown that fine-tuning large pre-trained language models on a collection
of tasks described via instructions, aka instruction-tuning, improves their zero and few-shot�…
of tasks described via instructions, aka instruction-tuning, improves their zero and few-shot�…
Poisoning language models during instruction tuning
Instruction-tuned LMs such as ChatGPT, FLAN, and InstructGPT are finetuned on datasets
that contain user-submitted examples, eg, FLAN aggregates numerous open-source�…
that contain user-submitted examples, eg, FLAN aggregates numerous open-source�…
# instag: Instruction tagging for analyzing supervised fine-tuning of large language models
Pre-trained large language models (LLMs) can understand and align with human
instructions by supervised fine-tuning (SFT). It is commonly believed that diverse and�…
instructions by supervised fine-tuning (SFT). It is commonly believed that diverse and�…
The flan collection: Designing data and methods for effective instruction tuning
We study the design decision of publicly available instruction tuning methods, by
reproducing and breaking down the development of Flan 2022 (Chung et al., 2022)�…
reproducing and breaking down the development of Flan 2022 (Chung et al., 2022)�…
Evaluating the zero-shot robustness of instruction-tuned language models
J Sun, C Shaib, BC Wallace�- arXiv preprint arXiv:2306.11270, 2023 - arxiv.org
Instruction fine-tuning has recently emerged as a promising approach for improving the zero-
shot capabilities of Large Language Models (LLMs) on new tasks. This technique has�…
shot capabilities of Large Language Models (LLMs) on new tasks. This technique has�…