Developing neural network image classification models often requires significant
architecture engineering. In this paper, we study a method to learn the model architectures�…

Neural networks are powerful and flexible models that work well for many difficult learning
tasks in image, speech and natural language understanding. Despite their success, neural�…

Abstract Model efficiency has become increasingly important in computer vision. In this
paper, we systematically study neural network architecture design choices for object�…

We present a simple self-training method that achieves 88.4% top-1 accuracy on ImageNet,
which is 2.0% better than the state-of-the-art model that requires 3.5 B weakly labeled�…

Transformer networks have a potential of learning longer-term dependency, but are limited
by a fixed-length context in the setting of language modeling. As a solution, we propose a�…

Masked language modeling (MLM) pre-training methods such as BERT corrupt the input by
replacing some tokens with [MASK] and then train a model to reconstruct the original tokens�…

We present SpecAugment, a simple data augmentation method for speech recognition.
SpecAugment is applied directly to the feature inputs of a neural network (ie, filter bank�…

The choice of activation functions in deep networks has a significant effect on the training
dynamics and task performance. Currently, the most successful and widely-used activation�…

Semi-supervised learning lately has shown much promise in improving deep learning
models when labeled data is scarce. Common among recent approaches is the use of�…

Many machine learning algorithms require the input to be represented as a fixed length
feature vector. When it comes to texts, one of the most common representations is bag-of�…