Definition of Pre-training

Pre-training is the initial process of training a neural network on a large amount of data before later 'fine-tuning'.