Masked Language Models (MLMs)

MLMs are a type of NLP model, which have a special token, usually [MASK], which is replaced with a word from the vocabulary. The model then predicts the word that was masked. For example, if the sentence is "The dog is [MASK] the cat", the model will predict "chasing" with high probability.