It lets BERT use both preceding and following context to predict more fluent and natural words.