Teacher forcing

Last updated Mar 15, 2023 Edit Source

• Maximum Likelihood Training (i.e., teacher forcing)
• Trained to generate the next word $\color{blue}y_t^$ given a set of preceding words $\color{red}{y_{<t}^}$.
• 모델의 출력값이 다음 타임 스텝에 입력값으로 들어가는 방식. $$L = -\sum_{t=1}^4 logP({\color{blue}y_t^} | {\color{red}{y_{<t}^}})$$