eos, bos 토큰

by r01ex - opened Jun 28

r01ex

Jun 28

•

현재 special_tokens_map.json 상 eos 토큰과 bos토큰 모두 "</s>"로 되어있는데 의도된건가요?

Owner Jul 11

네 의도된 겁니다. eos가 bos시작을 많은 상황에서 예측가능하기 때문입니다.

r01ex changed discussion status to closed Jul 11

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment