Learning Rate during pretraining
#58
by
shuyuej
- opened
May I know the initial learning rate and scheduler settings during the pre-training process?
Thank you very much, and have a nice day!
Best regards,
Shuyue
July 18th, 2024
May I also know the max_grad_norm during the pre-training process?
Thank you very much, and have a nice day!
Best regards,
Shuyue
July 18th, 2024