1,固定leanring rate
batch size设成256,512,1024,训练结果eval结果差不多
2,固定batch_size,
learning rate设成0.1,0.01,0.001,0.0001
紫色是0.1,基本不收敛。绿色是0.01,trainloss好奇怪,不过eval不不错
红色是0.001,看着图比较舒服,,蓝色是0.0001,变化比较缓慢
1,固定leanring rate
batch size设成256,512,1024,训练结果eval结果差不多
2,固定batch_size,
learning rate设成0.1,0.01,0.001,0.0001
紫色是0.1,基本不收敛。绿色是0.01,trainloss好奇怪,不过eval不不错
红色是0.001,看着图比较舒服,,蓝色是0.0001,变化比较缓慢