Questions about hyperparameters in Llama-3.1-Tulu-3-8B Reproduction

[wgimperial]

For the 8B model, the total number of episodes is set to 10,000,000, while for the 70B model, it is only 400,000. Why is there such a large difference?

[wgimperial]

The multi-node gpus training failed during the RLVR stage. Requesting assistance.The error message as follows:

[ProcessGroupGloo.cpp:143]Gloo connectFullMesh failed with[../third_party/gloo/gloo/transport/pair.cc:144]

It shouldn't be NCCL communication, so why is there a Gloo error? Do you have any troubleshooting suggestions？

[vwxyzjn]

For the 8B model, the total number of episodes is set to 10,000,000, while for the 70B model, it is only 400,000. Why is there such a large difference?

We kind of let it train for a while and find the best intermediate checkpoints. As a result, the max episode is a bit arbitrary.

The multi-node gpus training failed during the RLVR stage. Requesting assistance.The error message as follows:

We use gloo as the distributed group to sync latest policy weights to the inference vllm worker. Not sure why you are running into this issue. Maybe double check if you follow our installation here: https://github.com/allenai/open-instruct?tab=readme-ov-file#setup

[wgimperial]

we build docker image follow https://github.com/allenai/open-instruct?tab=readme-ov-file#setup

We are using SLURM to submit multi-node RLVR training job. Does it affect the training script?

[vwxyzjn]

See the notes on distributed launch here https://github.com/allenai/open-instruct/blob/main/docs/tulu3.md#llama-31-tulu-3-70b-reproduction