Don't run_build when run_binary_tests (pytorch#59982)

Summary:
pytorch#59889 wasn't a proper revert of pytorch#58778. This PR fixes that.

Pull Request resolved: pytorch#59982

Reviewed By: seemethere

Differential Revision: D29114129

Pulled By: samestep

fbshipit-source-id: b40563db6ff1153a5f759639978279f5fcbccaa9

.github/pytorch-circleci-labels.yml

@@ -9,6 +9,8 @@ labels_to_circle_params:
         - release/.*
       tags:
         - v[0-9]+(\.[0-9]+)*-rc[0-9]+
+    set_to_false:
+      - run_build
   ci/master:
     parameter: run_master_build
   ci/slow-gradcheck: