Rename MTEB(Retrieval w/Instructions) to followIR

[Muennighoff]

FollowIR models are gone from the leaderboard

[KennethEnevoldsen]

It is still there the name is simply different. @orionw I will just check in with you if you want to rename it to followIR (I am not sure who put the original name)

[Muennighoff]

I mean the FollowIR Cross-Encoders are gone? Maybe it is because we do not have an implementation?

[orionw]

We do have the implementation and results were previously in the results GitHub.

As far as naming, I’m fine either way. “FollowIR” is great. Sometime we can make a meta-benchmark of all the ones people have made.

[KennethEnevoldsen]

Hmm, so just a question of why they are not being fetched. Will look into it.

As far as naming, I’m fine either way. “FollowIR” is great.

Also fine with either. I will leave it to you then to submit a PR if you have a preference.

Sometime we can make a meta-benchmark of all the ones people have made.

Do you mean like an overview benchmark? Like MTEB(eng), MTEB(Multilingual), MTEB(Europe), FollowIR, LongEmbed, Code? (or a similar set)

Should be quite easy to do with the new aggregate task implementation. If this is what you mean we can create a new issue on it

[KennethEnevoldsen]

Renamed it to FollowIR as the introduced version on all MTEB benchmark #2024