Complementary Label Datasets

Abstract

This repo contains two datasets: CIFAR10 and CIFAR100 with human annotated complementary labels for complementary label learning tasks. We collect this complementary version of CIFAR100 in order to verify two important aspects of complementary label learning:

1. The distribution of complementary label is independent to the feature.
2. Collecting complementary labels can be cheaper than collecting ordinary labels when the number of classes is large.

Datasets

Here are the links to the main page of each datasets, containing the analysis of collected labels and the benchmark results by libcll, a pytorch-lightning based machine learning library designed for complementary learning task:

• CLCIFAR10
• CLCIFAR20

Download CLCIFAR10 and CLCIFAR20

# Download both CLCIFAR10 and CLCIFAR20
bash download.sh

Reference

• https://www.cs.toronto.edu/~kriz/cifar.html
• https://www.mturk.com
• https://github.com/empennage98/libcll