Large-Scale Experiments on DomainBed

We modified the DomainBed repo to incorporate natural language supervision.
For more information, please see the original repo.

Datasets

The list of available datasets with natural language supervision

• VLCS (Fang et al., 2013)
• PACS (Li et al., 2017)
• Office-Home (Venkateswara et al., 2017)
• A TerraIncognita (Beery et al., 2018) subset
• DomainNet (Peng et al., 2019)

Download the datasets:

python3 -m domainbed.scripts.download --data_dir=data

For natural language supervision, we use Per-Class definitions from Oxford English Dictionary.
Download the zipped file and place the texts directories into the corresponding datasets (Download Link).

Quick Start

Train a model:

python3 -m domainbed.scripts.train \
       --data_dir=data \
       --algorithm GVRT \
       --dataset VLCS \
       --test_env 0

Launch a sweep:

python -m domainbed.scripts.sweep launch \
       --data_dir=data \
       --output_dir=results \
       --command_launcher MyLauncher\
       --algorithms GVRT \
       --datasets VLCS PACS OfficeHome TerraIncognita DomainNet \
       --n_hparams 20 \
       --n_trials 3

To view the results of your sweep:

python -m domainbed.scripts.collect_results --input_dir=results

Experimental Results on DomainBed

We report averaged results across three independent runs.