Skip to content

Commit

Permalink
Improve alignment in effectiveness tables in regression documentation (…
Browse files Browse the repository at this point in the history 
…castorini#1920)

As we add more conditions in a regression test, the effectiveness tables increase in width.
At some point, they require horizontal scrolling. Combining separate tables (for each metric)
into a single table improves alignment in this case and greatly increases readability.
  • Loading branch information
@lintool
lintool authored Jun 26, 2022
1 parent a7c3b49 commit e90bb3c
Show file tree
Hide file tree
Showing 262 changed files with 856 additions and 2,601 deletions.
7 changes: 2 additions & 5 deletions docs/regressions-backgroundlinking18.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -77,12 +77,9 @@ tools/eval/trec_eval.9.0.4/trec_eval -c -M1000 -m map -c -M1000 -m ndcg_cut.5 sr

With the above commands, you should be able to reproduce the following results:

| MAP | BM25 | +RM3 | +RM3+DF |
| **MAP** | **BM25** | **+RM3** | **+RM3+DF**|
|:-------------------------------------------------------------------------------------------------------------|-----------|-----------|-----------|
| [TREC 2018 Topics](../src/main/resources/topics-and-qrels/topics.backgroundlinking18.txt) | 0.2490 | 0.2642 | 0.2692 |


| nDCG@5 | BM25 | +RM3 | +RM3+DF |
|:-------------------------------------------------------------------------------------------------------------|-----------|-----------|-----------|
| **nDCG@5** | **BM25** | **+RM3** | **+RM3+DF**|
| [TREC 2018 Topics](../src/main/resources/topics-and-qrels/topics.backgroundlinking18.txt) | 0.3293 | 0.3526 | 0.4171 |

7 changes: 2 additions & 5 deletions docs/regressions-backgroundlinking19.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -77,12 +77,9 @@ tools/eval/trec_eval.9.0.4/trec_eval -c -M1000 -m map -c -M1000 -m ndcg_cut.5 sr

With the above commands, you should be able to reproduce the following results:

| MAP | BM25 | +RM3 | +RM3+DF |
| **MAP** | **BM25** | **+RM3** | **+RM3+DF**|
|:-------------------------------------------------------------------------------------------------------------|-----------|-----------|-----------|
| [TREC 2019 Topics](../src/main/resources/topics-and-qrels/topics.backgroundlinking19.txt) | 0.3029 | 0.3786 | 0.3154 |


| nDCG@5 | BM25 | +RM3 | +RM3+DF |
|:-------------------------------------------------------------------------------------------------------------|-----------|-----------|-----------|
| **nDCG@5** | **BM25** | **+RM3** | **+RM3+DF**|
| [TREC 2019 Topics](../src/main/resources/topics-and-qrels/topics.backgroundlinking19.txt) | 0.4785 | 0.5217 | 0.5051 |

7 changes: 2 additions & 5 deletions docs/regressions-backgroundlinking20.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -77,12 +77,9 @@ tools/eval/trec_eval.9.0.4/trec_eval -c -M1000 -m map -c -M1000 -m ndcg_cut.5 sr

With the above commands, you should be able to reproduce the following results:

| MAP | BM25 | +RM3 | +RM3+DF |
| **MAP** | **BM25** | **+RM3** | **+RM3+DF**|
|:-------------------------------------------------------------------------------------------------------------|-----------|-----------|-----------|
| [TREC 2020 Topics](../src/main/resources/topics-and-qrels/topics.backgroundlinking20.txt) | 0.3286 | 0.4519 | 0.3438 |


| nDCG@5 | BM25 | +RM3 | +RM3+DF |
|:-------------------------------------------------------------------------------------------------------------|-----------|-----------|-----------|
| **nDCG@5** | **BM25** | **+RM3** | **+RM3+DF**|
| [TREC 2020 Topics](../src/main/resources/topics-and-qrels/topics.backgroundlinking20.txt) | 0.5231 | 0.5673 | 0.5316 |

12 changes: 3 additions & 9 deletions docs/regressions-beir-v1.0.0-arguana-flat-wp.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,16 +54,10 @@ tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 src/main/resources/topics

With the above commands, you should be able to reproduce the following results:

| nDCG@10 | BM25 |
| **nDCG@10** | **BM25** |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| BEIR (v1.0.0): ArguAna | 0.3639 |


| R@100 | BM25 |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@100** | **BM25** |
| BEIR (v1.0.0): ArguAna | 0.8791 |


| R@1000 | BM25 |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@1000** | **BM25** |
| BEIR (v1.0.0): ArguAna | 0.9602 |
12 changes: 3 additions & 9 deletions docs/regressions-beir-v1.0.0-arguana-flat.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,16 +53,10 @@ tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 src/main/resources/topics

With the above commands, you should be able to reproduce the following results:

| nDCG@10 | BM25 |
| **nDCG@10** | **BM25** |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| BEIR (v1.0.0): ArguAna | 0.3970 |


| R@100 | BM25 |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@100** | **BM25** |
| BEIR (v1.0.0): ArguAna | 0.9324 |


| R@1000 | BM25 |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@1000** | **BM25** |
| BEIR (v1.0.0): ArguAna | 0.9872 |
12 changes: 3 additions & 9 deletions docs/regressions-beir-v1.0.0-arguana-multifield.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,16 +54,10 @@ tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 src/main/resources/topics

With the above commands, you should be able to reproduce the following results:

| nDCG@10 | BM25 |
| **nDCG@10** | **BM25** |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| BEIR (v1.0.0): ArguAna | 0.4142 |


| R@100 | BM25 |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@100** | **BM25** |
| BEIR (v1.0.0): ArguAna | 0.9431 |


| R@1000 | BM25 |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@1000** | **BM25** |
| BEIR (v1.0.0): ArguAna | 0.9893 |
12 changes: 3 additions & 9 deletions docs/regressions-beir-v1.0.0-arguana-splade-distil-cocodenser-medium.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -89,18 +89,12 @@ tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 src/main/resources/topics

With the above commands, you should be able to reproduce the following results:

| nDCG@10 | SPLADE-distill CoCodenser Medium|
| **nDCG@10** | **SPLADE-distill CoCodenser Medium**|
|:-------------------------------------------------------------------------------------------------------------|-----------|
| BEIR (v1.0.0): ArguAna | 0.5210 |


| R@100 | SPLADE-distill CoCodenser Medium|
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@100** | **SPLADE-distill CoCodenser Medium**|
| BEIR (v1.0.0): ArguAna | 0.9822 |


| R@1000 | SPLADE-distill CoCodenser Medium|
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@1000** | **SPLADE-distill CoCodenser Medium**|
| BEIR (v1.0.0): ArguAna | 0.9950 |


Expand Down
12 changes: 3 additions & 9 deletions docs/regressions-beir-v1.0.0-arguana-unicoil-noexp.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -57,16 +57,10 @@ tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 src/main/resources/topics

With the above commands, you should be able to reproduce the following results:

| nDCG@10 | uniCOIL no expansion|
| **nDCG@10** | **uniCOIL no expansion**|
|:-------------------------------------------------------------------------------------------------------------|-----------|
| BEIR (v1.0.0): ArguAna | 0.3959 |


| R@100 | uniCOIL no expansion|
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@100** | **uniCOIL no expansion**|
| BEIR (v1.0.0): ArguAna | 0.9225 |


| R@1000 | uniCOIL no expansion|
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@1000** | **uniCOIL no expansion**|
| BEIR (v1.0.0): ArguAna | 0.9794 |
12 changes: 3 additions & 9 deletions docs/regressions-beir-v1.0.0-bioasq-flat-wp.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,16 +54,10 @@ tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 src/main/resources/topics

With the above commands, you should be able to reproduce the following results:

| nDCG@10 | BM25 |
| **nDCG@10** | **BM25** |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| BEIR (v1.0.0): BioASQ | 0.4191 |


| R@100 | BM25 |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@100** | **BM25** |
| BEIR (v1.0.0): BioASQ | 0.6880 |


| R@1000 | BM25 |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@1000** | **BM25** |
| BEIR (v1.0.0): BioASQ | 0.8420 |
12 changes: 3 additions & 9 deletions docs/regressions-beir-v1.0.0-bioasq-flat.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,16 +53,10 @@ tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 src/main/resources/topics

With the above commands, you should be able to reproduce the following results:

| nDCG@10 | BM25 |
| **nDCG@10** | **BM25** |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| BEIR (v1.0.0): BioASQ | 0.5225 |


| R@100 | BM25 |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@100** | **BM25** |
| BEIR (v1.0.0): BioASQ | 0.7687 |


| R@1000 | BM25 |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@1000** | **BM25** |
| BEIR (v1.0.0): BioASQ | 0.9030 |
12 changes: 3 additions & 9 deletions docs/regressions-beir-v1.0.0-bioasq-multifield.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,16 +54,10 @@ tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 src/main/resources/topics

With the above commands, you should be able to reproduce the following results:

| nDCG@10 | BM25 |
| **nDCG@10** | **BM25** |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| BEIR (v1.0.0): BioASQ | 0.4646 |


| R@100 | BM25 |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@100** | **BM25** |
| BEIR (v1.0.0): BioASQ | 0.7145 |


| R@1000 | BM25 |
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@1000** | **BM25** |
| BEIR (v1.0.0): BioASQ | 0.8428 |
12 changes: 3 additions & 9 deletions docs/regressions-beir-v1.0.0-bioasq-splade-distil-cocodenser-medium.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -89,18 +89,12 @@ tools/eval/trec_eval.9.0.4/trec_eval -c -m recall.1000 src/main/resources/topics

With the above commands, you should be able to reproduce the following results:

| nDCG@10 | SPLADE-distill CoCodenser Medium|
| **nDCG@10** | **SPLADE-distill CoCodenser Medium**|
|:-------------------------------------------------------------------------------------------------------------|-----------|
| BEIR (v1.0.0): BioASQ | 0.5035 |


| R@100 | SPLADE-distill CoCodenser Medium|
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@100** | **SPLADE-distill CoCodenser Medium**|
| BEIR (v1.0.0): BioASQ | 0.7422 |


| R@1000 | SPLADE-distill CoCodenser Medium|
|:-------------------------------------------------------------------------------------------------------------|-----------|
| **R@1000** | **SPLADE-distill CoCodenser Medium**|
| BEIR (v1.0.0): BioASQ | 0.8904 |


Expand Down
Loading

0 comments on commit e90bb3c

Please sign in to comment.