Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cannot reproduce paper results with default configuration #8

Closed
JoursBleu opened this issue May 2, 2022 · 1 comment
Closed

cannot reproduce paper results with default configuration #8

JoursBleu opened this issue May 2, 2022 · 1 comment

Comments

@JoursBleu
Copy link

We are using docker image “nvcr.io/nvidia/pytorch:22.03-py3” from NVIDIA
git clone https://github.com/sj-li/MS-TCN2.git
cd MS-TCN2
ln -s $data_path dat
# dataset in breakfast, 50salads, gtea
# split in (1-5) for 50salads and (1-4) for the other datasets
python3 main.py --action=train --dataset=$dataset --split=$split --num_epochs=100 --num_layers_PG=11 --num_layers_R=10 --num_R=3

The results in paper which is in TABLE 16:
image

And all the results mismatch with paper somehow:

<style> </style>
  F1 0.10 F1 0.25 F1 0.50 Edit Acc
50salads split 1 73.4 70.1 60.2 64.2 76.7
50salads split 2 72.2 69.2 59.8 64.1 81.7
50salads split 3 71.5 68.4 62.3 66.3 75.2
50salads split 4 79.3 77.0 69.1 71.6 83.3
50salads split 5 80.2 77.8 72.6 71.5 86.7
gtea split 1 80.1 77.4 64.6 71.5 74.4
gtea split 2 83.7 81.0 70.1 76.6 78.3
gtea split 3 90.6 89.1 79.2 88.2 79.2
gtea split 4 87.9 85.6 70.5 83.1 74.9

Are these mismatch reasonable, or did I miss any configuration?

@rickywrq
Copy link

@JoursBleu Have you figured out what is the reason for that?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants