Skip to content

Latest commit

 

History

History
66 lines (51 loc) · 11.7 KB

README.md

File metadata and controls

66 lines (51 loc) · 11.7 KB
Raw

STGCN

Introduction

STGCN is one of the first algorithms that adopt Graph Convolution Neural Networks for skeleton processing. We provide STGCN trained on NTURGB+D with 2D skeletons (HRNet) and 3D skeletons in both the original training setting and the PYSKL training setting. We provide checkpoints for four modalities: Joint, Bone, Joint Motion, and Bone Motion. The accuracy of each modality links to the weight file.

Citation

@inproceedings{yan2018spatial,
  title={Spatial temporal graph convolutional networks for skeleton-based action recognition},
  author={Yan, Sijie and Xiong, Yuanjun and Lin, Dahua},
  booktitle={Thirty-second AAAI conference on artificial intelligence},
  year={2018}
}
# If you use the STGCN with PYSKL practices in your work
@misc{duan2022pyskl,
    title={PYSKL: a toolbox for skeleton-based video understanding},
    author={PYSKL Contributors},
    howpublished = {\url{https://github.com/kennymckormick/pyskl}},
    year={2022}
}

Model Zoo

We release numerous checkpoints trained with various modalities, annotations on NTURGB+D and NTURGB+D 120. The accuracy of each modality links to the weight file.

Dataset Practice Annotation GPUs Training Epochs Joint Top1
Config Link: Weight Link		 Bone Top1
Config Link: Weight Link		 Joint Motion Top1
Config Link: Weight Link		 Bone-Motion Top1
Config Link: Weight Link		 Two-Stream Top1 Four Stream Top1
NTURGB+D XSub Vanilla Official 3D Skeleton 8 80 joint_config: 81.5 bone_config: 81.0 joint_motion_config: 79.9 bone_motion_config: 81.2 84.3 86.6
NTURGB+D XSub Vanilla HRNet 2D Skeleton 8 80 joint_config: 85.7 bone_config: 85.8 joint_motion_config: 81.6 bone_motion_config: 83.9 88.8 90.1
NTURGB+D XSub PYSKL Official 3D Skeleton 8 80 joint_config: 87.8 bone_config: 88.6 joint_motion_config: 85.8 bone_motion_config: 86.2 90.0 90.7
NTURGB+D XSub PYSKL HRNet 2D Skeleton 8 80 joint_config: 89.0 bone_config: 91.2 joint_motion_config: 86.7 bone_motion_config: 87.8 92.0 92.4
NTURGB+D XView Vanilla Official 3D Skeleton 8 80 joint_config: 90.1 bone_config: 87.7 joint_motion_config: 88.8 bone_motion_config: 88.3 91.4 93.2
NTURGB+D XView Vanilla HRNet 2D Skeleton 8 80 joint_config: 92.4 bone_config: 90.0 joint_motion_config: 92.0 bone_motion_config: 86.5 93.8 95.1
NTURGB+D XView PYSKL Official 3D Skeleton 8 80 joint_config: 95.5 bone_config: 95.0 joint_motion_config: 93.7 bone_motion_config: 92.8 96.2 96.5
NTURGB+D XView PYSKL HRNet 2D Skeleton 8 80 joint_config: 98.0 bone_config: 96.5 joint_motion_config: 95.6 bone_motion_config: 95.4 98.2 98.3
NTURGB+D 120 XSub PYSKL Official 3D Skeleton 8 80 joint_config: 82.1 bone_config: 83.7 joint_motion_config: 80.3 bone_motion_config: 80.6 85.6 86.2
NTURGB+D 120 XSub PYSKL HRNet 2D Skeleton 8 80 joint_config: 80.1 bone_config: 83.4 joint_motion_config: 78.6 bone_motion_config: 79.8 84.0 84.7
NTURGB+D 120 XSet PYSKL Official 3D Skeleton 8 80 joint_config: 84.5 bone_config: 85.8 joint_motion_config: 82.7 bone_motion_config: 83.0 87.5 88.4
NTURGB+D 120 XSet PYSKL HRNet 2D Skeleton 8 80 joint_config: 84.2 bone_config: 87.7 joint_motion_config: 82.5 bone_motion_config: 83.5 88.3 89.0

Note

  1. We use the linear-scaling learning rate (Initial LR ∝ Batch Size). If you change the training batch size, remember to change the initial LR proportionally.
  2. For Two-Stream results, we adopt the 1 (Joint):1 (Bone) fusion. For Four-Stream results, we adopt the 2 (Joint):2 (Bone):1 (Joint Motion):1 (Bone Motion) fusion.

Training & Testing

You can use the following command to train a model.

bash tools/dist_train.sh ${CONFIG_FILE} ${NUM_GPUS} [optional arguments]
# For example: train STGCN on NTURGB+D XSub (3D skeleton, Joint Modality) with 8 GPUs, with validation, with PYSKL practice, and test the last and the best (with best validation metric) checkpoint.
bash tools/dist_train.sh configs/stgcn/stgcn_pyskl_ntu60_xsub_3dkp/j.py 8 --validate --test-last --test-best

You can use the following command to test a model.

bash tools/dist_test.sh ${CONFIG_FILE} ${CHECKPOINT_FILE} ${NUM_GPUS} [optional arguments]
# For example: test STGCN on NTURGB+D XSub (3D skeleton, Joint Modality) with metrics `top_k_accuracy`, and dump the result to `result.pkl`.
bash tools/dist_test.sh configs/stgcn/stgcn_pyskl_ntu60_xsub_3dkp/j.py checkpoints/SOME_CHECKPOINT.pth 8 --eval top_k_accuracy --out result.pkl