PyTorch implementation of "Scaling and Masking: A New Paradigm of Data Sampling for Image and Video Quality Assessment" (arXiv), which has been accepted by AAAI-2024.
This code is modified from FAST-VQA.
For image quality assessment (IQA), please refer to IQA/demo_train_iqa_baseline.py.
For video quality assessment (VQA), please refer to VQA/demo_train.py to get the training result, and refer to VQA/demo_finetune.py to get the finetuning result. We also provide the training log for VQA.
The main idea/contribution lies in the data sampling, which can be found in IQA and VQA.
Make sure the configuration has been properly set in
- fast-sama-iqa.yml for IQA training;
- fast-SAMA-train.yml for VQA training on LSVQ;
- and fast-SAMA-tinetune.yml for VQA finetuning.
And please prepare the pretrained models of video-swin for VQA and swin-v2 for IQA.
We have provided the pretrained weights (trained on LSVQ train set): GoogleDrive / BaiDu (Code:xyns). please check the pretrained weights in ./VQA/pretrained_weights folder and put the weights in the folder.
To test on your own dataset or video files, please construct the dataset information as the examplar in ./VQA/examplar_data_labels, and set the configuration in fast-SAMA-test.yml. Run the file demo_test.py to check the details.
Different environment may induce possible fluctuation of performance.
Python 3.8.10
PyTorch 1.7.0
The installation can refer to FAST-VQA.
If you are interested in the work, or find the code helpful, please cite our work
@ARTICLE{sama,
title={Scaling and Masking: A New Paradigm of Data Sampling for Image and Video Quality Assessment},
author={Liu, Yongxu and Quan, yinghui and Xiao, guoyao and Li, Aobo and Wu, jinjian},
year={2024},
eprint={2401.02614},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
Feel free to contact me via [email protected] if any question or bug.