Reproduction of RoPINN

Our study focuses on evaluating the effectiveness of RoPINN in solving partial differential equations (PDEs) and conducting extensive hyperparameter tuning to validate the robust- ness and reliability of the original claim. Furthermore, we test the variability robustness under different seeds as well as the behaviour with disabled region size calibration

RoPINN (NeurIPS 2024)

RoPINN: Region Optimized Physics-Informed Neural Networks. See Paper or Slides.

This paper proposes and theoretically studies a new training paradigm of PINNs as region optimization and presents RoPINN as a practical algorithm, which can bring the following benefits:

• Better generalization bound: Introducing "region" can theoretically decrease generalization error and provide a general theoretical framework that first reveals the balance between generalization and optimization.
• Efficient practical algorithm: We present RoPINN with a trust region calibration strategy, which can effectively accomplish the region optimization and reduce the gradient estimation error caused by sampling.
• Boost extensive backbones: RoPINN consistently improves various PINN backbones (i.e. PINN, KAN and PINNsFormer) on a wide range of PDEs (19 different tasks) without extra gradient calculation.

Point Optimization vs. Region Optimization

Unlike conventional point optimization, our proposed region optimization extends the optimization process of PINNs from isolated points to their continuous neighborhood region.

Figure 1. Comparison between previous methods and RoPINN.

Practical Algorithm

We present RoPINN for PINN training based on Monte Carlo sampling, which can effectively accomplish the region optimization without extra gradient calculation. A trust region calibration strategy is proposed to reduce the gradient estimation error caused by sampling for more trustworthy optimization.

Figure 2. RoPINN algorithm.

Get Started

1. Install Python 3.8 or Python 3.9 and Pytorch 1.13.0. For convenience, execute the following command.

pip install -r requirements.txt

2. Train and evaluate model. We provide the experiment scripts of all benchmarks under the folder ./scripts/. You can reproduce the experiment results as the following examples:

bash scripts/1d_reaction_point.sh # canonical point optimization
bash scripts/1d_reaction_region.sh # RoPINN: region optimization
bash scripts/1d_wave_point.sh # canonical point optimization
bash scripts/1d_wave_region.sh # RoPINN: region optimization
bash scripts/convection_point.sh # canonical point optimization
bash scripts/convection_region.sh # RoPINN: region optimization

Specifically, we have included the following PINN models in this repo:

• PINN (Journal of Computational Physics 2019) [Paper]
• FLS - (IEEE Transactions on Artificial Intelligence 2022) [Paper]
• QRes - (SIAM 2021) [Paper]
• KAN - (arXiv 2024) [Paper]
• PINNsFormer - (ICLR 2024) [Paper]

Results

We have experimented with 19 different PDE tasks. See our paper for the full results.

Figure 3. Part of experimental results of RoPINN.

Citation

If you find this repo useful, please cite our paper.

@inproceedings{wu2024ropinn,
  title={RoPINN: Region Optimized Physics-Informed Neural Networks},
  author={Haixu Wu and Huakun Luo and Yuezhou Ma and Jianmin Wang and Mingsheng Long},
  booktitle={Advances in Neural Information Processing Systems},
  year={2024}
}

Contact

If you have any questions or want to use the code, please contact [email protected].

Acknowledgement

We appreciate the following GitHub repos a lot for their valuable code base or datasets:

https://github.com/AdityaLab/pinnsformer

https://github.com/i207M/PINNacle