Automatic Graph Neural Architecture Search

• Auto-GNAS is a general parallel graph neural architecture search framework for researchers and developers to achieve different tasks on graph.

• The framework of Auto-GNAS is as follows:

News

• 2022.02.10 Our work Auto-GNAS: A Parallel Graph Neural Architecture Search Framework is accepted by TPDS 2022.

• 2021.04.15 Our work GraphPAS: Parallel Architecture Search for Graph Neural Networks is accepted by SIGIR 2021 [Code].

Characters

1. Highly customizable

• Auto-GNAS supports user-defined almost all module functions easily.

2. Parallel estimation

• Auto-GNAS provides an interface for user-defined search algorithms to easily achieve parallel evaluation capabilities based on CPU or GPU.

Installing For Ubuntu 16.04

• Ensure you have installed CUDA 10.2 before installing other packages

1. Nvidia and CUDA 10.2:

[Nvidia Driver] 
https://www.nvidia.cn/Download/index.aspx?lang=cn

[CUDA 10.2 Download and Install Command] 
#Download:
wget https://developer.download.nvidia.com/compute/cuda/10.2/Prod/local_installers/cuda_10.2.89_440.33.01_linux.run
#Install:
sudo sh cuda_10.2.89_440.33.01_linux.run

2. Python environment: recommending using Conda package manager to install

conda create -n autognas python=3.7
source activate autognas

3. Pytorch 1.8.1: execute the following command in your conda env automsr

pip install torch==1.8.1+cu102 torchvision==0.9.1+cu102 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html

4. Pytorch Geometric 2.0.2: execute the following command in your conda env automsr

pip install torch-scatter==2.0.9 torch-sparse==0.6.12 torch-cluster==1.5.9 torch-spline-conv==1.2.1 torch-geometric==2.0.2 -f https://data.pyg.org/whl/torch-1.8.0+cu102.html

5. Ray 1.7.0: execute the following command in your conda env automsr

pip install ray==1.7.0

Quick start

Users can refer to the following easy cases to understand how to:

• Define your graph data
• Define your configuration file
• Implement node classification task
• Implement graph classification task
• Implement link prediction task

User-defined

Auto-GNAS is very friendly for users to implement customization, users can freely define their own functional components as long as they follow the custom specification and Auto-GNAS will automatically load user-defined components. Users can know the custom specification of each functional component in the following list, which is very simple. The list of definable components is as follows:

1. Search Space

• Attention
• Aggregation
• Multi-head number
• Hidden dimension
• Activation
• Layer number
  • user can defined this parameter in the ini configuration file at the [gnn_parameter] gnn_layers

2. Search Algorithm

3. GNN Training

• optimizer function
• loss function
• evaluator function

4. Downstream Task Model

API for Parallel Estimation

Users can use the API to implement parallel evaluation capabilities for their own search algorithms

from autognas.parallel import ParallelOperater,ParallelConfig

# obtain the user-defined graph data object
graph = UserCustomGraphData()

# open parallel estimation mode
ParallelConfig(True)

# initialize parallel operator object
parallel_estimation = ParallelOperater(graph)

# obtain multiple GNN architectures as a list from search algorithm
gnn_architecture_list = UserCustomSearchAlgorithm()

# obtain the parallel estimation result
parallel_result = parallel_estimation.estimation(gnn_architecture_list)

API for GNN Search

This is the top-level API of Auto-GNAS, users can use the default search algorithm to achieve different graph tasks

import os
import configparser
from autognas.auto_model import AutoModel
from autognas.parallel import ParallelConfig
from autognas.datasets.planetoid import Planetoid

# open parallel estimation mode 
ParallelConfig(True)

# obtain graph object
graph = Planetoid("cora").data

# obtain gnn training and search algorithm configuration
config = configparser.ConfigParser()
config = os.path.abspath(os.path.join(os.getcwd(), "..")) + "/config/node_classification_config/graphpas.ini"

# obtain configuration dict
search_parameter = dict(config.items('search_parameter'))
gnn_parameter = dict(config.items("gnn_parameter"))

# automatic search the optimal gnn architecture and logging experiment result
AutoModel(graph, search_parameter, gnn_parameter)

API for GNN Training

The API can help users realize the training and testing of GNN models for user-defined gnn architecture

from autognas.model.stack_gcn import StackGcn
from autognas.datasets.planetoid import Planetoid

# obtain graph data object
graph = Planetoid("cora").data

# gnn architecture list 
architecture=['gcn', 'sum',  1, 64, 'tanh', 'gcn', 'sum', 1, 64, 'tanh']

# initialize GNN model object based on graph data and gnn architecture
model = StackGcn(graph_data=graph,gnn_architecture=architecture)

# gnn model training 
model.fit()

# gnn model testing
model.evaluate()

Default Configuration

1. Search Space

Architecture Component	Value
Attention	gat / gcn / cos / const / sym-gat / linear / gene-linear
Aggregation	mean / max / sum
Multi-head number	1 / 2 / 4 / 6 / 8
Hidden Dimension	8 / 16 / 32 / 64 / 128 / 256
Activation	tanh / sigmoid / relu / linear / relu6 / elu / leaky_relu / softplus

2. Search Algorithm

• graphpas
• graphnas
• genetic search
• random search

3. GNN Training

Name	Value
Optimizer function	adam
loss function	nll_loss / binary_cross_entropy / binary_cross_entropy_with_logits / cross_entropy_loss
evaluator function	accuracy / recall / f1 score / precision / roc_auc_score

4. Downstream Task Model

• node classification
• graph classification
• link prediction

5. Datasets

from autognas.datasets.planetoid import Planetoid

# node classification:cora; citeseer; pubmed for your configuration
graph = Planetoid(data_name="cora",train_splits=0.5, val_splits=0.3, shuffle_flag=True, random_seed=123).data 

# node classification:cora; citeseer; pubmed for default configuration
graph = Planetoid("cora").data

# graph classification: AIDS; BZR; COX2; DHFR; MUTAG; PROTEINS for your configuration
graph = Planetoid(data_name="AIDS",train_splits=0.6, val_splits=0.2, shuffle_flag=True, random_seed=55).data 

# graph classification: AIDS; BZR; COX2; DHFR; MUTAG; PROTEINS for default configuration
graph = Planetoid("ADIS").data

# link prediction: cora_lp; citeseer_lp; pubmed_lp for your configuration
graph = Planetoid(data_name="cora_lp",train_splits=0.6, val_splits=0.2, shuffle_flag=True, random_seed=55).data 

# link prediction: cora_lp; citeseer_lp; pubmed_lp for default configuration
graph = Planetoid("ADIS").data

Future Updating

Efficiency

• Data mini batch operation is decoupled from estimation process √

Function

• Reconstruct input graph data class standard for autognas, require the input graph data class that have done mini-batch data pre-processing √

Citing

If you think Auto-GNAS is useful tool for you, please cite our paper, thank you for your support:

@ARTICLE{9714826,
  author={Chen, Jiamin and Gao, Jianliang and Chen, Yibo and Oloulade, Babatounde MOCTARD and Lyu, Tengfei and Li, Zhao},
  journal={IEEE Transactions on Parallel and Distributed Systems}, 
  title={Auto-GNAS: A Parallel Graph Neural Architecture Search Framework}, 
  year={2022},
  volume={},
  number={},
  pages={1-1},
  doi={10.1109/TPDS.2022.3151895}}

@inproceedings{chen2021graphpas,
  title={GraphPAS: Parallel Architecture Search for Graph Neural Networks},
  author={Chen, Jiamin and Gao, Jianliang and Chen, Yibo and Oloulade, Moctard Babatounde and Lyu, Tengfei and Li, Zhao},
  booktitle={Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval},
  pages={2182--2186},
  year={2021}
}