Skip to content

Commit

Permalink
Add files via upload
Browse files Browse the repository at this point in the history
  • Loading branch information
@OrigamiSL
OrigamiSL authored Nov 28, 2023
1 parent 4c2a196 commit 9689f61
Show file tree
Hide file tree
Showing 4 changed files with 84 additions and 20 deletions.
48 changes: 42 additions & 6 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,12 @@ Moreover, the default experiment settings/parameters of aforementioned seven bas
<th>2</th>
</tr>
<tr>
<th rowspan=2>Scaleformer</th>
<th rowspan=6>Scaleformer</th>
<th>Basic model</th>
<th>The basic model</th>
<th>FEDformer-f</th>
</tr>
<tr>
<th>scales</th>
<th>Scales in multi-scale</th>
<th>[16, 8, 4, 2, 1]</th>
Expand All @@ -227,7 +232,22 @@ Moreover, the default experiment settings/parameters of aforementioned seven bas
<th>2</th>
</tr>
<tr>
<th rowspan=2>PatchTST</th>
<th>mode_select</th>
<th>The mode selection method</th>
<th>random</th>
</tr>
<tr>
<th>modes</th>
<th>The number of modes</th>
<th>2</th>
</tr>
<tr>
<th>L</th>
<th>Ignore level</th>
<th>3</th>
</tr>
<tr>
<th rowspan=5>PatchTST</th>
<th>patch_len</th>
<th>Patch length</th>
<th>16</th>
Expand All @@ -238,6 +258,21 @@ Moreover, the default experiment settings/parameters of aforementioned seven bas
<th>8</th>
</tr>
<tr>
<th>n_head</th>
<th>The number of heads in multi-head attention mechanism</th>
<th>4</th>
</tr>
<tr>
<th>d_model</th>
<th>The hidden feature dimension</th>
<th>16</th>
</tr>
<tr>
<th>d_ff</th>
<th>Dimension of fcn</th>
<th>128</th>
</tr>
<tr>
<th rowspan=6>FiLM</th>
<th>d_model</th>
<th>The number of hidden dimensions</th>
Expand Down Expand Up @@ -291,7 +326,7 @@ We provide a complete command for training and testing FPPformer:

For multivariate forecasting:
```
python -u main.py --data <data> --features <features> --input_len <input_len> --pred_len <pred_len> --encoder_layer <encoder_layer> --patch_size <patch_size> --d_model <d_model> --learning_rate <learning_rate> --dropout <dropout> --batch_size <batch_size> --train_epochs <train_epochs> --patience <patience> --itr <itr> --train
python -u main.py --data <data> --features <features> --input_len <input_len> --pred_len <pred_len> --encoder_layer <encoder_layer> --patch_size <patch_size> --d_model <d_model> --Cross <Cross> --learning_rate <learning_rate> --dropout <dropout> --batch_size <batch_size> --train_epochs <train_epochs> --patience <patience> --itr <itr> --train
```
For univariate forecasting:
```
Expand All @@ -318,6 +353,7 @@ Here we provide a more detailed and complete command description for training an
| dropout | Dropout |
| encoder_layer | The number of encoder layers |
| patch_size | The size of each patch |
| Cross | whether to use cross-variable attention |
| itr | Experiments times |
| train_epochs | Train epochs of the second stage |
| batch_size | The batch size of training input data in the second stage |
Expand All @@ -326,7 +362,7 @@ Here we provide a more detailed and complete command description for training an


## Results
The experiment parameters of each data set are formated in the `Main.sh` files in the directory `./scripts/`. You can refer to these parameters for experiments, and you can also adjust the parameters to obtain better mse and mae results or draw better prediction figures. We provide the commands for obtain the results of FPPformer with longer input sequence lengths in the file `./scripts/LongInput.sh` and those of FPPformer with different encoder layers in the file `./scripts/ParaSen.sh`.
The experiment parameters of each data set are formated in the `Main.sh` files in the directory `./scripts/`. You can refer to these parameters for experiments, and you can also adjust the parameters to obtain better mse and mae results or draw better prediction figures. We provide the commands for obtain the results of FPPformer-Cross in the file `./scripts/Cross.sh`, those of FPPformer with longer input sequence lengths in the file `./scripts/LongInput.sh`, those of FPPformer with different encoder layers in the file `./scripts/ParaSen.sh`.

<p align="center">
<img src="./img/Multivariate.png" height = "500" alt="" align=center />
Expand All @@ -335,7 +371,7 @@ The experiment parameters of each data set are formated in the `Main.sh` files i
</p>

<p align="center">
<img src="./img/Univariate.png" height = "350" alt="" align=center />
<img src="./img/Univariate.png" height = "400" alt="" align=center />
<br><br>
<b>Figure 3.</b> Univariate forecasting results
</p>
Expand All @@ -348,7 +384,7 @@ Moreover, we present the full results of multivariate forecasting results with l
<b>Figure 4.</b> Multivariate forecasting results with long input lengths
</p>
<p align="center">
<img src="./img/Ablation.png" height = "300" alt="" align=center />
<img src="./img/Ablation.png" height = "400" alt="" align=center />
<br><br>
<b>Figure 5.</b> Ablation results with the prediction length of 720
</p>
Expand Down
27 changes: 19 additions & 8 deletions exp/exp_model.py
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
from data.data_loader import Dataset_ETT_hour, Dataset_ETT_min, Dataset_Custom
from exp.exp_basic import Exp_Basic
from FPPformer.FPPformer import FPPformer
from FPPformer.FPPformer_Cross import FPPformer_Cross

from utils.tools import EarlyStopping, adjust_learning_rate
from utils.metrics import metric
Expand All @@ -23,14 +24,24 @@ def __init__(self, args):
super(Exp_Model, self).__init__(args)

def _build_model(self):
model = FPPformer(
self.args.input_len,
self.args.pred_len,
self.args.encoder_layer,
self.args.patch_size,
self.args.d_model,
self.args.dropout
).float()
if self.args.Cross:
model = FPPformer_Cross(
self.args.input_len,
self.args.pred_len,
self.args.encoder_layer,
self.args.patch_size,
self.args.d_model,
self.args.dropout
).float()
else:
model = FPPformer(
self.args.input_len,
self.args.pred_len,
self.args.encoder_layer,
self.args.patch_size,
self.args.d_model,
self.args.dropout
).float()
if self.args.use_multi_gpu and self.args.use_gpu:
model = nn.DataParallel(model, device_ids=self.args.device_ids)
return model
Expand Down
15 changes: 9 additions & 6 deletions main.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,14 @@
parser.add_argument('--input_len', type=int, default=96, help='input length')
parser.add_argument('--pred_len', type=int, default=96, help='prediction length')

parser.add_argument('--enc_in', type=int, default=7, help='input variable size')
parser.add_argument('--dec_out', type=int, default=7, help='output variable size')
parser.add_argument('--enc_in', type=int, default=7, help='input size')
parser.add_argument('--dec_out', type=int, default=7, help='output size')
parser.add_argument('--d_model', type=int, default=28, help='hidden dims of model')
parser.add_argument('--encoder_layer', type=int, default=3)
parser.add_argument('--patch_size', type=int, default=6, help='the size of each patch')
parser.add_argument('--patch_size', type=int, default=6, help='nums of ODA layers')
parser.add_argument('--Cross', action='store_true',
help='whether to use cross-variable attention'
, default=False)

parser.add_argument('--dropout', type=float, default=0.05, help='dropout')
parser.add_argument('--num_workers', type=int, default=0, help='data loader num workers')
Expand Down Expand Up @@ -130,7 +133,7 @@
torch.cuda.empty_cache()
args.learning_rate = lr

path1 = './result.csv'
path1 = './result_Cross.csv'
if not os.path.exists(path1):
with open(path1, "a") as f:
write_csv = ['Time', 'Data', 'input_len', 'pred_len', 'encoder_layer', 'patch_size', 'Mean MSE',
Expand All @@ -141,9 +144,9 @@

mse = np.asarray(mse_total)
mae = np.asarray(mae_total)
avg_mse = np.mean(mse)
avg_mse = np.min(mse)
std_mse = np.std(mse)
avg_mae = np.mean(mae)
avg_mae = np.min(mae)
std_mae = np.std(mae)

print('|Mean|mse:{}, mae:{}|Std|mse:{}, mae:{}'.format(avg_mse, avg_mae, std_mse, std_mae))
Expand Down
14 changes: 14 additions & 0 deletions scripts/Cross.sh
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Solar
python -u main.py --data Solar --features M --input_len 96 --pred_len 96 --encoder_layer 3 --patch_size 6 --d_model 32 --learning_rate 0.0001 --dropout 0.1 --batch_size 16 --Cross --train_epochs 20 --itr 5 --train --patience 1

python -u main.py --data Solar --features M --input_len 96 --pred_len 192 --encoder_layer 3 --patch_size 6 --d_model 32 --learning_rate 0.0001 --dropout 0.1 --batch_size 16 --Cross --train_epochs 20 --itr 5 --train --patience 1

python -u main.py --data Solar --features M --input_len 96 --pred_len 336 --encoder_layer 3 --patch_size 6 --d_model 32 --learning_rate 0.0001 --dropout 0.1 --batch_size 16 --Cross --train_epochs 20 --itr 5 --train --patience 1

python -u main.py --data Solar --features M --input_len 96 --pred_len 720 --encoder_layer 3 --patch_size 6 --d_model 32 --learning_rate 0.0001 --dropout 0.1 --batch_size 16 --Cross --train_epochs 20 --itr 5 --train --patience 1

python -u main.py --data Solar --features M --input_len 192 --pred_len 720 --encoder_layer 3 --patch_size 6 --d_model 32 --learning_rate 0.0001 --dropout 0.1 --batch_size 16 --Cross --train_epochs 20 --itr 5 --train --patience 1

python -u main.py --data Solar --features M --input_len 384 --pred_len 720 --encoder_layer 3 --patch_size 6 --d_model 32 --learning_rate 0.0001 --dropout 0.1 --batch_size 16 --Cross --train_epochs 20 --itr 5 --train --patience 1

python -u main.py --data Solar --features M --input_len 576 --pred_len 720 --encoder_layer 3 --patch_size 6 --d_model 32 --learning_rate 0.0001 --dropout 0.1 --batch_size 8 --Cross --train_epochs 20 --itr 5 --train --patience 1

0 comments on commit 9689f61

Please sign in to comment.