Skip to content

Latest commit

 

History

History
94 lines (76 loc) · 3.35 KB

Pruner.md

File metadata and controls

94 lines (76 loc) · 3.35 KB

Pruner on NNI Compressor

Level Pruner

This is one basic pruner: you can set a target sparsity level (expressed as a fraction, 0.6 means we will prune 60%).

We first sort the weights in the specified layer by their absolute values. And then mask to zero the smallest magnitude weights until the desired sparsity level is reached.

Usage

Tensorflow code

from nni.compression.tensorflow import LevelPruner
config_list = [{ 'sparsity': 0.8, 'op_types': 'default' }]
pruner = LevelPruner(config_list)
pruner(model_graph)

PyTorch code

from nni.compression.torch import LevelPruner
config_list = [{ 'sparsity': 0.8, 'op_types': 'default' }]
pruner = LevelPruner(config_list)
pruner(model)

User configuration for Level Pruner

  • sparsity: This is to specify the sparsity operations to be compressed to

AGP Pruner

In To prune, or not to prune: exploring the efficacy of pruning for model compression, authors Michael Zhu and Suyog Gupta provide an algorithm to prune the weight gradually.

We introduce a new automated gradual pruning algorithm in which the sparsity is increased from an initial sparsity value si (usually 0) to a final sparsity value sf over a span of n pruning steps, starting at training step t0 and with pruning frequency ∆t: The binary weight masks are updated every ∆t steps as the network is trained to gradually increase the sparsity of the network while allowing the network training steps to recover from any pruning-induced loss in accuracy. In our experience, varying the pruning frequency ∆t between 100 and 1000 training steps had a negligible impact on the final model quality. Once the model achieves the target sparsity sf , the weight masks are no longer updated. The intuition behind this sparsity function in equation

Usage

You can prune all weight from 0% to 80% sparsity in 10 epoch with the code below.

First, you should import pruner and add mask to model.

Tensorflow code

from nni.compression.tensorflow import AGP_Pruner
config_list = [{
    'initial_sparsity': 0,
    'final_sparsity': 0.8,
    'start_epoch': 0,
    'end_epoch': 10,
    'frequency': 1,
    'op_types': 'default'
}]
pruner = AGP_Pruner(config_list)
pruner(tf.get_default_graph())

PyTorch code

from nni.compression.torch import AGP_Pruner
config_list = [{
    'initial_sparsity': 0,
    'final_sparsity': 0.8,
    'start_epoch': 0,
    'end_epoch': 10,
    'frequency': 1,
    'op_types': 'default'
}]
pruner = AGP_Pruner(config_list)
pruner(model)

Second, you should add code below to update epoch number when you finish one epoch in your training code.

Tensorflow code

pruner.update_epoch(epoch, sess)

PyTorch code

pruner.update_epoch(epoch)

You can view example for more information

User configuration for AGP Pruner

  • initial_sparsity: This is to specify the sparsity when compressor starts to compress
  • final_sparsity: This is to specify the sparsity when compressor finishes to compress
  • start_epoch: This is to specify the epoch number when compressor starts to compress, default start from epoch 0
  • end_epoch: This is to specify the epoch number when compressor finishes to compress
  • frequency: This is to specify every frequency number epochs compressor compress once, default frequency=1