Different results when running test.cpp

[ba5t1an]

I’ve used the test.cpp as a reference to test evaluation on my own images with a pretrained model. But whenever I call predict_classes it gives different class predictions for the same images after restarting the test program.

Here is an example:

After training the net, I want to evaluate it on two images. When I start the compiled program, it prints the float* which is output of the network, the expected class and the predicted one.

Run0 gives:

output: 0, 0, 1,
classes: 2 2
output: 0, 0, 1,
classed: 1 2
test time: 0.0060997 seconds
records: 2
speed: 327.885 records/second
accuracy: 50%

Run1 gives:

output: 2.40694e-08, 4.19541e-13, 1,
classes: 2 2
output: 1.12632e-08, 1, 4.18518e-14,
classes: 1 1
test time: 0.002543 seconds
records: 2
speed: 786.473 records/second
accuracy: 100%

The output of Run1 seems way more plausible as Run0. The problem here is, that it changes with every run.

Update: I’ve tracked the problem further and it seems to only happens with network having a semi_stochastic_pooling layer.

Update: Same happens when applying dropout layers. Seems like whenever stochastic variables are involved, your library breaks.

[gnawice]

Dropout layers are not used unless training. So maybe there is a bug in your test app.

I took a look at the pooling code since I've not looked at it for some time. It does seem that you will get the randomness during inference with the current implementation. If implemented as explained in the paper, it would basically use a weighted average during inference. I believe replacing the pooling layer with max pooling after training is a good work around (you should be able to just change the text in the model file) if you need repeat-ability. If average pooling is ever added, this should be revisited.