This repository has been archived by the owner on Apr 10, 2024. It is now read-only.
change fft image parametrization to use only one op for the batch #200
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Current Lucid FFT image parametrization creates a separate
irfft
op per image in a batch. If you create just one FFT op and send all images through it, it is much faster. Essentially eachirfft
op you put in the graph incurs a ~7ms CPU cost for a forward pass and 8ms CPU cost per backward pass. These per op costs are paid in series on each sess.run!Seems to be related to a per-op recomputation of an execution plan that's done each time in a session.run. This behavior is described here: tensorflow/tensorflow#6541 (comment)
Results on a V100 for a neuron vis with batch of 32 and 128 steps:
https://colab.research.google.com/drive/1jfQP9E6M8piGaHxy75Tl23VeTx-QscL9#scrollTo=VFN7L_QNuwOI
time with current parametrization: 45.98s
time with updated parametrization: 12.89s
TF timelines that show the issue with a batch of 16 items:
![Screenshot 2019-10-01 11 53 46](https://user-images.githubusercontent.com/800877/66002536-c54e2a80-e458-11e9-9f12-4b7ec47fb67c.png)
before (250 ms):
after (30 ms):
![Screenshot 2019-10-01 12 24 57](https://user-images.githubusercontent.com/800877/66002601-e7e04380-e458-11e9-94bd-a079ca0f1e59.png)