forked from Theano/Theano
-
Notifications
You must be signed in to change notification settings - Fork 0
/
HISTORY.txt
3513 lines (3091 loc) · 144 KB
/
HISTORY.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
.. _HISTORY:
=================
Old Release Notes
=================
Theano 1.0.0 (15th of November, 2017)
=====================================
This is a final release of Theano, version ``1.0.0``, with a lot of
new features, interface changes, improvements and bug fixes.
We recommend that everybody update to this version.
Highlights (since 0.9.0):
- Announcing that `MILA will stop developing Theano <https://groups.google.com/d/msg/theano-users/7Poq8BZutbY/rNCIfvAEAwAJ>`_
- conda packages now available and updated in our own conda channel ``mila-udem``
To install it: ``conda install -c mila-udem theano pygpu``
- Support NumPy ``1.13``
- Support pygpu ``0.7``
- Moved Python ``3.*`` minimum supported version from ``3.3`` to ``3.4``
- Added conda recipe
- Replaced deprecated package ``nose-parameterized`` with up-to-date package ``parameterized`` for Theano requirements
- Theano now internally uses ``sha256`` instead of ``md5`` to work on systems that forbid ``md5`` for security reason
- Removed old GPU backend ``theano.sandbox.cuda``. New backend ``theano.gpuarray`` is now the official GPU backend
- Make sure MKL uses GNU OpenMP
- **NB**: Matrix dot product (``gemm``) with ``mkl`` from conda
could return wrong results in some cases. We have reported the problem upstream
and we have a work around that raises an error with information about how to fix it.
- Improved elemwise operations
- Speed-up elemwise ops based on SciPy
- Fixed memory leaks related to elemwise ops on GPU
- Scan improvements
- Speed up Theano scan compilation and gradient computation
- Added meaningful message when missing inputs to scan
- Speed up graph toposort algorithm
- Faster C compilation by massively using a new interface for op params
- Faster optimization step, with new optional destroy handler
- Documentation updated and more complete
- Added documentation for RNNBlock
- Updated ``conv`` documentation
- Support more debuggers for ``PdbBreakpoint``
- Many bug fixes, crash fixes and warning improvements
A total of 71 people contributed to this release since 0.9.0, see list below.
Interface changes:
- Merged duplicated diagonal functions into two ops: ``ExtractDiag`` (extract a diagonal to a vector),
and ``AllocDiag`` (set a vector as a diagonal of an empty array)
- Removed op ``ExtractDiag`` from ``theano.tensor.nlinalg``, now only in ``theano.tensor.basic``
- Generalized ``AllocDiag`` for any non-scalar input
- Added new parameter ``target`` for MRG functions
- Renamed ``MultinomialWOReplacementFromUniform`` to ``ChoiceFromUniform``
- Changed ``grad()`` method to ``L_op()`` in ops that need the outputs to compute gradient
- Removed or deprecated Theano flags:
- ``cublas.lib``
- ``cuda.enabled``
- ``enable_initial_driver_test``
- ``gpuarray.sync``
- ``home``
- ``lib.cnmem``
- ``nvcc.*`` flags
- ``pycuda.init``
Convolution updates:
- Implemented separable convolutions for 2D and 3D
- Implemented grouped convolutions for 2D and 3D
- Added dilated causal convolutions for 2D
- Added unshared convolutions
- Implemented fractional bilinear upsampling
- Removed old ``conv3d`` interface
- Deprecated old ``conv2d`` interface
GPU:
- Added a meta-optimizer to select the fastest GPU implementations for convolutions
- Prevent GPU initialization when not required
- Added disk caching option for kernels
- Added method ``my_theano_function.sync_shared()`` to help synchronize GPU Theano functions
- Added useful stats for GPU in profile mode
- Added Cholesky op based on ``cusolver`` backend
- Added GPU ops based on `magma library <http://icl.cs.utk.edu/magma/software/>`_:
SVD, matrix inverse, QR, cholesky and eigh
- Added ``GpuCublasTriangularSolve``
- Added atomic addition and exchange for ``long long`` values in ``GpuAdvancedIncSubtensor1_dev20``
- Support log gamma function for all non-complex types
- Support GPU SoftMax in both OpenCL and CUDA
- Support offset parameter ``k`` for ``GpuEye``
- ``CrossentropyCategorical1Hot`` and its gradient are now lifted to GPU
- cuDNN:
- Official support for ``v6.*`` and ``v7.*``
- Added spatial transformation operation based on cuDNN
- Updated and improved caching system for runtime-chosen cuDNN convolution algorithms
- Support cuDNN v7 tensor core operations for convolutions with runtime timed algorithms
- Better support and loading on Windows and Mac
- Support cuDNN v6 dilated convolutions
- Support cuDNN v6 reductions for contiguous inputs
- Optimized ``SUM(x^2)``, ``SUM(ABS(X))`` and ``MAX(ABS(X))`` operations with cuDNN reductions
- Added new Theano flags ``cuda.include_path``, ``dnn.base_path`` and ``dnn.bin_path``
to help configure Theano when CUDA and cuDNN can not be found automatically
- Extended Theano flag ``dnn.enabled`` with new option ``no_check`` to help speed up cuDNN importation
- Disallowed ``float16`` precision for convolution gradients
- Fixed memory alignment detection
- Added profiling in C debug mode (with theano flag ``cmodule.debug=True``)
- Added Python scripts to help test cuDNN convolutions
- Automatic addition of cuDNN DLL path to ``PATH`` environment variable on Windows
- Updated ``float16`` support
- Added documentation for GPU float16 ops
- Support ``float16`` for ``GpuGemmBatch``
- Started to use ``float32`` precision for computations that don't support ``float16`` on GPU
New features:
- Implemented truncated normal distribution with box-muller transform
- Added ``L_op()`` overriding option for ``OpFromGraph``
- Added NumPy C-API based fallback implementation for ``[sd]gemv_`` and ``[sd]dot_``
- Implemented ``topk`` and ``argtopk`` on CPU and GPU
- Implemented ``max()`` and ``min()`` functions for booleans and unsigned integers types
- Added ``tensor6()`` and ``tensor7()`` in ``theano.tensor`` module
- Added boolean indexing for sub-tensors
- Added covariance matrix function ``theano.tensor.cov``
- Added a wrapper for `Baidu's CTC <https://github.com/baidu-research/warp-ctc>`_ cost and gradient functions
- Added scalar and elemwise CPU ops for modified Bessel function of order 0 and 1 from ``scipy.special``
- Added Scaled Exponential Linear Unit (SELU) activation
- Added sigmoid_binary_crossentropy function
- Added tri-gamma function
- Added ``unravel_index`` and ``ravel_multi_index`` functions on CPU
- Added modes ``half`` and ``full`` for ``Images2Neibs`` ops
- Implemented gradient for ``AbstractBatchNormTrainGrad``
- Implemented gradient for matrix pseudoinverse op
- Added new prop `replace` for ``ChoiceFromUniform`` op
- Added new prop ``on_error`` for CPU ``Cholesky`` op
- Added new Theano flag ``deterministic`` to help control how Theano optimize certain ops that have deterministic versions.
Currently used for subtensor Ops only.
- Added new Theano flag ``cycle_detection`` to speed-up optimization step by reducing time spending in inplace optimizations
- Added new Theano flag ``check_stack_trace`` to help check the stack trace during optimization process
- Added new Theano flag ``cmodule.debug`` to allow a debug mode for Theano C code. Currently used for cuDNN convolutions only.
- Added new Theano flag ``pickle_test_value`` to help disable pickling test values
Others:
- Kept stack trace for optimizations in new GPU backend
- Added deprecation warning for the softmax and logsoftmax vector case
- Added a warning to announce that C++ compiler will become mandatory in next Theano release ``0.11``
- Added ``R_op()`` for ``ZeroGrad``
- Added description for rnnblock
Other more detailed changes:
- Fixed invalid casts and index overflows in ``theano.tensor.signal.pool``
- Fixed gradient error for elemwise ``minimum`` and ``maximum`` when compared values are the same
- Fixed gradient for ``ARange``
- Removed ``ViewOp`` subclass during optimization
- Removed useless warning when profile is manually disabled
- Added tests for abstract conv
- Added options for `disconnected_outputs` to Rop
- Removed ``theano/compat/six.py``
- Removed ``COp.get_op_params()``
- Support of list of strings for ``Op.c_support_code()``, to help not duplicate support codes
- Macro names provided for array properties are now standardized in both CPU and GPU C codes
- Moved all C code files into separate folder ``c_code`` in every Theano module
- Many improvements for Travis CI tests (with better splitting for faster testing)
- Many improvements for Jenkins CI tests: daily testings on Mac and Windows in addition to Linux
Commiters since 0.9.0:
- Frederic Bastien
- Steven Bocco
- João Victor Tozatti Risso
- Arnaud Bergeron
- Mohammed Affan
- amrithasuresh
- Pascal Lamblin
- Reyhane Askari
- Alexander Matyasko
- Shawn Tan
- Simon Lefrancois
- Adam Becker
- Vikram
- Gijs van Tulder
- Faruk Ahmed
- Thomas George
- erakra
- Andrei Costinescu
- Boris Fomitchev
- Zhouhan LIN
- Aleksandar Botev
- jhelie
- xiaoqie
- Tegan Maharaj
- Matt Graham
- Cesar Laurent
- Gabe Schwartz
- Juan Camilo Gamboa Higuera
- Tim Cooijmans
- Anirudh Goyal
- Saizheng Zhang
- Yikang Shen
- vipulraheja
- Florian Bordes
- Sina Honari
- Chiheb Trabelsi
- Shubh Vachher
- Daren Eiri
- Joseph Paul Cohen
- Laurent Dinh
- Mohamed Ishmael Diwan Belghazi
- Jeff Donahue
- Ramana Subramanyam
- Bogdan Budescu
- Dzmitry Bahdanau
- Ghislain Antony Vaillant
- Jan Schlüter
- Nan Jiang
- Xavier Bouthillier
- fo40225
- mrTsjolder
- wyjw
- Aarni Koskela
- Adam Geitgey
- Adrian Keet
- Adrian Seyboldt
- Anmol Sahoo
- Chong Wu
- Holger Kohr
- Jayanth Koushik
- Lilian Besson
- Lv Tao
- Michael Manukyan
- Murugesh Marvel
- NALEPA
- Rebecca N. Palmer
- Zotov Yuriy
- dareneiri
- lrast
- morrme
- naitonium
Theano 1.0.0rc1 (30th of October, 2017)
=======================================
This release contains new features, improvements and bug fixes to prepare the upcoming release.
We recommend that every developer updates to this version.
Highlights:
- Make sure MKL uses GNU OpenMP
- **NB**: Matrix dot product (``gemm``) with ``mkl`` from conda
could return wrong results in some cases. We have reported the problem upstream
and we have a work around that raises an error with information about how to fix it.
- Optimized ``SUM(x^2)``, ``SUM(ABS(X))`` and ``MAX(ABS(X))`` operations with cuDNN reductions
- Added Python scripts to help test cuDNN convolutions
- Fixed invalid casts and index overflows in ``theano.tensor.signal.pool``
A total of 71 people contributed to this release since 0.9.0, see list below.
Commiters since 0.9.0:
- Frederic Bastien
- Steven Bocco
- João Victor Tozatti Risso
- Arnaud Bergeron
- Mohammed Affan
- amrithasuresh
- Pascal Lamblin
- Reyhane Askari
- Alexander Matyasko
- Shawn Tan
- Simon Lefrancois
- Adam Becker
- Vikram
- Gijs van Tulder
- Faruk Ahmed
- Thomas George
- erakra
- Andrei Costinescu
- Boris Fomitchev
- Zhouhan LIN
- Aleksandar Botev
- jhelie
- xiaoqie
- Tegan Maharaj
- Matt Graham
- Cesar Laurent
- Gabe Schwartz
- Juan Camilo Gamboa Higuera
- Tim Cooijmans
- Anirudh Goyal
- Saizheng Zhang
- Yikang Shen
- vipulraheja
- Florian Bordes
- Sina Honari
- Chiheb Trabelsi
- Shubh Vachher
- Daren Eiri
- Joseph Paul Cohen
- Laurent Dinh
- Mohamed Ishmael Diwan Belghazi
- Jeff Donahue
- Ramana Subramanyam
- Bogdan Budescu
- Dzmitry Bahdanau
- Ghislain Antony Vaillant
- Jan Schlüter
- Nan Jiang
- Xavier Bouthillier
- fo40225
- mrTsjolder
- wyjw
- Aarni Koskela
- Adam Geitgey
- Adrian Keet
- Adrian Seyboldt
- Anmol Sahoo
- Chong Wu
- Holger Kohr
- Jayanth Koushik
- Lilian Besson
- Lv Tao
- Michael Manukyan
- Murugesh Marvel
- NALEPA
- Rebecca N. Palmer
- Zotov Yuriy
- dareneiri
- lrast
- morrme
- naitonium
Theano 0.10.0beta4 (16th of October, 2017)
==========================================
This release contains new features, improvements and bug fixes to prepare the upcoming release candidate.
We recommend that every developer updates to this version.
Highlights:
- Announcing that `MILA will stop developing Theano <https://groups.google.com/d/msg/theano-users/7Poq8BZutbY/rNCIfvAEAwAJ>`_
- Bug fixes, crash fixes, warning improvements and documentation updates
A total of 70 people contributed to this release since 0.9.0, see list below.
Interface changes:
- Generalized ``AllocDiag`` for any non-scalar input
Convolution updates:
- Implemented fractional bilinear upsampling
cuDNN (GPU):
- Disallowed ``float16`` precision for convolution gradients
- Fixed memory alignment detection
- Added profiling in C debug mode (with theano flag ``cmodule.debug=True``)
New features:
- Implemented truncated normal distribution with box-muller transform
- Added ``L_op()`` overriding option for ``OpFromGraph``
- Added NumPy C-API based fallback implementation for ``[sd]gemv_`` and ``[sd]dot_``
Other more detailed changes:
- Improved stack trace follow-up for GPU optimizations
- Fixed gradient error for elemwise ``minimum`` and ``maximum`` when compared values are the same
- Fixed gradient for ``ARange``
- Removed ``ViewOp`` subclass during optimization
Commiters since 0.9.0:
- Frederic Bastien
- João Victor Tozatti Risso
- Arnaud Bergeron
- Steven Bocco
- Mohammed Affan
- amrithasuresh
- Pascal Lamblin
- Reyhane Askari
- Alexander Matyasko
- Shawn Tan
- Simon Lefrancois
- Adam Becker
- Vikram
- Gijs van Tulder
- Faruk Ahmed
- Thomas George
- erakra
- Andrei Costinescu
- Boris Fomitchev
- Zhouhan LIN
- Aleksandar Botev
- jhelie
- xiaoqie
- Tegan Maharaj
- Matt Graham
- Cesar Laurent
- Gabe Schwartz
- Juan Camilo Gamboa Higuera
- Tim Cooijmans
- Anirudh Goyal
- Saizheng Zhang
- Yikang Shen
- vipulraheja
- Florian Bordes
- Sina Honari
- Chiheb Trabelsi
- Shubh Vachher
- Daren Eiri
- Joseph Paul Cohen
- Laurent Dinh
- Mohamed Ishmael Diwan Belghazi
- Jeff Donahue
- Ramana Subramanyam
- Bogdan Budescu
- Dzmitry Bahdanau
- Ghislain Antony Vaillant
- Jan Schlüter
- Nan Jiang
- Xavier Bouthillier
- fo40225
- mrTsjolder
- wyjw
- Aarni Koskela
- Adam Geitgey
- Adrian Keet
- Adrian Seyboldt
- Anmol Sahoo
- Chong Wu
- Holger Kohr
- Jayanth Koushik
- Lilian Besson
- Lv Tao
- Michael Manukyan
- Murugesh Marvel
- NALEPA
- Zotov Yuriy
- dareneiri
- lrast
- morrme
- naitonium
Theano 0.10.0beta3 (20th of September, 2017)
============================================
This release contains new features, improvements and bug fixes to prepare the upcoming release candidate.
We recommend that every developer updates to this version.
Highlights:
- conda packages now available and updated in our own conda channel ``mila-udem``.
To install it: ``conda install -c mila-udem -c mila-udem/label/pre theano pygpu``
- Improved elemwise operations
- Speed-up elemwise ops based on SciPy
- Fixed memory leak related to elemwise ops on GPU
- Fixed pygpu detection
- Bug fixes, crash fixes, warning improvements and documentation updates
A total of 69 people contributed to this release since 0.9.0, see list below.
Interface changes:
- Removed op ``ExtractDiag`` from ``theano.tensor.nlinalg``, now only in ``theano.tensor.basic``
Convolution updates:
- Added dilated causal convolutions for 2D
New features:
- Implemented ``topk`` and ``argtopk`` on CPU and GPU
- Added ``unravel_index`` and ``ravel_multi_index`` functions on CPU
- Implemented ``max()`` and ``min()`` functions for booleans and unsigned integers types
Others:
- Added ``R_op()`` for ``ZeroGrad``
- Added description for rnnblock
Commiters since 0.9.0:
- Frederic Bastien
- João Victor Tozatti Risso
- Arnaud Bergeron
- Steven Bocco
- Mohammed Affan
- amrithasuresh
- Pascal Lamblin
- Reyhane Askari
- Alexander Matyasko
- Simon Lefrancois
- Adam Becker
- Shawn Tan
- Vikram
- Gijs van Tulder
- Thomas George
- Andrei Costinescu
- Faruk Ahmed
- Boris Fomitchev
- Zhouhan LIN
- Aleksandar Botev
- jhelie
- xiaoqie
- Tegan Maharaj
- Matt Graham
- Cesar Laurent
- Gabe Schwartz
- Juan Camilo Gamboa Higuera
- Tim Cooijmans
- Anirudh Goyal
- Saizheng Zhang
- Yikang Shen
- vipulraheja
- Florian Bordes
- Sina Honari
- erakra
- Chiheb Trabelsi
- Shubh Vachher
- Daren Eiri
- Joseph Paul Cohen
- Laurent Dinh
- Mohamed Ishmael Diwan Belghazi
- Jeff Donahue
- Ramana Subramanyam
- Bogdan Budescu
- Dzmitry Bahdanau
- Ghislain Antony Vaillant
- Jan Schlüter
- Nan Jiang
- Xavier Bouthillier
- fo40225
- wyjw
- Aarni Koskela
- Adam Geitgey
- Adrian Keet
- Adrian Seyboldt
- Anmol Sahoo
- Chong Wu
- Holger Kohr
- Jayanth Koushik
- Lilian Besson
- Lv Tao
- Michael Manukyan
- Murugesh Marvel
- NALEPA
- Zotov Yuriy
- dareneiri
- lrast
- morrme
- naitonium
Theano 0.10.0beta2 (7th of September, 2017)
===========================================
This release contains new features, improvements and bug fixes to prepare the upcoming release candidate.
We recommend that every developer updates to this version.
Highlights:
- Support NumPy ``1.13``
- Support pygpu ``0.7``
- Added conda recipe
- Optional faster optimization step with new destroy handler
- Added documentation for RNNBlock
- Bug fixes, crash fixes, warning improvements and documentation updates
A total of 67 people contributed to this release since 0.9.0, see list below.
Interface changes:
- Added new parameter ``target`` for MRG functions
Convolution updates:
- Added unshared convolutions
- Added 3D separable convolutions
- Added 3D grouped convolutions
- Removed old ``conv3d`` interface
- Deprecated old ``conv2d`` interface
- Updated ``conv`` documentation
GPU:
- Added a meta-optimizer to select the fastest GPU implementations for convolutions
- cuDNN:
- Official support for ``v6.*`` and ``v7.*``, support for ``v5.*`` will be removed in next release
- Added spatial transformation operation based on cuDNN
- Updated and improved caching system for runtime-chosen cuDNN convolution algorithms
- Support cuDNN v7 tensor core operations for convolutions with runtime timed algorithms
- Restricted cuDNN reductions to contiguous inputs
- Automatic addition of cuDNN DLL path to ``PATH`` environment variable on Windows
New features:
- Added ``tensor6()`` and ``tensor7()`` in ``theano.tensor`` module
- Added boolean indexing for sub-tensors
- Added covariance matrix function ``theano.tensor.cov``
- Added new Theano flag ``pickle_test_value`` to help disable pickling test values
Others:
- Kept stack trace for optimizations in new GPU backend
Other more detailed changes:
- Moved all C code files into separate folder ``c_code`` in every Theano module
- Improvements for Jenkins tests
Commiters since 0.9.0:
- Frederic Bastien
- João Victor Tozatti Risso
- Arnaud Bergeron
- Steven Bocco
- Mohammed Affan
- amrithasuresh
- Pascal Lamblin
- Reyhane Askari
- Alexander Matyasko
- Simon Lefrancois
- Shawn Tan
- Gijs van Tulder
- Thomas George
- Vikram
- Andrei Costinescu
- Faruk Ahmed
- Boris Fomitchev
- Zhouhan LIN
- Aleksandar Botev
- jhelie
- xiaoqie
- Tegan Maharaj
- Matt Graham
- Cesar Laurent
- Gabe Schwartz
- Juan Camilo Gamboa Higuera
- Tim Cooijmans
- Anirudh Goyal
- Saizheng Zhang
- vipulraheja
- Florian Bordes
- Sina Honari
- Yikang Shen
- erakra
- Chiheb Trabelsi
- Shubh Vachher
- Daren Eiri
- Joseph Paul Cohen
- Laurent Dinh
- Mohamed Ishmael Diwan Belghazi
- Jeff Donahue
- Ramana Subramanyam
- Bogdan Budescu
- Dzmitry Bahdanau
- Ghislain Antony Vaillant
- Jan Schlüter
- Xavier Bouthillier
- fo40225
- Aarni Koskela
- Adam Becker
- Adam Geitgey
- Adrian Keet
- Adrian Seyboldt
- Anmol Sahoo
- Chong Wu
- Holger Kohr
- Jayanth Koushik
- Lilian Besson
- Lv Tao
- Michael Manukyan
- Murugesh Marvel
- NALEPA
- Zotov Yuriy
- dareneiri
- lrast
- morrme
- wyjw
Theano 0.10.0beta1 (9th of August, 2017)
========================================
This release contains a lot of bug fixes, improvements and new features to prepare the upcoming release candidate.
We recommend that every developer updates to this version.
Highlights:
- Moved Python 3.* minimum supported version from 3.3 to 3.4
- Replaced deprecated package ``nose-parameterized`` with up-to-date package ``parameterized`` for Theano requirements
- Theano now internally uses ``sha256`` instead of ``md5`` to work on systems that forbide ``md5`` for security reason
- Removed old GPU backend ``theano.sandbox.cuda``. New backend ``theano.gpuarray`` is now the official GPU backend
- Support more debuggers for ``PdbBreakpoint``
- Scan improvements
- Speed up Theano scan compilation and gradient computation
- Added meaningful message when missing inputs to scan
- Speed up graph toposort algorithm
- Faster C compilation by massively using a new interface for op params
- Faster optimization step
- Documentation updated and more complete
- Many bug fixes, crash fixes and warning improvements
A total of 65 people contributed to this release since 0.9.0, see list below.
Interface changes:
- Merged duplicated diagonal functions into two ops: ``ExtractDiag`` (extract a diagonal to a vector),
and ``AllocDiag`` (set a vector as a diagonal of an empty array)
- Renamed ``MultinomialWOReplacementFromUniform`` to ``ChoiceFromUniform``
- Removed or deprecated Theano flags:
- ``cublas.lib``
- ``cuda.enabled``
- ``enable_initial_driver_test``
- ``gpuarray.sync``
- ``home``
- ``lib.cnmem``
- ``nvcc.*`` flags
- ``pycuda.init``
- Changed ``grad()`` method to ``L_op()`` in ops that need the outputs to compute gradient
Convolution updates:
- Extended Theano flag ``dnn.enabled`` with new option ``no_check`` to help speed up cuDNN importation
- Implemented separable convolutions
- Implemented grouped convolutions
GPU:
- Prevent GPU initialization when not required
- Added disk caching option for kernels
- Added method ``my_theano_function.sync_shared()`` to help synchronize GPU Theano functions
- Added useful stats for GPU in profile mode
- Added Cholesky op based on ``cusolver`` backend
- Added GPU ops based on `magma library <http://icl.cs.utk.edu/magma/software/>`_:
SVD, matrix inverse, QR, cholesky and eigh
- Added ``GpuCublasTriangularSolve``
- Added atomic addition and exchange for ``long long`` values in ``GpuAdvancedIncSubtensor1_dev20``
- Support log gamma function for all non-complex types
- Support GPU SoftMax in both OpenCL and CUDA
- Support offset parameter ``k`` for ``GpuEye``
- ``CrossentropyCategorical1Hot`` and its gradient are now lifted to GPU
- Better cuDNN support
- Official support for ``v5.*`` and ``v6.*``
- Better support and loading on Windows and Mac
- Support cuDNN v6 dilated convolutions
- Support cuDNN v6 reductions
- Added new Theano flags ``cuda.include_path``, ``dnn.base_path`` and ``dnn.bin_path``
to help configure Theano when CUDA and cuDNN can not be found automatically.
- Updated ``float16`` support
- Added documentation for GPU float16 ops
- Support ``float16`` for ``GpuGemmBatch``
- Started to use ``float32`` precision for computations that don't support ``float16`` on GPU
New features:
- Added a wrapper for `Baidu's CTC <https://github.com/baidu-research/warp-ctc>`_ cost and gradient functions
- Added scalar and elemwise CPU ops for modified Bessel function of order 0 and 1 from ``scipy.special``.
- Added Scaled Exponential Linear Unit (SELU) activation
- Added sigmoid_binary_crossentropy function
- Added tri-gamma function
- Added modes ``half`` and ``full`` for ``Images2Neibs`` ops
- Implemented gradient for ``AbstractBatchNormTrainGrad``
- Implemented gradient for matrix pseudoinverse op
- Added new prop `replace` for ``ChoiceFromUniform`` op
- Added new prop ``on_error`` for CPU ``Cholesky`` op
- Added new Theano flag ``deterministic`` to help control how Theano optimize certain ops that have deterministic versions.
Currently used for subtensor Ops only.
- Added new Theano flag ``cycle_detection`` to speed-up optimization step by reducing time spending in inplace optimizations
- Added new Theano flag ``check_stack_trace`` to help check the stack trace during optimization process
- Added new Theano flag ``cmodule.debug`` to allow a debug mode for Theano C code. Currently used for cuDNN convolutions only.
Others:
- Added deprecation warning for the softmax and logsoftmax vector case
- Added a warning to announce that C++ compiler will become mandatory in next Theano release ``0.11``
Other more detailed changes:
- Removed useless warning when profile is manually disabled
- Added tests for abstract conv
- Added options for `disconnected_outputs` to Rop
- Removed ``theano/compat/six.py``
- Removed ``COp.get_op_params()``
- Support of list of strings for ``Op.c_support_code()``, to help not duplicate support codes
- Macro names provided for array properties are now standardized in both CPU and GPU C codes
- Started to move C code files into separate folder ``c_code`` in every Theano module
- Many improvements for Travis CI tests (with better splitting for faster testing)
- Many improvements for Jenkins CI tests: daily testings on Mac and Windows in addition to Linux
Commiters since 0.9.0:
- Frederic Bastien
- Arnaud Bergeron
- amrithasuresh
- João Victor Tozatti Risso
- Steven Bocco
- Pascal Lamblin
- Mohammed Affan
- Reyhane Askari
- Alexander Matyasko
- Simon Lefrancois
- Shawn Tan
- Thomas George
- Faruk Ahmed
- Zhouhan LIN
- Aleksandar Botev
- jhelie
- xiaoqie
- Tegan Maharaj
- Matt Graham
- Cesar Laurent
- Gabe Schwartz
- Juan Camilo Gamboa Higuera
- AndroidCloud
- Saizheng Zhang
- vipulraheja
- Florian Bordes
- Sina Honari
- Vikram
- erakra
- Chiheb Trabelsi
- Shubh Vachher
- Daren Eiri
- Gijs van Tulder
- Laurent Dinh
- Mohamed Ishmael Diwan Belghazi
- mila
- Jeff Donahue
- Ramana Subramanyam
- Bogdan Budescu
- Ghislain Antony Vaillant
- Jan Schlüter
- Xavier Bouthillier
- fo40225
- Aarni Koskela
- Adam Becker
- Adam Geitgey
- Adrian Keet
- Adrian Seyboldt
- Andrei Costinescu
- Anmol Sahoo
- Chong Wu
- Holger Kohr
- Jayanth Koushik
- Jenkins
- Lilian Besson
- Lv Tao
- Michael Manukyan
- Murugesh Marvel
- NALEPA
- Ubuntu
- Zotov Yuriy
- dareneiri
- lrast
- morrme
- yikang
Theano 0.9.0 (20th of March, 2017)
==================================
This is a final release of Theano, version ``0.9.0``, with a lot of
new features, interface changes, improvements and bug fixes.
We recommend that everybody update to this version.
Highlights (since 0.8.0):
- Better Python 3.5 support
- Better numpy 1.12 support
- Conda packages for Mac, Linux and Windows
- Support newer Mac and Windows versions
- More Windows integration:
- Theano scripts (``theano-cache`` and ``theano-nose``) now works on Windows
- Better support for Windows end-lines into C codes
- Support for space in paths on Windows
- Scan improvements:
- More scan optimizations, with faster compilation and gradient computation
- Support for checkpoint in scan (trade off between speed and memory usage, useful for long sequences)
- Fixed broadcast checking in scan
- Graphs improvements:
- More numerical stability by default for some graphs
- Better handling of corner cases for theano functions and graph optimizations
- More graph optimizations with faster compilation and execution
- smaller and more readable graph
- New GPU back-end:
- Removed warp-synchronous programming to get good results with newer CUDA drivers
- More pooling support on GPU when cuDNN isn't available
- Full support of ignore_border option for pooling
- Inplace storage for shared variables
- float16 storage
- Using PCI bus ID of graphic cards for a better mapping between theano device number and nvidia-smi number
- Fixed offset error in ``GpuIncSubtensor``
- Less C code compilation
- Added support for bool dtype
- Updated and more complete documentation
- Bug fixes related to merge optimizer and shape inference
- Lot of other bug fixes, crashes fixes and warning improvements
A total of 123 people contributed to this release since 0.8.0, see list below.
Interface changes:
- Merged ``CumsumOp/CumprodOp`` into ``CumOp``
- In MRG module:
- Replaced method ``multinomial_wo_replacement()`` with new method ``choice()``
- Random generator now tries to infer the broadcast pattern of its output
- New pooling interface
- Pooling parameters can change at run time
- Moved ``softsign`` out of sandbox to ``theano.tensor.nnet.softsign``
- Using floatX dtype when converting empty list/tuple
- ``Roll`` make the shift be modulo the size of the axis we roll on
- ``round()`` default to the same as NumPy: half_to_even
Convolution updates:
- Support of full and half modes for 2D and 3D convolutions including in ``conv3d2d``
- Allowed pooling of empty batch
- Implement ``conv2d_transpose`` convenience function
- Multi-cores convolution and pooling on CPU
- New abstract 3d convolution interface similar to the 2d convolution interface
- Dilated convolution
GPU:
- cuDNN: support versoin 5.1 and wrap batch normalization (2d and 3d) and RNN functions
- Multiple-GPU, synchrone update (via platoon, use NCCL)
- Gemv(matrix-vector product) speed up for special shape
- cublas gemv workaround when we reduce on an axis with a dimensions size of 0
- Warn user that some cuDNN algorithms may produce unexpected results in certain environments
for convolution backward filter operations
- ``GPUMultinomialFromUniform`` op now supports multiple dtypes
- Support for ``MaxAndArgMax`` for some axis combination
- Support for solve (using cusolver), erfinv and erfcinv
- Implemented ``GpuAdvancedSubtensor``
New features:
- ``OpFromGraph`` now allows gradient overriding for every input
- Added Abstract Ops for batch normalization that use cuDNN when available and pure Theano CPU/GPU alternatives otherwise
- Added gradient of solve, tensorinv (CPU), tensorsolve (CPU), searchsorted (CPU), DownsampleFactorMaxGradGrad (CPU)
- Added Multinomial Without Replacement
- Allowed partial evaluation of compiled function
- More Rop support
- Indexing support ellipsis: ``a[..., 3]```, ``a[1,...,3]``
- Added ``theano.tensor.{tensor5,dtensor5, ...}``
- compiledir_format support device
- Added New Theano flag ``conv.assert_shape`` to check user-provided shapes at runtime (for debugging)
- Added new Theano flag ``cmodule.age_thresh_use``
- Added new Theano flag ``cuda.enabled``
- Added new Theano flag ``nvcc.cudafe`` to enable faster compilation and import with old CUDA back-end
- Added new Theano flag ``print_global_stats`` to print some global statistics (time spent) at the end
- Added new Theano flag ``profiling.ignore_first_call``, useful to profile the new gpu back-end
- remove ProfileMode (use Theano flag ``profile=True`` instead)
Others:
- Split op now has C code for CPU and GPU
- ``theano-cache list`` now includes compilation times
- Speed up argmax only on GPU (without also needing the max)
- More stack trace in error messages
- Speed up cholesky grad
- ``log(sum(exp(...)))`` now get stability optimized
Other more detailed changes:
- Added Jenkins (gpu tests run on pull requests in addition to daily buildbot)
- Removed old benchmark directory and other old files not used anymore
- Use of 64-bit indexing in sparse ops to allow matrix with more then 2\ :sup:`31`\ -1 elements
- Allowed more then one output to be an destructive inplace
- More support of negative axis
- Added the keepdims parameter to the norm function
- Make scan gradient more deterministic
Commiters since 0.8.0:
- Frederic Bastien
- Arnaud Bergeron
- Pascal Lamblin
- Steven Bocco
- Ramana Subramanyam
- Simon Lefrancois
- Gijs van Tulder
- Benjamin Scellier
- khaotik
- Chiheb Trabelsi
- Chinnadhurai Sankar
- Cesar Laurent
- Reyhane Askari