-
Notifications
You must be signed in to change notification settings - Fork 205
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Major bugfix + new efficiency-focused functionality. #84
Conversation
…bit structure that could get erroneously good convergence orders
Thanks for this update! Could you also split bullets 2-4 into 2 separate PRs? One for 2-3, and another for 4. That'll help a lot in terms of going through the code. It might be too pesky to split this into 4 PRs, but I think it's reasonable to factor out the f_and_g functionality, as I think it would affect a few things. |
* fixes for diagnostics * Added every applicable method to every diagnostic. Added ito_general. Added run_all. * fixed levy area appproximation choices * added no_grad wrappers * switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders * bugfix for Ex2 * increased Ex complexity slightly * introduced the neural SDEs as separate without replacing Ex1-3 * recursion fix * Renamed Ex1->ExDiagonal etc. * Bugfix for calling integrate twice. (#86) * Bugfix for calling integrate twice. * Increment version. * Fix scipy version imcompatibility with Py3.6. * Fix numpy incompatibility with Py3.6. Co-authored-by: Xuechen Li <[email protected]> * fixes for diagnostics * Added every applicable method to every diagnostic. Added ito_general. Added run_all. * fixed levy area appproximation choices * added no_grad wrappers * switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders * bugfix for Ex2 * increased Ex complexity slightly * introduced the neural SDEs as separate without replacing Ex1-3 * recursion fix * Renamed Ex1->ExDiagonal etc. * Simplify. * Fix docstring. Co-authored-by: Xuechen Li <[email protected]>
…into dev-call-optim
For efficiency purposes, I'm committing minor tweaks so we can merge. For the record, I'm rebasing from |
* Major bugfix. New efficiency improvements. * bugfix * bugfix for diagonal noise. Improved Milstein efficiency. * Documentation update * fixes for diagnostics * Added every applicable method to every diagnostic. Added ito_general. Added run_all. * fixed levy area appproximation choices * added no_grad wrappers * switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders * bugfix for Ex2 * increased Ex complexity slightly * introduced the neural SDEs as separate without replacing Ex1-3 * bugfix for Milstein * recursion fix * Added tests for specialised functions and fixed bugs as a result * Renamed Ex1->ExDiagonal etc. * Bugfix for calling integrate twice. * Dev diagnostics fixes (#85) * fixes for diagnostics * Added every applicable method to every diagnostic. Added ito_general. Added run_all. * fixed levy area appproximation choices * added no_grad wrappers * switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders * bugfix for Ex2 * increased Ex complexity slightly * introduced the neural SDEs as separate without replacing Ex1-3 * recursion fix * Renamed Ex1->ExDiagonal etc. * Bugfix for calling integrate twice. (#86) * Bugfix for calling integrate twice. * Increment version. * Fix scipy version imcompatibility with Py3.6. * Fix numpy incompatibility with Py3.6. Co-authored-by: Xuechen Li <[email protected]> * fixes for diagnostics * Added every applicable method to every diagnostic. Added ito_general. Added run_all. * fixed levy area appproximation choices * added no_grad wrappers * switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders * bugfix for Ex2 * increased Ex complexity slightly * introduced the neural SDEs as separate without replacing Ex1-3 * recursion fix * Renamed Ex1->ExDiagonal etc. * Simplify. * Fix docstring. Co-authored-by: Xuechen Li <[email protected]> * Major bugfix. New efficiency improvements. * bugfix * bugfix for diagonal noise. Improved Milstein efficiency. * Documentation update * bugfix for Milstein * Added tests for specialised functions and fixed bugs as a result * Patch license. Co-authored-by: Xuechen Li <[email protected]>
* Improve usability. (#81) * Improve usability. * Minor fix. * Fixes from comments. * Fixes. * Dev diagnostics fixes (#85) * fixes for diagnostics * Added every applicable method to every diagnostic. Added ito_general. Added run_all. * fixed levy area appproximation choices * added no_grad wrappers * switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders * bugfix for Ex2 * increased Ex complexity slightly * introduced the neural SDEs as separate without replacing Ex1-3 * recursion fix * Renamed Ex1->ExDiagonal etc. * Bugfix for calling integrate twice. (#86) * Bugfix for calling integrate twice. * Increment version. * Fix scipy version imcompatibility with Py3.6. * Fix numpy incompatibility with Py3.6. Co-authored-by: Xuechen Li <[email protected]> * fixes for diagnostics * Added every applicable method to every diagnostic. Added ito_general. Added run_all. * fixed levy area appproximation choices * added no_grad wrappers * switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders * bugfix for Ex2 * increased Ex complexity slightly * introduced the neural SDEs as separate without replacing Ex1-3 * recursion fix * Renamed Ex1->ExDiagonal etc. * Simplify. * Fix docstring. Co-authored-by: Xuechen Li <[email protected]> * Major bugfix + new efficiency-focused functionality. (#84) * Major bugfix. New efficiency improvements. * bugfix * bugfix for diagonal noise. Improved Milstein efficiency. * Documentation update * fixes for diagnostics * Added every applicable method to every diagnostic. Added ito_general. Added run_all. * fixed levy area appproximation choices * added no_grad wrappers * switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders * bugfix for Ex2 * increased Ex complexity slightly * introduced the neural SDEs as separate without replacing Ex1-3 * bugfix for Milstein * recursion fix * Added tests for specialised functions and fixed bugs as a result * Renamed Ex1->ExDiagonal etc. * Bugfix for calling integrate twice. * Dev diagnostics fixes (#85) * fixes for diagnostics * Added every applicable method to every diagnostic. Added ito_general. Added run_all. * fixed levy area appproximation choices * added no_grad wrappers * switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders * bugfix for Ex2 * increased Ex complexity slightly * introduced the neural SDEs as separate without replacing Ex1-3 * recursion fix * Renamed Ex1->ExDiagonal etc. * Bugfix for calling integrate twice. (#86) * Bugfix for calling integrate twice. * Increment version. * Fix scipy version imcompatibility with Py3.6. * Fix numpy incompatibility with Py3.6. Co-authored-by: Xuechen Li <[email protected]> * fixes for diagnostics * Added every applicable method to every diagnostic. Added ito_general. Added run_all. * fixed levy area appproximation choices * added no_grad wrappers * switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders * bugfix for Ex2 * increased Ex complexity slightly * introduced the neural SDEs as separate without replacing Ex1-3 * recursion fix * Renamed Ex1->ExDiagonal etc. * Simplify. * Fix docstring. Co-authored-by: Xuechen Li <[email protected]> * Major bugfix. New efficiency improvements. * bugfix * bugfix for diagonal noise. Improved Milstein efficiency. * Documentation update * bugfix for Milstein * Added tests for specialised functions and fixed bugs as a result * Patch license. Co-authored-by: Xuechen Li <[email protected]> * Minor formatting. * Simplify detach logic in core adjoint functions. * Improve usability. (#81) * Improve usability. * Minor fix. * Fixes from comments. * Fixes. * Dev diagnostics fixes (#85) * fixes for diagnostics * Added every applicable method to every diagnostic. Added ito_general. Added run_all. * fixed levy area appproximation choices * added no_grad wrappers * switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders * bugfix for Ex2 * increased Ex complexity slightly * introduced the neural SDEs as separate without replacing Ex1-3 * recursion fix * Renamed Ex1->ExDiagonal etc. * Bugfix for calling integrate twice. (#86) * Bugfix for calling integrate twice. * Increment version. * Fix scipy version imcompatibility with Py3.6. * Fix numpy incompatibility with Py3.6. Co-authored-by: Xuechen Li <[email protected]> * fixes for diagnostics * Added every applicable method to every diagnostic. Added ito_general. Added run_all. * fixed levy area appproximation choices * added no_grad wrappers * switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders * bugfix for Ex2 * increased Ex complexity slightly * introduced the neural SDEs as separate without replacing Ex1-3 * recursion fix * Renamed Ex1->ExDiagonal etc. * Simplify. * Fix docstring. Co-authored-by: Xuechen Li <[email protected]> * Major bugfix + new efficiency-focused functionality. (#84) * Major bugfix. New efficiency improvements. * bugfix * bugfix for diagonal noise. Improved Milstein efficiency. * Documentation update * fixes for diagnostics * Added every applicable method to every diagnostic. Added ito_general. Added run_all. * fixed levy area appproximation choices * added no_grad wrappers * switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders * bugfix for Ex2 * increased Ex complexity slightly * introduced the neural SDEs as separate without replacing Ex1-3 * bugfix for Milstein * recursion fix * Added tests for specialised functions and fixed bugs as a result * Renamed Ex1->ExDiagonal etc. * Bugfix for calling integrate twice. * Dev diagnostics fixes (#85) * fixes for diagnostics * Added every applicable method to every diagnostic. Added ito_general. Added run_all. * fixed levy area appproximation choices * added no_grad wrappers * switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders * bugfix for Ex2 * increased Ex complexity slightly * introduced the neural SDEs as separate without replacing Ex1-3 * recursion fix * Renamed Ex1->ExDiagonal etc. * Bugfix for calling integrate twice. (#86) * Bugfix for calling integrate twice. * Increment version. * Fix scipy version imcompatibility with Py3.6. * Fix numpy incompatibility with Py3.6. Co-authored-by: Xuechen Li <[email protected]> * fixes for diagnostics * Added every applicable method to every diagnostic. Added ito_general. Added run_all. * fixed levy area appproximation choices * added no_grad wrappers * switched all diagnostics to neural network problems, which don't exhibit structure that could get erroneously good convergence orders * bugfix for Ex2 * increased Ex complexity slightly * introduced the neural SDEs as separate without replacing Ex1-3 * recursion fix * Renamed Ex1->ExDiagonal etc. * Simplify. * Fix docstring. Co-authored-by: Xuechen Li <[email protected]> * Major bugfix. New efficiency improvements. * bugfix * bugfix for diagonal noise. Improved Milstein efficiency. * Documentation update * bugfix for Milstein * Added tests for specialised functions and fixed bugs as a result * Patch license. Co-authored-by: Xuechen Li <[email protected]> * Minor formatting. * Simplify detach logic in core adjoint functions. * Increment version count. Co-authored-by: Patrick Kidger <[email protected]>
Heyhey. I've been tweaking some of the SDE-GAN stuff (for the paper, not the example), and realised that it could be made much more efficient with library support... and what's the point of being a developer of the library if not for that?
solver.integrate
twice insdeint.py::integrate
(!) (I can make this a separate PR if you want to put this in immediately.)f_and_g
,g_prod
, andf_and_g_prod
as functions that the SDE can define, which will be used in the solvers if possible.f
andg
no longer actually have to be defined if the solver doesn't need them. [1]f_and_g
function that is more efficient than calling them separately. (Unrelated to what I'm doing, but why not.) [2]g
computations.(N.B. This PR depends upon #85.)
[1] My use case for this: First of all, I'd like drift and diffusion to just be different final layers at the end of the same MLP, hence
f_and_g
. Second, for the discriminator, the diffusion ends up being a product of matrices. Doing (matrix-matrix)-vector would be inefficient, henceg_prod
/f_and_g_prod
.[2] There's still some improvements that could be made here: for simplicity logqp still requires
f
/g
/h
to all be specified, without exploitingf_and_g
orf_and_g_prod
if they're available. The logical extension of this would be to also addf_and_g_and_h
,f_and_g_prod_and_h
as possible methods.