# DifferentialEquations.jl: Scientific Machine Learning (SciML) Enabled Simulation and Estimation

This is a suite for numerically solving differential equations written in Julia and available for use in Julia, Python, and R. The purpose of this package is to supply efficient Julia implementations of solvers for various differential equations. Equations within the realm of this package include:

- Discrete equations (function maps, discrete stochastic (Gillespie/Markov) simulations)
- Ordinary differential equations (ODEs)
- Split and Partitioned ODEs (Symplectic integrators, IMEX Methods)
- Stochastic ordinary differential equations (SODEs or SDEs)
- Stochastic differential-algebraic equations (SDAEs)
- Random differential equations (RODEs or RDEs)
- Differential algebraic equations (DAEs)
- Delay differential equations (DDEs)
- Neutral, retarded, and algebraic delay differential equations (NDDEs, RDDEs, and DDAEs)
- Stochastic delay differential equations (SDDEs)
- Experimental support for stochastic neutral, retarded, and algebraic delay differential equations (SNDDEs, SRDDEs, and SDDAEs)
- Mixed discrete and continuous equations (Hybrid Equations, Jump Diffusions)
- (Stochastic) partial differential equations ((S)PDEs) (with both finite difference and finite element methods)

The well-optimized DifferentialEquations solvers benchmark as the some of the fastest implementations, using classic algorithms and ones from recent research which routinely outperform the "standard" C/Fortran methods, and include algorithms optimized for high-precision and HPC applications. At the same time, it wraps the classic C/Fortran methods, making it easy to switch over to them whenever necessary. Solving differential equations with different methods from different languages and packages can be done by changing one line of code, allowing for easy benchmarking to ensure you are using the fastest method possible.

DifferentialEquations.jl integrates with the Julia package sphere with:

- GPU accleration through CUDA.jl and DiffEqGPU.jl
- Automated sparsity detection with SparsityDetection.jl
- Automatic Jacobian coloring with SparseDiffTools.jl, allowing for fast solutions to problems with sparse or structured (Tridiagonal, Banded, BlockBanded, etc.) Jacobians
- Allowing the specification of linear solvers for maximal efficiency
- Progress meter integration with the Juno IDE for estimated time to solution
- Automatic plotting of time series and phase plots
- Built-in interpolations
- Wraps for common C/Fortran methods like Sundials and Hairer's radau
- Arbitrary precision with BigFloats and Arbfloats
- Arbitrary array types, allowing the definition of differential equations on matrices and distributed arrays
- Unit checked arithmetic with Unitful

Additionally, DifferentialEquations.jl comes with built-in analysis features, including:

- Forward and Adjoint Sensitivity Analysis (Automatic Differentiation) for fast gradient computations
- Parameter Estimation and Bayesian Analysis
- Neural differential equations with DiffEqFlux.jl for efficient scientific machine learning (scientific ML) and scientific AI.
- Automatic distributed, multithreaded, and GPU Parallel Ensemble Simulations
- Global Sensitivity Analysis
- Uncertainty Quantification

If you have any questions, or just want to chat about solvers/using the package, please feel free to use the Gitter channel. For bug reports, feature requests, etc., please submit an issue. If you're interested in contributing, please see the Developer Documentation.

## Supporting and Citing

The software in this ecosystem was developed as part of academic research. If you would like to help support it, please star the repository as such metrics may help us secure funding in the future. If you use SciML software as part of your research, teaching, or other activities, we would be grateful if you could cite our work.

```
@article{rackauckas2017differentialequations,
title={Differentialequations.jl--a performant and feature-rich ecosystem for solving differential equations in julia},
author={Rackauckas, Christopher and Nie, Qing},
journal={Journal of Open Research Software},
volume={5},
number={1},
year={2017},
publisher={Ubiquity Press}
}
```

is necessary for any use of DifferentialEquations.jl or the packages that are maintained as part of its suite (OrdinaryDiffEq.jl, Sundials.jl, DiffEqDevTools.jl, etc.). Additionally, many of the solvers utilize novel algorithms, and if these algorithms are used we asked that you cite the methods. Please see our citation page for guidelines.

## Getting Started: Installation And First Steps

### Installing from Julia

To install the package, use the following command inside the Julia REPL:

```
using Pkg
Pkg.add("DifferentialEquations")
```

To load the package, use the command:

`using DifferentialEquations`

This will add solvers and dependencies for all kinds of Differential Equations (e.g. ODEs or SDEs etc., see the Supported Equations section below). If you are interested in only one type of equation solvers of `DifferentialEquations.jl`

or simply want a more lightweight version, see the Low Dependency Usage page.

To understand the package in more detail, check out the following tutorials in this manual. **It is highly recommended that new users start with the ODE tutorial**. Example IJulia notebooks can also be found in DiffEqTutorials.jl. If you find any example where there seems to be an error, please open an issue.

For the most up to date information on using the package, please join the Gitter channel.

Using the bleeding edge for the latest features and development is only recommended for power users. Information on how to get to the bleeding edge is found in the developer documentation.

### Installing from Python

Use of DifferentialEquations.jl from the Python programming language is available through the diffeqpy module. To install diffeqpy, use pip:

`pip install diffeqpy`

Using diffeqpy requires that Julia is installed and in the path, along with DifferentialEquations.jl and PyCall.jl. To install Julia, download a generic binary from the JuliaLang site and add it to your path. To install Julia packages required for diffeqpy, open up Python interpreter then run:

```
>>> import diffeqpy
>>> diffeqpy.install()
```

and you're good! In addition, to improve the performance of your code it is recommended that you use Numba to JIT compile your derivative functions. To install Numba, use:

`pip install numba`

diffeqpy supports the majority of DifferentialEquations.jl with very similar syntax, see the diffeqpy README for more details. One important point to note is that Numba is generally an order of magnitude slower than Julia in terms of the generated differential equation solver code, and thus it is recommended to use `julia.Main.eval`

for Julia-side derivative function implementations for maximal efficiency. See this blog post for more information.

### Installing from R

Use of DifferentialEquations.jl from the R programming language is available through the diffeqr module. diffeqr is registered into CRAN. Thus to add the package, use:

`install.packages("diffeqr")`

To install the master branch of the package (for developers), use:

`devtools::install_github('SciML/diffeqr', build_vignettes=T)`

You will need a working installation of Julia in your path. To install Julia, download a generic binary from the JuliaLang site and add it to your path. The download and installation of DifferentialEquations.jl will happen on the first invocation of `diffeqr::diffeq_setup()`

.

Currently, use from R supported a subset of DifferentialEquations.jl which is documented through CRAN.

### IJulia Notebook Tutorials

You can access extra tutorials supplied in the DiffEqTutorials.jl repository via the commands:

```
using Pkg
pkg"add https://github.com/SciML/DiffEqTutorials.jl"
using DiffEqTutorials
DiffEqTutorials.open_notebooks()
```

Or you can view the webpages for the rendered tutorials at the links found in the repository.

### Video Tutorial

### Tutorials

The following tutorials will introduce you to the functionality of DifferentialEquations.jl. More examples can be found by checking out the IJulia notebooks in the examples folder.

- Ordinary Differential Equations
- Example 1 : Solving Scalar Equations
- Example 2: Solving Systems of Equations
- Defining Parameterized Functions
- Example 3: Solving Nonhomogeneous Equations using Parameterized Functions
- Example 4: Using Other Types for Systems of Equations
- Going Beyond ODEs: How to Use the Documentation
- Stochastic Differential Equations
- Example 1: Scalar SDEs
- Example 2: Systems of SDEs with Diagonal Noise
- Example 3: Systems of SDEs with Scalar Noise
- Example 4: Systems of SDEs with Non-Diagonal Noise
- Example 4: Colored Noise
- Delay Differential Equations
- Differential Algebraic Equations
- Discrete Stochastic (Gillespie) Equations
- Defining a Model using Reactions
- Building and Solving the Problem
- SSAStepper
- Controlling Saving Behavior
- Defining the Jumps Directly:
`ConstantRateJump`

- Defining the Jumps Directly:
`MassActionJump`

- Defining the Jumps Directly: Mixing
`ConstantRateJump`

and`MassActionJump`

- Adding Jumps to a Differential Equation
- Adding a VariableRateJump
- RegularJumps and Tau-Leaping
- Jump Diffusion Equations
- Boundary Value Problems
- Additional Tutorials

### Removing and Reducing Compile Times

In some situations one may wish to decrease the compile time associated with DifferenitalEquations.jl usage. If that's the case, there's two strategies to employ. One strategy is to use the low dependency usage. DifferentialEquations.jl is a metapackage composed of many smaller packages, and thus one could directly use a single component, such as `OrdinaryDiffEq.jl`

for the pure Julia ODE solvers, and decrease the compile times by ignoring the rest (note: the interface is exactly the same, except using a solver other than those in OrdinaryDiffEq.jl will error). We recommend that downstream packages only rely on exactly the packages they need.

The other strategy is to use PackageCompiler.jl to create a system image that precompiles the whole package. To do this, one simply does:

```
using PackageCompiler
PackageCompiler.create_sysimage([:DifferentialEquations,:Plots];replace_default=true)
```

Note that there are some drawbacks to adding a package in your system image, for example the package will never update until you manually rebuild the system image again. For more information on the consequences, see this portion of the PackageCompiler manual

### Basics

These pages introduce you to the core of DifferentialEquations.jl and the common interface. It explains the general workflow, options which are generally available, and the general tools for analysis.

- Overview of DifferentialEquations.jl
- Defining Problems
- Solving the Problems
- Analyzing the Solution
- Add-on Tools
- Development and Testing Tools
- Common Solver Options
- Default Algorithm Hinting
- Output Control
- Stepsize Control
- Memory Optimizations
- Miscellaneous
- Progress Monitoring
- Error Calculations
- Examples
- Solution Handling
- Accessing the Values
- Array Interface
- Using the AbstractArray Interface
- Interpolations and Calculating Derivatives
- Comprehensions
- Special Fields
- Return Codes (RetCodes)
- Problem-Specific Features
- Plot Functions
- Standard Plots Using the Plot Recipe
- Density
- Choosing Variables
- Animations
- Plotting Without the Plot Recipe
- Integrator Interface
- Problem Interface
- In-place vs Out-of-Place Function Definition Forms
- Type Specifications
- Functional and Condensed Problem Inputs
- Lower Level
`__init`

and`__solve`

- Modification of problem types
- Frequently Asked Questions
- Solver Compatibility Chart

### Problem Types

These pages describe building the problem types to define differential equations for the solvers, and the special features of the different solution types.

- Discrete Problems
- ODE Problems
- Dynamical, Hamiltonian and 2nd Order ODE Problems
- Mathematical Specification of a Dynamical ODE Problem
- Mathematical Specification of a 2nd Order ODE Problem
- Hamiltonian Problems
- Split ODE Problems
- Steady State Problems
- BVP Problems
- SDE Problems
- RODE Problems
- DDE Problems
- Mathematical Specification of a DDE Problem
- Functional Forms of the History Function
- Declaring Lags
- Neutral and Retarded Delay Differential Equations
- Problem Type
- Example Problems
- DAE Problems
- Jump Problems

### Solver Algorithms

These pages describe the solvers and available algorithms in detail.

- Discrete Solvers
- ODE Solvers
- Dynamical, Hamiltonian, and 2nd Order ODE Solvers
- Split ODE Solvers
- Steady State Solvers
- BVP Solvers
- Jump Problem and Jump Diffusion Solvers
- Recommended Methods
- Special Methods for Pure Jump Problems
- RegularJump Compatible Methods
- Regular Jump Diffusion Compatible Methods
- SDE Solvers
- Recommended Methods
- Special Noise Forms
- Iterated Integral Approximations
- Special Keyword Arguments
- Full List of Methods
- RODE Solvers
- DDE Solvers
- DAE Solvers
- Solver Benchmarks

### Additional Features

These sections discuss extra performance enhancements, event handling, and other in-depth features.

- DiffEqFunctions (Jacobians, Gradients, etc.) and Jacobian Types
- Function Type Definitions
- Inplace Specification and No-Recompile Mode
- Specifying Jacobian Types
- Examples
- Symbolically Calculating the Functions
- DiffEq-Specific Array Types
- DiffEqOperators
- Noise Processes
- Using Noise Processes
- Noise Process Types
- Non-Standard Noise Processes
- Examples Using Non-Standard Noise Processes
- Specifying (Non)Linear Solvers
- Event Handling and Callback Functions
- The Callback Types
- Using Callbacks
- DiscreteCallback Examples
- ContinuousCallback Examples
- VectorContinuousCallback Example
- Callback Library
- Manifold Conservation and Projection
- AutoAbstol
- PositiveDomain
- GeneralDomain
- Stepsize Limiters
- FunctionCallingCallback
- SavingCallback
- PresetTimeCallback
- IterativeCallback
- PeriodicCallback
- TerminateSteadyState
- Parallel Ensemble Simulations
- Performing an Ensemble Simulation
- Analyzing an Ensemble Experiment
- Example 1: Solving an ODE With Different Initial Conditions
- Example 2: Solving an SDE with Different Parameters
- Example 3: Using the Reduction to Halt When Estimator is Within Tolerance
- Example 4: Using the Analysis Tools
- I/O: Saving and Loading Solution Data
- Low Dependency Usage
- Progress Bar Integration

### Analysis Tools

Because DifferentialEquations.jl has a common interface on the solutions, it is easy to add functionality to the entire DiffEq ecosystem by developing it to the solution interface. These pages describe the add-on analysis tools which are available.

- ParameterizedFunctions
- Parameter Estimation and Bayesian Analysis
- Installation
- Recommended Methods
- Optimization-Based Methods
- Bayesian Methods
- Optimization-Based ODE Inference Examples
- Parameter Estimation for Stochastic Differential Equations and Ensembles
- Bayesian Inference Examples
- Bifurcation Analysis
- Local Sensitivity Analysis (Automatic Differentiation)
- Lower Level Sensitivity Analysis Interfaces
- Local Forward Sensitivity Analysis via ODEForwardSensitivityProblem
- Adjoint Sensitivity Analysis via adjoint_sensitivities (Backpropogation)
- Second Order Sensitivity Analysis via second
*order*sensitivities (Experimental) - Global Sensitivity Analysis
- Uncertainty Quantification
- Installation
- ProbInts
- Example 1: FitzHugh-Nagumo
- Example 2: Adaptive ProbInts on FitzHugh-Nagumo
- Example 3: Adaptive ProbInts on the Lorenz Attractor
- Neural Networks
- Algorithm Development and Testing

### Modeling Tools

While DifferentialEquations.jl can be used to directly build any differential or difference equation (/ discrete stochastic) model, in many cases it can be helpful to have a tailored-built API for making certain types of common models easier. This is provided by the modeling functionality.

### Extra Details

These are just assorted extra explanations for the curious.

- Timestepping Method Descriptions
- Common Setup
- Proportional Control (Standard Control)
- Proportional-Integral Control (PI-Control)
- Gustafsson Acceleration
- Mathematics of Sensitivity Analysis

## Acknowledgements

#### Core Contributors

JuliaDiffEq and DifferentialEquations.jl has been a collaborative effort by many individuals. Significant contributions have been made by the following individuals:

- Chris Rackauckas (@ChrisRackauckas) (lead developer)
- Yingbo Ma (@YingboMa)
- David Widmann (@devmotion)
- Hendrik Ranocha (@ranocha)
- Ethan Levien (@elevien)
- Tom Short (@tshort)
- @dextorious
- Samuel Isaacson (@isaacsas)

#### Google Summer of Code Alumni

- Yingbo Ma (@YingboMa)
- Shivin Srivastava (@shivin9)
- Ayush Pandey (@Ayush-iitkgp)
- Xingjian Guo (@MSeeker1340)
- Shubham Maddhashiya (@sipah00)
- Vaibhav Kumar Dixit (@Vaibhavdixit02)