Description

The gradient function is used to compute the derivative of a multivariate function that maps $\mathbb{R}^n$ to $\mathbb{R}$. For example, x -> sin(x[1]) + cos(x[2]) is a multivariate function that might be passed to the gradient function.

The gradient function comes in three core variants:

  • A pure function that directly computes the gradient of a function f at a fixed value of x and returns a newly allocated array as a result.
  • A mutating function that computes the gradient of a function f at a fixed value of x and stores the result into a user-provided output array. This function returns nothing since its action is focused on mutation. To ensure thread safety, this mutating variant also requires that a buffer be provided that is part of the internal implementation of the function, but can be provided explicitly to minimize unnecessary memory allocations.
  • A higher-order function that constructs a new function f_prime that will compute the gradient of f at any point x that is provided as an argument to f_prime. This newly constructed function is the return value for this variant of the gradient function.

In addition, the gradient function allows one to chose the mode of numerical differentiation that is used to compute an approximate derivative. The default is use central finite differences, which evaluates f at two nearby points for each dimension of the input array x and provides much higher accuracy than other strategies at a cost of roughly double the amount of computation. Other modes of numerical differentiation are available, but must be opted-in to explicitly.

Primary Methods

The primary methods that most users will want to use are the following:

  • Use gradient(f::Function, x::AbstractArray) to approximate the gradient of f at x. This is the pure variant described above.
  • Use gradient!(output::AbstractArray, f::Function, x::AbstractArray, buffer::AbstractArray) to approximate the gradient of f at x and store the result into the output array.
  • Use gradient(f::Function) to generate a new function that approximates the true gradient function of f. The new function f_prime can be evaluated at any point x that is desired after it is constructed.

Detailed Method-Level Documentation

# FiniteDiff.gradient! — Method.

gradient!{T <: AbstractFloat}(
    output::AbstractArray,
    f::Function,
    x::AbstractArray{T},
    buffer::AbstractArray{T},
)

Description

Evaluate the gradient of f at x using finite differences. Defaults to central mode finite differences. See the documentation for that mode for more details.

Arguments

  • output::AbstractArray: An array that will be mutated to contain the gradient of f at x.
  • f::Function: The function to be differentiated.
  • x::AbstractArray{T}: The value of x at which to evaluate the gradient of f.
  • buffer::AbstractArray{T}: A buffer that is equivalent to similar(x). Used as a temporary copy of x that can be mutated safely during the execution of the function.

Returns

  • nothing::Void: This function is called for its side effects.

Examples

import FiniteDiff: gradient!
x = [0.0, 0.0]
output, buffer = Array(Float64, 2), Array(Float64, 2)
gradient!(
    output,
    x -> sin(x[1]) + 2 * sin(x[2]),
    x,
    buffer,
)

source

# FiniteDiff.gradient — Function.

gradient{T <: AbstractFloat}(
    f::Function,
    x::AbstractArray{T},
    mode::Mode = CentralMode(),
)

Description

Evaluate the gradient of f at x using finite differences. Defaults to central mode finite differences. See the documentation for that mode for more details.

NOTE: Allocates new memory to store the output as well as memory to use as a buffer.

Arguments

  • f::Function: The function to be differentiated.
  • x::AbstractArray{T}: The value of x at which to evaluate the gradient of f.
  • m::Mode: An instance of the Mode type. This will determine the mode of finite differences that will be used.

Returns

  • output::AbstractArray: The gradient.

Examples

import FiniteDiff: gradient!, ForwardMode
x = [0.0, 0.0]
output = gradient!(x -> sin(x[1]) + 2 * sin(x[2]), x, ForwardMode())

source

# FiniteDiff.gradient — Function.

gradient(f::Function, mode::Mode = CentralMode(); mutates::Bool = false)

Description

Construct a new function that will evaluate the gradient of f at any value of x. The user can specify the mode as a positional argument and can also indicate whether to return a mutating or non-mutating function using a keyword argument.

Arguments

  • f::Function: The function to be differentiated.
  • m::Mode: An instance of the Mode type. This will determine the mode of finite differences that will be used.

Keyword Arguments

  • mutates::Bool = false: Determine whether the resulting function will mutate its inputs or will be a pure function.

Returns

  • g′::Function: A function to evaluate the gradient of f at a new point x.

Examples

import FiniteDiff: gradient!, ForwardMode
x = [0.0, 0.0]
g′ = gradient(x -> sin(x[1]) + 2 * sin(x[2]), ForwardMode(), mutates = false)
output = g′(x)

source

# FiniteDiff.gradient! — Method.

gradient!{T <: AbstractFloat}(
    output::AbstractArray,
    f::Function,
    x::AbstractArray{T},
    buffer::AbstractArray{T},
    ::ForwardMode,
)

Description

Evaluate the gradient of f at x using forward-mode finite differences. Store the results into output. Work with a user-provided buffer to ensure that we can work without copies, but also without mutating x. In Rust jargon, we take ownership of both output and buffer, but do not require ownership of x. We just need read-only access to x.

Arguments

  • output::AbstractArray: An array that will be mutated to contain the gradient of f at x.
  • f::Function: The function to be differentiated.
  • x::AbstractArray{T}: The value of x at which to evaluate the gradient of f.
  • buffer::AbstractArray{T}: A buffer that is equivalent to similar(x). Used as a temporary copy of x that can be mutated safely during the execution of the function.
  • ::ForwardMode: An instance of the ForwardMode type.

Returns

  • nothing::Void: This function is called for its side effects.

Examples

import FiniteDiff: gradient!, ForwardMode
x = [0.0, 0.0]
output, buffer = Array(Float64, 2), Array(Float64, 2)
gradient!(
    output,
    x -> sin(x[1]) + 2 * sin(x[2]),
    x,
    buffer,
    ForwardMode(),
)

source

# FiniteDiff.gradient! — Method.

gradient!{T <: AbstractFloat}(
    output::AbstractArray,
    f::Function,
    x::AbstractArray{T},
    buffer::AbstractArray{T},
    ::BackwardMode,
)

Description

Evaluate the gradient of f at x using backward-mode finite differences. Store the results into output. Work with a user-provided buffer to ensure that we can work without copies, but also without mutating x. In Rust jargon, we take ownership of both output and buffer, but do not require ownership of x. We just need read-only access to x.

Arguments

  • output::AbstractArray: An array that will be mutated to contain the gradient of f at x.
  • f::Function: The function to be differentiated.
  • x::AbstractArray{T}: The value of x at which to evaluate the gradient of f.
  • buffer::AbstractArray{T}: A buffer that is equivalent to similar(x). Used as a temporary copy of x that can be mutated safely during the execution of the function.
  • ::BackwardMode: An instance of the BackwardMode type.

Returns

  • nothing::Void: This function is called for its side effects.

Examples

import FiniteDiff: gradient!, BackwardMode
x = [0.0, 0.0]
output, buffer = Array(Float64, 2), Array(Float64, 2)
gradient!(
    output,
    x -> sin(x[1]) + 2 * sin(x[2]),
    x,
    buffer,
    BackwardMode(),
)

source

# FiniteDiff.gradient! — Method.

gradient!{T <: AbstractFloat}(
    output::AbstractArray,
    f::Function,
    x::AbstractArray{T},
    buffer::AbstractArray{T},
    ::CentralMode,
)

Description

Evaluate the gradient of f at x using central-mode finite differences. Store the results into output. Work with a user-provided buffer to ensure that we can work without copies, but also without mutating x. In Rust jargon, we take ownership of both output and buffer, but do not require ownership of x. We just need read-only access to x.

Arguments

  • output::AbstractArray: An array that will be mutated to contain the gradient of f at x.
  • f::Function: The function to be differentiated.
  • x::AbstractArray{T}: The value of x at which to evaluate the gradient of f.
  • buffer::AbstractArray{T}: A buffer that is equivalent to similar(x). Used as a temporary copy of x that can be mutated safely during the execution of the function.
  • ::CentralMode: An instance of the CentralMode type.

Returns

  • nothing::Void: This function is called for its side effects.

Examples

import FiniteDiff: gradient!, CentralMode
x = [0.0, 0.0]
output, buffer = Array(Float64, 2), Array(Float64, 2)
gradient!(
    output,
    x -> sin(x[1]) + 2 * sin(x[2]),
    x,
    buffer,
    CentralMode(),
)

source

# FiniteDiff.gradient! — Method.

gradient!{T <: AbstractFloat}(
    output::AbstractArray,
    f::Function,
    x::AbstractArray{T},
    buffer::AbstractArray{Complex{T}},
    ::ComplexMode,
)

Description

Evaluate the gradient of f at x using complex-mode finite differences. Store the results into output. Work with a user-provided buffer to ensure that we can work without copies, but also without mutating x. In Rust jargon, we take ownership of both output and buffer, but do not require ownership of x. We just need read-only access to x.

NOTE: The buffer must have complex elements to allow the computation of complex-mode finite differences.

NOTE: This mode of finite differences will work correctly only when:

  • f supports complex inputs.
  • f is an analytic function in the complex analysis sense of the word.

Arguments

  • output::AbstractArray: An array that will be mutated to contain the gradient of f at x.
  • f::Function: The function to be differentiated.
  • x::AbstractArray{T}: The value of x at which to evaluate the gradient of f.
  • buffer::AbstractArray{Complex{T}}: A buffer that is equivalent to similar(x, Complex{eltype(x)}). Used as a temporary copy of x that can be mutated safely during the execution of the function.
  • ::ComplexMode: An instance of the ComplexMode type.

Returns

  • nothing::Void: This function is called for its side effects.

Examples

import FiniteDiff: gradient!, ComplexMode
x = [0.0, 0.0]
output, buffer = Array(Float64, 2), Array(Complex{Float64}, 2)
gradient!(
    output,
    x -> sin(x[1]) + 2 * sin(x[2]),
    x,
    buffer,
    ComplexMode(),
)

source