Module cgl_rs::math::activations

Some popular activation functions and their derivatives.

Functions

• relu
  Calculates the ReLU activation function for a given input.
• relu_derivative
  Calculates the derivative of the ReLU activation function for a given input.
• relu_leaky
  Calculates the leaky ReLU activation function for a given input.
• relu_leaky_derivative
  Calculates the derivative of the leaky ReLU activation function for a given input.
• relu_smooth
  Calculates the smooth ReLU activation function for a given input.
• relu_smooth_derivative
  Calculates the derivative of the smooth ReLU activation function for a given input.
• sigmoid
  Calculates the sigmoid activation function for a given input.
• sigmoid_derivative
  Calculates the derivative of the sigmoid activation function for a given input.
• step
  Calculates the step activation function for a given input.
• step_derivative
  Calculates the derivative of the step activation function for a given input.
• tanh
  Calculates the hyperbolic tangent activation function for a given input.
• tanh_derivative
  Calculates the derivative of the hyperbolic tangent activation function for a given input.