Softplus Function Calculator

Compute the softplus function softplus(x)=ln(1+eˣ), its derivative (the sigmoid), and the gap from ReLU. Visualize this machine learning activation with a built-in graph.

Input

Enter x to compute the softplus function softplus(x)=ln(1+eˣ), its first derivative (the sigmoid), and the gap from ReLU.

Input value x

Enter any real number (for example 1, -2.5, or 0).

Result

softplus(1) = ln(1+e^(1))

1.3132616875

Derivative σ(1) (sigmoid)

0.7310585786

ReLU(1) = max(1, 0)

softplus − ReLU gap

0.3132616875

Softplus vs ReLU graph

softplus(x)

ReLU(x)

How it works

The softplus function is defined as softplus(x)=ln(1+eˣ) and returns a positive value for every real number x. Its output is always greater than 0 and approaches x itself as x grows large.
The first derivative of softplus is exactly the standard sigmoid (logistic) function σ(x)=1/(1+e^(-x)). This derivative ranges from 0 to 1 and is used when updating weights with gradient methods.
Softplus is a smooth approximation of ReLU(x)=max(x,0). While ReLU has a sharp corner at the origin, softplus is differentiable everywhere, so the gradient never abruptly vanishes.
It is used as an activation function in machine learning and neural networks. Because the output is always positive, it also suits layers that must emit positive quantities such as a variance.
To avoid exponential overflow, the value is evaluated in the form max(x,0)+ln(1+e^(-|x|)). For very large x, softplus(x)≈x; for very negative x, softplus(x)≈0.

Reviews

Tell us what you think of this calculator.