Introduction The logistic sigmoid is one of the most popular activation functions used when working with the probability distributions associated with deep learning. It is a function that produces the \(\phi\) parameter of a Bernoulli distribution du...
Introduction The rectified linear unit function, ReLU, is one of the three most commonly used activation functions used in deep learning, including the sigmoid (sig) and hyperbolic tangent (tan) functions. It is a type of ramp function that is somewh...
Introduction The softmax function is a generalization of the logistic function to multiple dimensions. It is often used as the last activation function of a neural network to normalize the output of a network and predict the probabilities associated ...
