The Soboleva modified hyperbolic tangent, also known as (parametric) Soboleva modified hyperbolic tangent activation function ([P]SMHTAF),^{[nb 1]} is a special S-shaped function based on the hyperbolic tangent, given by

Equation	Left tail control	Right tail control
${\displaystyle \operatorname {smht} x={\frac {e^{ax}-e^{-bx}}{e^{cx}+e^{-dx}}}.}$

History

This function was originally proposed as "modified hyperbolic tangent"^{[nb 1]} by Ukrainian scientist Elena V. Soboleva (Елена В. Соболева) as a utility function for multi-objective optimization and choice modelling in decision-making.^[1][2][3]

Practical usage

The function has since been introduced into neural network theory and practice.^[4]

It was also used in economics for modelling consumption and investment,^[5] to approximate current-voltage characteristics of field-effect transistors and light-emitting diodes,^[6] to design antenna feeders,^[7] and analyze plasma temperatures and densities in the divertor region of fusion reactors.^[8]

Sensitivity to parameters

Derivative of the function is defined by the formula:

$${\displaystyle \operatorname {smht} '(x)\doteq {\frac {ae^{ax}+be^{-bx}}{e^{cx}+e^{-dx}}}-\operatorname {smht} (x){\frac {ce^{cx}-de^{-dx}}{e^{cx}+e^{-dx}}}}$$

The following conditions are keeping the function limited on y-axes: a ≤ c, b ≤ d.

A family of recurrence-generated parametric Soboleva modified hyperbolic tangent activation functions (NPSMHTAF, FPSMHTAF) was studied with parameters a = c and b = d.^[9] It is worth noting that in this case, the function is not sensitive to flipping the left and right-sides parameters:

Equation	Left prevalence	Right prevalence
${\displaystyle {\frac {e^{ax}-e^{-bx}}{e^{ax}+e^{-bx}}}={\frac {e^{bx}-e^{-ax}}{e^{bx}+e^{-ax}}}}$

The function is sensitive to ratio of the denominator coefficients and often is used without coefficients in the numerator:

Equation	Basic chart	Scaled function
${\displaystyle \operatorname {ssht} x={\frac {e^{x}-e^{-x}}{e^{\alpha x}+e^{-\beta x}}}.}$ Extremum estimates: ${\displaystyle x_{\min }={\frac {1}{2}}\ln {\frac {\beta -1}{\beta +1}};}$ ${\displaystyle x_{\max }={\frac {1}{2}}\ln {\frac {\alpha +1}{\alpha -1}}.}$

With parameters a = b = c = d = 1 the modified hyperbolic tangent function reduces to the conventional tanh(x) function, whereas for a = b = 1 and c = d = 0, the term becomes equal to sinh(x).

See also

• Activation function
• e (mathematical constant)
• Equal incircles theorem, based on sinh
• Hausdorff distance
• Inverse hyperbolic functions
• List of integrals of hyperbolic functions
• Poinsot's spirals
• Sigmoid function

Notes

References

Further reading

• Iliev, Anton; Kyurkchiev, Nikolay; Markov, Svetoslav (2017). "A Note on the New Activation Function of Gompertz Type". Biomath Communications. Faculty of Mathematics and Informatics, University of Plovdiv "Paisii Hilendarski", Plovdiv, Bulgaria / Institute of Mathematics and Informatics, Bulgarian Academy of Sciences, Sofia, Bulgaria: Biomath Forum (BF). 4 (2). doi:10.11145/10.11145/bmc.2017.10.201. ISSN 2367-5233. Archived from the original on 2020-06-20. Retrieved 2020-06-19. (20 pages)