(x + √(x² + 1)) is a bit like a "soft relu". When x<0 it goes to zero. When x>0 ...

yobbo 8 months ago | parent | context | favorite | on: Logarithmic Scales

(x + √(x² + 1)) is a bit like a "soft relu". When x<0 it goes to zero. When x>0 it goes to 2x.