

Avoiding overflow trick: log(a+b) from log(a) and log(b) - patpng
https://facwiki.cs.byu.edu/nlp/index.php/Log_Domain_Computations

======
afsina
I was dealing and implementing this last week. From what I understood,
calculating log(a+b) from log(a) and log(b) has reasons other than underflow-
overflow problem.

1- it is usually a necessity because For example in Speech Recognition and
many NLP applications people always deal with log probabilities and log
likelihoods.Yes they do it for running away from the underflow problem but all
formulation and data is based on log.

2- Also, when dealing with log probabilities, it is possible to generate a
look-up table to make this calculation much faster (Around 30 folds) with a
good approximation instead of calculating the log(1+exp(log(b)-log(a))) part.

This is helpful for example when trying to find how likely a piece of speech
signal is generated from an utterance of a letter (not formally speaking)
using Gaussian Mixture Models. LogSum is extensively used there. It is also
used in Forward-Backward algorithm used in Hidden Markov Model generation and
decoding.

Edited for some grammar and missing information.

~~~
tensor
Useful to you?

[https://github.com/akhudek/feast/blob/master/src/btl/trunk/b...](https://github.com/akhudek/feast/blob/master/src/btl/trunk/btl/logint.h)

~~~
afsina
Pretty good, thanks. But because I am using Java this may not be that useful
for me. I will definitely check the math tough. I will send it to my colleague
as well who deals with C++ GMMs.

------
Radim
Don't try this at home, there are nasty edge cases around Infs/NaNs.

Use your math library instead; in Python, that'd be _numpy.logaddexp_ or
_scipy.misc.logsumexp_.

------
japaget
Site is down, Google cache:

[http://webcache.googleusercontent.com/search?q=cache:https:/...](http://webcache.googleusercontent.com/search?q=cache:https://facwiki.cs.byu.edu/nlp/index.php/Log_Domain_Computations&hl=en&biw=1528&bih=859&strip=1)

------
aaronjg
I've run into this problem often when dealing with log probabilities in
maximum likelihood estimation where the optimizer can try some numbers that
are way off from the optimal value and you still want to return an accurate
number. There is an package in R for handling these operations automatically
[1].

[1]
[http://cran.r-project.org/web/packages/Brobdingnag/index.htm...](http://cran.r-project.org/web/packages/Brobdingnag/index.html)

------
sanxiyn
See also <http://en.wikipedia.org/wiki/Logarithmic_number_system>

