But in industry Worse Is Better and Kalman Filters is ofent just the optimal way to remove noise you yourself added in your simulation.
I've never seen any implementation use of Kalman filters where the covariance matrix is actually sound ... and I usually go with just lowpassfilters or moving averages.
It's industry standard in aerospace, granted one usually uses the EKF with nonlinear dynamics so the covariance matrix is not estimated perfectly. That setup is also flexible enough to let you introduce of band measurements from the past but with the correct timestamp and correct.
It's kind of interesting that how particular fields chose to embrace certain particular methods; till I have worked with aerospace domain,I never had heard of the Kalman filter, whereas some of the newer and even more fundamental methods like wavelets and compressed sensing appear to have such low traction in Aerospace till now.
You probably know it by some other name. It is so classical its unlikely that one hasn't run into it in some form or the other. If you assume a hidden Markov model with Gaussian state transition and Gaussian output and work out the recursive update equation KF is what you will get. It might not have been call a Kalman filter. May be Weiner filter will ring a bell, under certain assumptions they become the same thing. If not Weiner filter, recursive least squares would surely ring a bell.
Kalman filters were invented for moon lander navigation in the Apollo program, so they had taken firm root especially in the days of limited data processing capability.
A while ago (around 2 or so years) I did quite a bit of mechatronics and Kalman Filters were used resonable effectively for data fusion in IMUs. I wouldn't be surprised if there were better methods around but all the standard chipsets used them at that time.
EDIT: I forgot but I've seen them used more recently by CERN for particle trajectory prediction as part of their hit tracking system. I only know this since they wrote about it in some of their supporting documentation for their latest kaggle comp [0].
We do control stuff in marine radar and target tracking in noisy environments. We use EKF and UKF all the time, and in our cases the covariance matrix is definitely sound, useful, and provides information that we use.
Ye well maybe it's a domain thing. I do driveline programming for cars.
As soVeryTired wrote "In general I find Kalman filters a little suspect where the underlying dynamics of the system can change over time, and when the physical process that gives rise to the dynamics isn't known."
This maybe applies in a less degree to radars etc?
In general we use only one sensor source at a time for eg. rpm rather then fusing them. If one is obvously off just select another. Maybe it's rather about how exact you need the estimate to be.
Also chiming in - I do control stuff in aerospace, Kalman Filters are the industry standard, have been so for decades, verging on low-tech now. Lots of interest in online particle filters in my little niche currently.
What you're saying is definitely appropriate for finance. In general I find Kalman filters a little suspect where the underlying dynamics of the system can change over time, and when the physical process that gives rise to the dynamics isn't known.
I've used them in radar with an actual covariance matrix. I know some people use them in robotics but I've always relied on Particle Filters. And a woman I've been dating has used them in her econometrics work and I'm pretty sure she used an actual covariance matrix given the academic context.
But in industry Worse Is Better and Kalman Filters is ofent just the optimal way to remove noise you yourself added in your simulation.
I've never seen any implementation use of Kalman filters where the covariance matrix is actually sound ... and I usually go with just lowpassfilters or moving averages.