If you have the means to collect the data then this seems pretty tractable to me. Mostly because you don't need to deal with the raw optical data but rather just the derived trajectories. So data formats and volumes shouldn't be a distraction.
I'm assuming you've done some tutorials or courses on basics of DL and can program Python. At that point the easiest first step would be to just train an MLP to convert a single time step from inertial data to match the optical prediction (presumably they are in the same coordinate system and temporally close enough).
The crux of building something good would be in how you handle the temporal aspect I'd imagine. Clearly you want to use multiple samples over time from the inertial to get more accurate positional estimates. I'd imagine a fixed window of the past n inertial samples would be a good start. I wouldn't worry about more complicated temporal modeling, e.g. RNN or transformer, unless you can't get satisfactory results with the MLP.
My gut says there's probably a non-ML approach to this too, some sort of Kalman Filter etc. Always best to avoid ML if a simpler solution exists :)
Awesome thank you! Really appreciate you taking the time to reply. I’m still learning Python and working my way through this course https://fleuret.org/dlc/ but without a problem I care about its hard. Your encouragement and tips are much appreciated! Knowing that this idea isn’t too ambitious will give me the motivation to keep pushing. Thank you.
I'm assuming you've done some tutorials or courses on basics of DL and can program Python. At that point the easiest first step would be to just train an MLP to convert a single time step from inertial data to match the optical prediction (presumably they are in the same coordinate system and temporally close enough).
The crux of building something good would be in how you handle the temporal aspect I'd imagine. Clearly you want to use multiple samples over time from the inertial to get more accurate positional estimates. I'd imagine a fixed window of the past n inertial samples would be a good start. I wouldn't worry about more complicated temporal modeling, e.g. RNN or transformer, unless you can't get satisfactory results with the MLP.
My gut says there's probably a non-ML approach to this too, some sort of Kalman Filter etc. Always best to avoid ML if a simpler solution exists :)