
Ask HN: Are there ML systems that can reason across multiple input modalities? - hsikka
I&#x27;m working on Modular Neural Networks, and I think it may be an interesting architecture. For example, you could train smaller subnets on individual problem spaces across modalities and have them feed to a larger intermediary. Is anyone working on this?
======
ArtWomb
Sounds close to what Berkeley AI and Robotics lab is developing with model
agnostic meta-learning. pytorch implementation here:

[https://github.com/tristandeleu/pytorch-maml-
rl](https://github.com/tristandeleu/pytorch-maml-rl)

And demo where robotic arm learns to navigate a simple task after watching a
single video of human doing same ;)

[https://bair.berkeley.edu/blog/2018/06/28/daml/](https://bair.berkeley.edu/blog/2018/06/28/daml/)

~~~
hsikka
Very cool, thank you!

------
screye
Yes. Multimodal emotion recognition is one such area.

There are indepdenent pipelines for each mode, and derived features are then
fused and used for final prediction.

You may also want to check out casacded models.

~~~
hsikka
Awesome thank you!

