Support for specialized CPU functions won't happen and doesn't make sense.
However, it is quite feasible to emulate, on a GPU, a networked group of a general purpose CPUs( ie, run MIMD[1] programs on SIMD[2] architecture). This, MOG[3], has been a project of Henry G. Dietz of the University of Kentucky. Unfortunately, the project seems to have stalled at a "rough" level. He claims that he can run MIMD programs at 1/4 efficiency while also running SIMD programs at near full efficiency. His video is instructive [4].
Edit: Note that this isn't intended for deep learning applications as such but rather for traditional supercomputing applications (weather prediction, other physics simulations ,etc).
However, it is quite feasible to emulate, on a GPU, a networked group of a general purpose CPUs( ie, run MIMD[1] programs on SIMD[2] architecture). This, MOG[3], has been a project of Henry G. Dietz of the University of Kentucky. Unfortunately, the project seems to have stalled at a "rough" level. He claims that he can run MIMD programs at 1/4 efficiency while also running SIMD programs at near full efficiency. His video is instructive [4].
Edit: Note that this isn't intended for deep learning applications as such but rather for traditional supercomputing applications (weather prediction, other physics simulations ,etc).
[1] https://en.wikipedia.org/wiki/MIMD [2] https://en.wikipedia.org/wiki/SIMD [3] http://aggregate.org/MOG/ [4] https://www.youtube.com/watch?v=FZ6efZFlzRQ