A) From a set of 2d images, construct 3d models of a person in that pose. Map the original image pixels onto that model (as a texture?)
B) Rotate those models in 3d space to look for coincidence, and work backwards to figure out the respective angles, and merge the 3d models.
C) Iterate to improve the solution
Much as the SIFT method for generating scale free "interesting points" for 2d images enabled the panorama stitching software of the past decade, this enables a new software space that should be quite interesting.