That's going to be something similar to IPAdapter FaceID: https://ipadapterfaceid.com
Basically you use a facial structure representation that you'd use for face recognition (which of course Apple already compute on all your photos) together with some additional feature representations to guide the image generation.
No need for additional fine-tuning. A similar approach could likely be used for handwriting generation.
