Huh. No issues like that at all with any of my Linux desktops and analog jacks (mostly Devuan). They all seem to detect and maintain sources fine.
The problem I personally have (if there are any pulseaudio experts out there) is it continually re-enabling ephemeral audio sources like HDMI and bluetooth when they are plugged back in, even if I'd set them to "off" before unplugging them - fortunately since I set the analog to default it usually doesn't mess anything up, but it's a bit untidy, and also annoying for the one USB webcam that has a mic plugged in, where I get a surprise input if I don't watch it.
It'd be great to have it persist settings for identically named sources even after they are removed.
The problem I personally have (if there are any pulseaudio experts out there) is it continually re-enabling ephemeral audio sources like HDMI and bluetooth when they are plugged back in, even if I'd set them to "off" before unplugging them - fortunately since I set the analog to default it usually doesn't mess anything up, but it's a bit untidy, and also annoying for the one USB webcam that has a mic plugged in, where I get a surprise input if I don't watch it.
It'd be great to have it persist settings for identically named sources even after they are removed.