Which programming languages properly handle irrational and transcendental numbers such that loss of precision, etc. is not a problem theoretically? (It doesn't matter whether the ability is built-in to the language or available via existing libraries.)
You're looking for the search term "symbolic math". You can get specialized programs like Matlab, Octave, or Mathematica and hope their general purpose stuff will be enough to carry you through, or you can hook to them from your general-purpose language. You can also search for a "symbolic math library" for the language of your choice. Test anything you find very thoroughly for your needs.
Without more details, I doubt you'll get more than a search term. With more details, maybe somebody else can give you a more specific recommendation. (Not me, though, I've done little more than fiddle in this domain.)
There is an implementation of exact real arithmetic that has been verified in in Coq[1] and also an implementation in Haskell[2] which may or may not do what you want to do.
It really does depend on what you want to compute. For instance, I can provide a simple implementation in Haskell that includes addition, multiplication and square roots:
data Real = Real
add :: Real -> Real -> Real
add _ _ = Real
multiply :: Real -> Real -> Real
multiply _ _ = Real
squareRoot :: Real -> Real -> Real
squareRoot _ _ = Real
zero :: Real
zero = Real
one :: Real
one = Real
but it isn't very interesting as you can't print things or test for equality...Any implementation of real numbers that includes those is going to be much harder[3].
Without more details, I doubt you'll get more than a search term. With more details, maybe somebody else can give you a more specific recommendation. (Not me, though, I've done little more than fiddle in this domain.)