The bit is used for information because information is almost defined as the expected length of an optimally (context-free) coded word, using a base 2 code. Some physicists prefer base e, in which case it's measured in nats.
To put it another way, the triple point of water is known to be 273.16 K, and the boiling point 373.15 K. Now if this mapped on to 0 °C and 100 °C, then we'd be left with a situation where 100 kelvins would be equivalent to 100.01 Celsius degrees. Instead, by defining the triple point of water as 0.01 °C (still 273.16 K), we now get an interval of 100 kelvins being equal to an interval of 100 Celsius degrees.
This is also why the term centigrade is no longer used... it's no longer 100 even.
We would probably need either very tiny or very large multiplies of this new Kelvin for anything practical, though. (Depending on your choice for the other units. The The Planck unit of temperature is way too big for practical uses for example.)