I don’t like the tokens to power analogy. Power is a well defined physical quantity. Tokens are just an ML implementation detail.
> To quantify this for his new steam engine, James Watt developed the unit “horsepower”, which he and Matthew Boulton standardized at 33,000 foot-pounds in 1783.
> By analogy to “horsepower”, AI can be measured in terms of “thought-hours”. Whereas horsepower quantifies physical work relative to horses, a thought-hour quantifies cognitive work relative to humans.
The idea of “the <mechanical device> does an amount of <type of work> equal to what the <biological creature> does per hour” fits, but the crucial part is the end of the first quote; having enough understanding to actually standardize that new made up unit in a physically meaningful way. For thoughts, intuitively, it would seem to map to information, entropy, that sort of stuff, but of course nobody has worked it out yet.
I think we’re still in the era where people are digging up black rocks and noticing that sometimes if they stick it in their furnace, it’ll heat it up. Or make the iron into steel. It is a mystery!
> It would be absurd to quantify the net generation of electricity generators in the U.S. in 2023 as “15.05 quintillion joules”. We have watt-hours for that (3,600 joules) and physics prefixes to quantify the orders of magnitude.
I don’t understand this bit at all. There’s nothing particularly absurd about using joules for these quantities of energy other than the fact that it is unconventional. We’d just have to switch the SI prefix.
What's interesting is how difficult somes things are to reason about. This goes to complexity and time, not necessarily straight computation. We don't have good categorizations of complexity and emergence, let alone cognition, to be able to quantify it.
All we have are proxies, like energy use. The kicker is, that as the systems grow more efficient from advances in the algorithm, even that won't be a good proxy for measuring units of "thought."
These systems will clearly be able to "out-think" some humans, just by virtue of them holding more knowledge than some humans. Our definitions of reasoning and "thinking" need an update.
Current systems may not be able to out-think the greatest human minds, but they can certainly summarize their data sets better than many humans can, especially when allowed multiple rounds to evaluate their own output before submitting it.
> To quantify this for his new steam engine, James Watt developed the unit “horsepower”, which he and Matthew Boulton standardized at 33,000 foot-pounds in 1783.
> By analogy to “horsepower”, AI can be measured in terms of “thought-hours”. Whereas horsepower quantifies physical work relative to horses, a thought-hour quantifies cognitive work relative to humans.
The idea of “the <mechanical device> does an amount of <type of work> equal to what the <biological creature> does per hour” fits, but the crucial part is the end of the first quote; having enough understanding to actually standardize that new made up unit in a physically meaningful way. For thoughts, intuitively, it would seem to map to information, entropy, that sort of stuff, but of course nobody has worked it out yet.
I think we’re still in the era where people are digging up black rocks and noticing that sometimes if they stick it in their furnace, it’ll heat it up. Or make the iron into steel. It is a mystery!
> It would be absurd to quantify the net generation of electricity generators in the U.S. in 2023 as “15.05 quintillion joules”. We have watt-hours for that (3,600 joules) and physics prefixes to quantify the orders of magnitude.
I don’t understand this bit at all. There’s nothing particularly absurd about using joules for these quantities of energy other than the fact that it is unconventional. We’d just have to switch the SI prefix.