I'd agree. It's entirely possible to be an early adopter on something that proves to have real value, and reap rewards from taking that risk.
Altcoin mining for conversion to Bitcoin, however, has the marks of Ponzi: returns (altcoin:bitcoin exchange rate) that require a steady stream of new investment (new alts, as old alts become unprofitable) that provide little to no value other than providing the return.
That doesn't make it a Ponzi scheme.