Aside from the practicality of expanding fast enough (Dyson sphere around the sun in about 1400 years, needing to leave the galaxy in 2400 years), how far can you feasibly bring resources back to earth from to fuel that growth here? If you can't do that at some point, you have to stop growing locally even if the species is expanding. And besides, without FTL travel we're not going to be colonizing anything, just spreading seeds of new societies and, eventually, species that happen to be our offspring.
