Right, but if you're never going to look at the method again and/or remove it next week for whatever reason, you've just wasted a bunch of time that could be spent on more valuable things.
My thinking is definitely more focused on early stage start ups/personal side projects than bigger mature projects.
But what I'm trying to get across is that I'm starting to wonder if "big clunky code base that's hard to support, maintain and understand" is in the same class of good to have problems for early stage start ups/side projects as "I chose a technology that is tough to scale to 500 million users" as long as you have a comprehensive test suite.
I tend to vary the amount of refactoring I do depending on how much code is likely to depend on it. For instance I'd spend more effort in the DOM than I would in the controller (remember this is degrees of effort not lots/none). There's also nothing to stop you doing refactoring _before_ you start on a piece of code. Maybe you didn't think it would be worth refactoring at the time but now you're adding a new feature you can see that applying the refactoring would make it easier.