>Strict libertarians are all about keeping our nose (and military) out of other people's business.
That just makes libertarians bad, not you a bad libertarian. Libertarianism is marked by a lack of concern for others and a reticence to help them. Whether it's social darwinism, isolationism, or opposition to social programs to help the less fortunate, the defining feature of libertarianism seems to be saying "fuck you, help your own self" to anyone in need.
That's more of a caricature of libertarianism. Libertarians are against government for various reasons (some view it as inefficient and corrupt, others because they believe it's morally wrong to violate property rights and force people to do things.) However none but objectivists/ayn-rand-types would say "fuck you, help your own self". They would just support private charities and the like over government.
I am not a libertarian anymore mostly because I believe people should be forced to help others, but I do understand what they believe.
That's a pretty silly idea of libertarianism. I think you're equating it with things like anarcho-capitalism and minarchism instead of proper anarchism.
Maybe I'm grossly misunderstanding libertarianism, but I've never seen a libertarian say, "we have a duty to help those in need". I've never seen a libertarian say, "the fortunate have a moral obligation to help the less fortunate", or "the fact that so few have so much while so many have so little is a fundamental injustice that needs to be rectified".
Instead, I read, "if you feel like helping someone it's good to do so, but you have no moral obligation to it", and "the problem with the poor is that we're not incentivizing them to get out of poverty", both of which display a profoundly alien concept of human morality and indeed human nature. Libertarianism is the conceit of the fortunate that they are deserving, and that the unfortunate are undeserving.
You have to judge based on the libertarian. Many political philosophies attract hoards of what I call "SWE Groupies." When you apply the Small World Effect to political philosophy, you will find that you can easily blame all the worlds problems on anything you want (the government, the patriarchy, the market, etc) because everything is interconnected. Based on purely anecdotal evidence, it seems that almost everyone who eventually wanders away from the political philosophy they were born into falls into the trap of blaming exactly one of these entities when the usual state of affairs is that they all hold blame to an uneven and debatable extent that changes depending on the issue in question. The worst result of such naivete is that it creates blind spots. Libertarians fail to propose measures to control market failures, socialists fail to propose measures to control bureaucracy, etc. In any case, most people are eventually confronted with enough counterexamples to shake them out of their blind faith and they move on to a mix-and-match philosophy that proposes systems more similar to real, proven governments (funny how that works).
This leaves the problem of finding a label for yourself, which can no longer be done in an absolutely precise manner. Milton Friedman, a "philosophically mature" libertarian, might agree with a devout socialist on a great many things (giving money to poor people), even though they identify with philosophies that look like polar opposites to the corresponding SWE Groupies. I bet that you show segments of the video I linked to SWE Libertarians and get them to swear up and down that this guy was a socialist.
People like Milton Friedman don't deserve to be lumped with people like Ayn Rand, and just because SV is infested with SWE Groupies of the libertarian persuasion (it's convenient to be a libertarian when the market is smiling upon your profession) doesn't mean that you won't find yourself on the wrong end of a political philosophy debate if you attack libertarianism as a whole rather than individual points raised in the argument.
That just makes libertarians bad, not you a bad libertarian. Libertarianism is marked by a lack of concern for others and a reticence to help them. Whether it's social darwinism, isolationism, or opposition to social programs to help the less fortunate, the defining feature of libertarianism seems to be saying "fuck you, help your own self" to anyone in need.