The definition of libertarian: People who believe the initiation of force is immoral.
Thus, those trying to forcibly take your property are initiating force. Nothing about that being immoral precludes you from defending yourself or your property.
How is this different from the traditional role of government? When – with the exception of violations of civil liberties (which we probably agree are unethical) – does the government initiate force against those who are not, by consensus of the population, harming others?
That is, what is your definition of "harm" which is broad enough to cover, say, non-violent theft (as my example) and incarceration, but narrow enough to exclude, say, polluting a river? (Or choose another example where you think the government overreaches.)
Further questions:
If I am Libertarian and sovereign me believe polluting a river harms me, so I shoot you; does that give you or your family, who does not believe polluting a river to be harm/violence, the Libertarian-moral standing to shoot me? i.e. do we settle this Hatfield and McCoy style? Would the winners be those with the biggest guns, or do Libertarians, in an attempt to belay further violence, put things to a vote and only initiate force against those who go against the vote?
If I am Libertarian, but am infirm and thus unable to properly defend myself, is it Libertarian-moral to outsource my force-initiation to an outside group? If so, how do the services this group provide differ materially from the services the executive branch of the government provides? If not, what is my recourse; should I die because I am physically weak?