All a priori axioms are just like all axioms in math: arbitrary but assumed to be true, from which bigger systems can be crafted.
The nail in the coffin for me against libertarian philosophy was the deep realization that individual-human-body free will does not exist. Humans are not separate from the rest of nature, and a prioritization of ownership of the “self” rather than group-ownership is entirely arbitrary. For example, Eastern cultures (e.g., Japan) place a much higher priority on honoring their family, and in what sense is that less valid than American individualism?
The issue is that individual-human-bodies don’t technically exist. You and I are connected through gravity, and in no real sense can be considered separate. In that way, there is no “self” that is doing violence, rather it is the math of nature working itself out.
That is, defining the body as the boundary of a “self” is arbitrary, and it’s really just as valid to define the boundary and “self” as a family, country, or species. Just as you don’t consider one cell of your body doing “violence” on another, so is it just as valid to see not one “person” as doing violence on another.
Of course, I prefer liberty to slavery. I do tend to believe that there is a practical argument for libertarianism, but the philosophy itself is fundamentally arbitrary. Because this is the case, libertarians (as well as conservatives and liberals) should not claim that they have the ultimate truth. Instead, we should try novel micro-governments like seasteading and the Free State Project in order to see if there are better ways for humans to organize.