Libertarians love to claim he was one so they can ride his coattails