The problem with Linux phones is that Linux was never designed with Security in mind, and on mobile devices you want security before everything else, otherwise without security you don't even have privacy: what if they take your phone or you lose it? The current state of Linux doesn't help with that, it's like running around with an unlocked and rooted android device.
So right now, unfortunately, the only solution is Graphene OS.