Yes, that is how I verify accounts before following.
I'm very wary of delegating authority to any individual or well-defined group, even just white-listing.
I think people would be more tolerant of spam-whitelisting authorities than you think - "yeah he allows TON spam, but at least he never approves Flat Earthers" etc etc.
That way lies madness and #fediverse...