log: Avoid treating remote misbehvior as local system error

Summary:

When logging failures of CheckBlockHeader (high-hash), they are always logged as system error. This is problematic for several reasons:

• Submitting a blockheader that fails CheckBlockHeader over RPC will result in a debug log line that starts with ERROR. Proper behaviour should be to log not anything and instead only return the failure reason to the RPC user. This pull does not fix this issue entirely, but is a good first step in the right direction.
• A misbehaving peer that sends us an invalid block header that fails CheckBlockHeader will result in a debug log line that starts with ERROR. Proper behavior should be to log the remote peer misbehavior if logging for that category was enabled. This pull fixes this issue for CheckBlockHeader and other functions can be adjusted as well if needed in follow-ups. This should be a good first step in the right direction.

refactor: Switch ValidationState mode to C++11 enum class

log: Avoid treating remote misbehvior as local system error

This is a backport of core#19526

Test Plan: ninja all check-all

Reviewers: #bitcoin_abc, majcosta

Reviewed By: #bitcoin_abc, majcosta

Differential Revision: https://reviews.bitcoinabc.org/D10021