It seems like the only reason for using hex strings in this method was to have a convenient way to convert to an integer from the input data interpreted as big-endian.
In Python3 we have int.from_bytes(..., 'big') for that purpose, hence there is no need for that anymore and we can simply operate on bytes only.
This is a backport of core#24324
Depends on D11136