A couple of functions and a constant are converted to bytes, to avoid unnecessary hex decoding in the new function. All exisiting callsites for these functions are adjusted to expect bytes.

Signatures and keys are stored as bytes (can be trivially converted to hex if necessary)

This diff introduces a few conversions to hex -- are these new conversions less costly than hex decoding? Why was everything in hex before ... ? Was it just easier to write it that way, was hex decoding unanticipated until later features came in?

In D14458#324554, @bytesofman wrote:

This diff introduces a few conversions to hex -- are these new conversions less costly than hex decoding? Why was everything in hex before ... ? Was it just easier to write it that way, was hex decoding unanticipated until later features came in?

The end goal here is to have bytes everywhere for internal data structures, which will ultimately make everything faster and less costly, and convert to hex only for screen display or file IO (exporting coins to JSON...)
To avoid making one giant impossible to review diff, the conversions to hex are necessary when interfacing converted function to functions that still work with hex. This should be transitory.

I don't know why the choice was made to work mostly with hex, but it didn't make much sense to start with. I may have been influenced by Python 2's bad design (string and bytes were the same type, so working with bytes was a bit more confusing, you never knew if you were dealing with raw data or hex data.

The tests are the one place where hex is still going to be used, just for convenience and readability ("coffeedeadbeef" vs b"\xco\xff\xee\xde\xad\xbe\xef").

Maybe what would make sense is to only store the xpubkey when it is actually different from the pubkey. It will require more codechanges, though, because of all the code that reads x_pubkeys and assumes that it will contain a pubkey when the xpub is not necessary.