libconsensus: Add input validation of flags

Makes it an error to use flags that have not been defined
on the libconsensus API.

There has been some confusion as to what pass to libconsensus, and
(combined with mention in the release notes) this should clear it up.

Using undocumented flags is a risk because their meaning,
and what combinations are allowed, changes from release to release.
E.g. it is no longer possible to pass (CLEANSTACK | P2SH) without
running into an assertion after the segwit changes.