travis: use debug for one build

For the all-off build, enable the wallet and debug. This ensures that debug
options will catch wallet problems as well.

In order to make sure the no-wallet path is still tested, disable the wallet
in the other x86_64 build.