This diff moves the last CAddrMan protected member to be private, at the cost of an extra friend class. This will make moving to PIMPL easier.

There is no change in behavior.