The unbroadcast_txids set only contains the txid. If for some reason a tx gets rejected when loading the mempool from the database, there is no point in loading the corresponding txid in the unbroadcast set, as we don't have the tx any longer for broadcasting.
This is a partial backport of core#18044 and core#19879
https://github.com/bitcoin/bitcoin/pull/18044/commits/c7eb6b4f1fe5bd76388a023529977674534334a7
https://github.com/bitcoin/bitcoin/pull/19879/commits/cb79b9dbf4cd06e17c8c65b36bf15c3ea2641de4