add a wallet dependency to sqlite3
ccd03b6cf337
Actions

Description

add a wallet dependency to sqlite3

Summary:
PR description:

Add sqlite as an alternative wallet database and use it for new descriptor wallets

This PR adds a new class SQLiteDatabase which is a subclass of WalletDatabase. This provides access to a SQLite database that is used to store the wallet records. To keep compatibility with BDB and to complexity of the change down, we don't make use of many SQLite's features. We use it strictly as a key-value store. We create a table main which has two columns, key and value both with the type blob.

For new descriptor wallets, we will create a SQLiteDatabase instead of a BerkeleyDatabase. There is no requirement that all SQLite wallets are descriptor wallets, nor is there a requirement that all descriptor wallets be SQLite wallets. This allows for existing descriptor wallets to work as well as keeping open the option to migrate existing wallets to SQLite.

We keep the name wallet.dat for SQLite wallets. We are able to determine which database type to use by searching for specific magic bytes in the wallet.dat file. SQLite begins it's files with a null terminated string SQLite format 3. BDB has 0x00053162 at byte 12 (note that the byte order of this integer depends on the system endianness). So when we see that there is a wallet.dat file that we want to open, we check for the magic bytes to determine which database system to use.
I decided to keep the wallet.dat naming to keep things like backup script to continue to function as they won't need to be modified to look for a different file name. It also simplifies a couple of things in the implementation and the tests as wallet.dat is something that is specifically being looked for. If we don't want this behavior, then I do have another branch which creates wallet.sqlite files instead, but I find that this direction is easier.

This is a backport of core#19077 [1 & 2 & 24/26]
https://github.com/bitcoin/bitcoin/pull/19077/commits/54729f3f4e6765dfded590af5fb28c88331685f8
https://github.com/bitcoin/bitcoin/pull/19077/commits/e87df8258090138d5c22ac46b8602b618620e8a1
https://github.com/bitcoin/bitcoin/pull/19077/commits/6c6639ac9f6e1677da066cf809f9e3fa4d2e7c32

The last commit is a partial backport. All the necessary documentation to build Bitcoin ABC after adding the new dependency are included, but not general documentation (files.md)

Test Plan: