Changeset View
Changeset View
Standalone View
Standalone View
doc/developer-notes.md
Show First 20 Lines • Show All 203 Lines • ▼ Show 20 Lines | |||||
# output goes to doc/doxygen/html/ | # output goes to doc/doxygen/html/ | ||||
``` | ``` | ||||
Development tips and tricks | Development tips and tricks | ||||
--------------------------- | --------------------------- | ||||
### Compiling for debugging | ### Compiling for debugging | ||||
Run configure with `--enable-debug` to add additional compiler flags that | Run `cmake` with `-DCMAKE_BUILD_TYPE=Debug` to add additional compiler flags | ||||
produce better debugging builds. | that produce better debugging builds. | ||||
### Compiling for gprof profiling | ### Compiling for gprof profiling | ||||
Run configure with the `--enable-gprof` option, then make. | |||||
With `cmake` and `ninja`: | |||||
``` | ``` | ||||
cmake -GNinja .. -DENABLE_HARDENING=OFF -DENABLE_PROFIILING=gprof | cmake -GNinja .. -DENABLE_HARDENING=OFF -DENABLE_PROFIILING=gprof | ||||
``` | ``` | ||||
### debug.log | ### debug.log | ||||
If the code is behaving strangely, take a look in the debug.log file in the data directory; | If the code is behaving strangely, take a look in the debug.log file in the data directory; | ||||
error and debugging messages are written there. | error and debugging messages are written there. | ||||
Show All 11 Lines | |||||
For details on functional tests, see `functional-tests.md` | For details on functional tests, see `functional-tests.md` | ||||
### Writing script integration tests | ### Writing script integration tests | ||||
Script integration tests are built using `src/test/script_tests.cpp`: | Script integration tests are built using `src/test/script_tests.cpp`: | ||||
1. Uncomment the line with `#define UPDATE_JSON_TESTS` | 1. Uncomment the line with `#define UPDATE_JSON_TESTS` | ||||
2. Add a new TestBuilder to the `script_build` test to cover your test case. | 2. Add a new TestBuilder to the `script_build` test to cover your test case. | ||||
3. `make && ./src/test/test_bitcoin --run_test=script_tests` | 3. `ninja check-bitcoin-script_tests` | ||||
4. Copy your newly generated test JSON from `<build-dir>/src/script_tests.json.gen` to `src/test/data/script_tests.json`. | 4. Copy your newly generated test JSON from `<build-dir>/src/script_tests.json.gen` to `src/test/data/script_tests.json`. | ||||
Please commit your TestBuilder along with your generated test JSON and cleanup the uncommented #define before code review. | Please commit your TestBuilder along with your generated test JSON and cleanup the uncommented #define before code review. | ||||
### Testnet and Regtest modes | ### Testnet and Regtest modes | ||||
Run with the `-testnet` option to run with "play bitcoins" on the test network, if you | Run with the `-testnet` option to run with "play bitcoins" on the test network, if you | ||||
are testing multi-machine code that needs to operate across the internet. | are testing multi-machine code that needs to operate across the internet. | ||||
If you are testing something that can run on one machine, run with the `-regtest` option. | If you are testing something that can run on one machine, run with the `-regtest` option. | ||||
In regression test mode, blocks can be created on-demand; see [test/functional/](/test/functional) for tests | In regression test mode, blocks can be created on-demand; see [test/functional/](/test/functional) for tests | ||||
that run in `-regtest` mode. | that run in `-regtest` mode. | ||||
### DEBUG_LOCKORDER | ### DEBUG_LOCKORDER | ||||
Bitcoin ABC is a multi-threaded application, and deadlocks or other | Bitcoin ABC is a multi-threaded application, and deadlocks or other | ||||
multi-threading bugs can be very difficult to track down. The `--enable-debug` | multi-threading bugs can be very difficult to track down. | ||||
configure option adds `-DDEBUG_LOCKORDER` to the compiler flags. This inserts | The `-DCMAKE_BUILD_TYPE=Debug` cmake option adds `-DDEBUG_LOCKORDER` to the | ||||
run-time checks to keep track of which locks are held, and adds warnings to the | compiler flags. This inserts run-time checks to keep track of which locks are | ||||
debug.log file if inconsistencies are detected. | held, and adds warnings to the debug.log file if inconsistencies are detected. | ||||
### Valgrind suppressions file | ### Valgrind suppressions file | ||||
Valgrind is a programming tool for memory debugging, memory leak detection, and | Valgrind is a programming tool for memory debugging, memory leak detection, and | ||||
profiling. The repo contains a Valgrind suppressions file | profiling. The repo contains a Valgrind suppressions file | ||||
([`valgrind.supp`](contrib/valgrind.supp)) | ([`valgrind.supp`](contrib/valgrind.supp)) | ||||
which includes known Valgrind warnings in our dependencies that cannot be fixed | which includes known Valgrind warnings in our dependencies that cannot be fixed | ||||
in-tree. Example use: | in-tree. Example use: | ||||
Show All 30 Lines | |||||
to the `cmake` command line. | to the `cmake` command line. | ||||
### Sanitizers | ### Sanitizers | ||||
Bitcoin ABC can be compiled with various "sanitizers" enabled, which add | Bitcoin ABC can be compiled with various "sanitizers" enabled, which add | ||||
instrumentation for issues regarding things like memory safety, thread race | instrumentation for issues regarding things like memory safety, thread race | ||||
conditions, or undefined behavior. This is controlled with the | conditions, or undefined behavior. This is controlled with the | ||||
`--with-sanitizers` configure flag, which should be a comma separated list of | `-DENABLE_SANITIZERS` cmake flag, which should be a semicolon separated list of | ||||
sanitizers to enable. The sanitizer list should correspond to supported | sanitizers to enable. The sanitizer list should correspond to supported | ||||
`-fsanitize=` options in your compiler. These sanitizers have runtime overhead, | `-fsanitize=` options in your compiler. These sanitizers have runtime overhead, | ||||
so they are most useful when testing changes or producing debugging builds. | so they are most useful when testing changes or producing debugging builds. | ||||
Some examples: | Some examples: | ||||
```bash | ```bash | ||||
# Enable both the address sanitizer and the undefined behavior sanitizer | # Enable both the address sanitizer and the undefined behavior sanitizer | ||||
./configure --with-sanitizers=address,undefined | cmake -GNinja .. -DENABLE_SANITIZERS="address;undefined" | ||||
# Enable the thread sanitizer | # Enable the thread sanitizer | ||||
./configure --with-sanitizers=thread | cmake -GNinja .. -DENABLE_SANITIZERS=thread | ||||
``` | ``` | ||||
If you are compiling with GCC you will typically need to install corresponding | If you are compiling with GCC you will typically need to install corresponding | ||||
"san" libraries to actually compile with these flags, e.g. libasan for the | "san" libraries to actually compile with these flags, e.g. libasan for the | ||||
address sanitizer, libtsan for the thread sanitizer, and libubsan for the | address sanitizer, libtsan for the thread sanitizer, and libubsan for the | ||||
undefined sanitizer. If you are missing required libraries, the configure script | undefined sanitizer. If you are missing required libraries, the cmake script | ||||
will fail with a linker error when testing the sanitizer flags. | will fail with an error when testing the sanitizer flags. | ||||
Note that the sanitizers will give a better output if they are run with a Debug | |||||
build configuration. | |||||
The test suite should pass cleanly with the `thread` and `undefined` sanitizers, | There are a number of known problems for which suppressions files are provided | ||||
but there are a number of known problems when using the `address` sanitizer. The | under `test/sanitizer_suppressions`. These files are intended to be used with | ||||
address sanitizer is known to fail in | the `suppressions` option from the sanitizers. | ||||
The address sanitizer is known to fail in | |||||
[sha256_sse4::Transform](/src/crypto/sha256_sse4.cpp) which makes it unusable | [sha256_sse4::Transform](/src/crypto/sha256_sse4.cpp) which makes it unusable | ||||
unless you also use `--disable-asm` when running configure. We would like to fix | unless you also use `-DCRYPTO_USE_ASM=OFF` when running cmake. | ||||
sanitizer issues, so please send pull requests if you can fix any errors found | We would like to fix sanitizer issues, so please send pull requests if you can | ||||
by the address sanitizer (or any other sanitizer). | fix any errors found by the address sanitizer (or any other sanitizer). | ||||
Not all sanitizer options can be enabled at the same time, e.g. trying to build | Not all sanitizer options can be enabled at the same time, e.g. trying to build | ||||
with `--with-sanitizers=address,thread` will fail in the configure script as | with `-DENABLE_SANITIZERS=="address;thread" will fail in the cmake script as | ||||
these sanitizers are mutually incompatible. Refer to your compiler manual to | these sanitizers are mutually incompatible. Refer to your compiler manual to | ||||
learn more about these options and which sanitizers are supported by your | learn more about these options and which sanitizers are supported by your | ||||
compiler. | compiler. | ||||
Examples: | |||||
Build and run the test suite with the address sanitizer enabled: | |||||
```bash | |||||
mkdir build_asan | |||||
cd build_asan | |||||
export ASAN_OPTIONS="malloc_context_size=0" | |||||
export LSAN_OPTIONS="suppressions=${PWD}/../test/sanitizer_suppressions/lsan" | |||||
cmake -GNinja .. \ | |||||
-DCMAKE_BUILD_TYPE=Debug \ | |||||
-DENABLE_SANITIZERS=address \ | |||||
-DCRYPTO_USE_ASM=OFF | |||||
ninja check check-functional | |||||
``` | |||||
Build and run the test suite with the thread sanitizer enabled (it can take a | |||||
very long time to complete): | |||||
```bash | |||||
mkdir build_tsan | |||||
cd build_tsan | |||||
export TSAN_OPTIONS="suppressions=${PWD}/../test/sanitizer_suppressions/tsan" | |||||
cmake -GNinja .. \ | |||||
-DCMAKE_BUILD_TYPE=Debug \ | |||||
-DENABLE_SANITIZERS=thread | |||||
ninja check check-functional | |||||
``` | |||||
Build and run the test suite with the undefined sanitizer enabled: | |||||
```bash | |||||
mkdir build_ubsan | |||||
cd build_ubsan | |||||
export UBSAN_OPTIONS="suppressions=${PWD}/../test/sanitizer_suppressions/ubsan" | |||||
cmake -GNinja .. \ | |||||
-DCMAKE_BUILD_TYPE=Debug \ | |||||
-DENABLE_SANITIZERS=undefined | |||||
ninja check check-functional | |||||
``` | |||||
Additional resources: | Additional resources: | ||||
* [AddressSanitizer](https://clang.llvm.org/docs/AddressSanitizer.html) | * [AddressSanitizer](https://clang.llvm.org/docs/AddressSanitizer.html) | ||||
* [LeakSanitizer](https://clang.llvm.org/docs/LeakSanitizer.html) | * [LeakSanitizer](https://clang.llvm.org/docs/LeakSanitizer.html) | ||||
* [MemorySanitizer](https://clang.llvm.org/docs/MemorySanitizer.html) | * [MemorySanitizer](https://clang.llvm.org/docs/MemorySanitizer.html) | ||||
* [ThreadSanitizer](https://clang.llvm.org/docs/ThreadSanitizer.html) | * [ThreadSanitizer](https://clang.llvm.org/docs/ThreadSanitizer.html) | ||||
* [UndefinedBehaviorSanitizer](https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html) | * [UndefinedBehaviorSanitizer](https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html) | ||||
* [GCC Instrumentation Options](https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html) | * [GCC Instrumentation Options](https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html) | ||||
* [Google Sanitizers Wiki](https://github.com/google/sanitizers/wiki) | * [Google Sanitizers Wiki](https://github.com/google/sanitizers/wiki) | ||||
* [Issue #12691: Enable -fsanitize flags in Travis](https://github.com/bitcoin/bitcoin/issues/12691) | * [Issue #12691: Enable -fsanitize flags in Travis](https://github.com/bitcoin/bitcoin/issues/12691) | ||||
Locking/mutex usage notes | Locking/mutex usage notes | ||||
------------------------- | ------------------------- | ||||
The code is multi-threaded, and uses mutexes and the | The code is multi-threaded, and uses mutexes and the | ||||
`LOCK` and `TRY_LOCK` macros to protect data structures. | `LOCK` and `TRY_LOCK` macros to protect data structures. | ||||
Deadlocks due to inconsistent lock ordering (thread 1 locks `cs_main` and then | Deadlocks due to inconsistent lock ordering (thread 1 locks `cs_main` and then | ||||
`cs_wallet`, while thread 2 locks them in the opposite order: result, deadlock | `cs_wallet`, while thread 2 locks them in the opposite order: result, deadlock | ||||
as each waits for the other to release its lock) are a problem. Compile with | as each waits for the other to release its lock) are a problem. Compile with | ||||
`-DDEBUG_LOCKORDER` (or use `--enable-debug`) to get lock order inconsistencies | `-DDEBUG_LOCKORDER` (or use `-DCMAKE_BUILD_TYPE=Debug`) to get lock order | ||||
reported in the debug.log file. | inconsistencies reported in the debug.log file. | ||||
Re-architecting the core code so there are better-defined interfaces | Re-architecting the core code so there are better-defined interfaces | ||||
between the various components is a goal, with any necessary locking | between the various components is a goal, with any necessary locking | ||||
done by the components (e.g. see the self-contained `CBasicKeyStore` class | done by the components (e.g. see the self-contained `CBasicKeyStore` class | ||||
and its `cs_KeyStore` lock for example). | and its `cs_KeyStore` lock for example). | ||||
Threads | Threads | ||||
------- | ------- | ||||
Show All 23 Lines | |||||
- Shutdown : Does an orderly shutdown of everything. | - Shutdown : Does an orderly shutdown of everything. | ||||
Ignoring IDE/editor files | Ignoring IDE/editor files | ||||
-------------------------- | -------------------------- | ||||
In closed-source environments in which everyone uses the same IDE it is common | In closed-source environments in which everyone uses the same IDE it is common | ||||
to add temporary files it produces to the project-wide `.gitignore` file. | to add temporary files it produces to the project-wide `.gitignore` file. | ||||
However, in open source software such as Bitcoin Core, where everyone uses | However, in open source software such as Bitcoin ABC, where everyone uses | ||||
their own editors/IDE/tools, it is less common. Only you know what files your | their own editors/IDE/tools, it is less common. Only you know what files your | ||||
editor produces and this may change from version to version. The canonical way | editor produces and this may change from version to version. The canonical way | ||||
to do this is thus to create your local gitignore. Add this to `~/.gitconfig`: | to do this is thus to create your local gitignore. Add this to `~/.gitconfig`: | ||||
``` | ``` | ||||
[core] | [core] | ||||
excludesfile = /home/.../.gitignore_global | excludesfile = /home/.../.gitignore_global | ||||
``` | ``` | ||||
(alternatively, type the command `git config --global core.excludesfile ~/.gitignore_global` | (alternatively, type the command `git config --global core.excludesfile ~/.gitignore_global` | ||||
on a terminal) | on a terminal) | ||||
Then put your favourite tool's temporary filenames in that file, e.g. | Then put your favorite tool's temporary filenames in that file, e.g. | ||||
``` | ``` | ||||
# NetBeans | # NetBeans | ||||
nbproject/ | nbproject/ | ||||
``` | ``` | ||||
Another option is to create a per-repository excludes file `.git/info/exclude`. | Another option is to create a per-repository excludes file `.git/info/exclude`. | ||||
These are not committed but apply only to one repository. | These are not committed but apply only to one repository. | ||||
If a set of tools is used by the build system or scripts the repository (for | If a set of tools is used by the build system or scripts the repository (for | ||||
example, lcov) it is perfectly acceptable to add its files to `.gitignore` | example, lcov) it is perfectly acceptable to add its files to `.gitignore` | ||||
and commit them. | and commit them. | ||||
Development guidelines | Development guidelines | ||||
============================ | ============================ | ||||
A few non-style-related recommendations for developers, as well as points to | A few non-style-related recommendations for developers, as well as points to | ||||
pay attention to for reviewers of Bitcoin Core code. | pay attention to for reviewers of Bitcoin ABC code. | ||||
Wallet | Wallet | ||||
------- | ------- | ||||
- Make sure that no crashes happen with run-time option `-disablewallet`. | - Make sure that no crashes happen with run-time option `-disablewallet`. | ||||
- *Rationale*: In RPC code that conditionally uses the wallet (such as | - *Rationale*: In RPC code that conditionally uses the wallet (such as | ||||
`validateaddress`) it is easy to forget that global pointer `pwalletMain` | `validateaddress`) it is easy to forget that global pointer `pwalletMain` | ||||
▲ Show 20 Lines • Show All 123 Lines • ▼ Show 20 Lines | |||||
Please name variables so that their names do not shadow variables defined in the source code. | Please name variables so that their names do not shadow variables defined in the source code. | ||||
Threads and synchronization | Threads and synchronization | ||||
---------------------------- | ---------------------------- | ||||
- Build and run tests with `-DDEBUG_LOCKORDER` to verify that no potential | - Build and run tests with `-DDEBUG_LOCKORDER` to verify that no potential | ||||
deadlocks are introduced. As of 0.12, this is defined by default when | deadlocks are introduced. As of 0.12, this is defined by default when | ||||
configuring with `--enable-debug` | configuring with `-DCMAKE_BUILD_TYPE=Debug` | ||||
- When using `LOCK`/`TRY_LOCK` be aware that the lock exists in the context of | - When using `LOCK`/`TRY_LOCK` be aware that the lock exists in the context of | ||||
the current scope, so surround the statement and the code that needs the lock | the current scope, so surround the statement and the code that needs the lock | ||||
with braces | with braces | ||||
OK: | OK: | ||||
```c++ | ```c++ | ||||
▲ Show 20 Lines • Show All 188 Lines • ▼ Show 20 Lines | |||||
It is possible for LevelDB changes to inadvertently change consensus | It is possible for LevelDB changes to inadvertently change consensus | ||||
compatibility between nodes. This happened in Bitcoin 0.8 (when LevelDB was | compatibility between nodes. This happened in Bitcoin 0.8 (when LevelDB was | ||||
first introduced). When upgrading LevelDB you should review the upstream changes | first introduced). When upgrading LevelDB you should review the upstream changes | ||||
to check for issues affecting consensus compatibility. | to check for issues affecting consensus compatibility. | ||||
For example, if LevelDB had a bug that accidentally prevented a key from being | For example, if LevelDB had a bug that accidentally prevented a key from being | ||||
returned in an edge case, and that bug was fixed upstream, the bug "fix" would | returned in an edge case, and that bug was fixed upstream, the bug "fix" would | ||||
be an incompatible consensus change. In this situation the correct behavior | be an incompatible consensus change. In this situation the correct behavior | ||||
would be to revert the upstream fix before applying the updates to Bitcoin's | would be to revert the upstream fix before applying the updates to Bitcoin ABC's | ||||
copy of LevelDB. In general you should be wary of any upstream changes affecting | copy of LevelDB. In general you should be wary of any upstream changes affecting | ||||
what data is returned from LevelDB queries. | what data is returned from LevelDB queries. | ||||
Git and GitHub tips | Git and GitHub tips | ||||
--------------------- | --------------------- | ||||
- Github is not typically the source of truth for pull requests. See CONTRIBUTING.md for instructions | - Github is not typically the source of truth for pull requests. See CONTRIBUTING.md for instructions | ||||
on setting up your repo correctly. | on setting up your repo correctly. | ||||
▲ Show 20 Lines • Show All 148 Lines • Show Last 20 Lines |