Differential D9809 Diff 29247 doc/functional-tests.md

Changeset View

Standalone View

doc/functional-tests.md

	# Functional tests			# Functional tests

	The [/test/](/test/) directory contains integration tests that test bitcoind			The [/test/](/test/) directory contains integration tests that test ecashd
	and its utilities in their entirety. It does not contain unit tests, which			and its utilities in their entirety. It does not contain unit tests, which
	can be found in [/src/test](/src/test), [/src/wallet/test](/src/wallet/test),			can be found in [/src/test](/src/test), [/src/wallet/test](/src/wallet/test),
	etc.			etc.

	There are currently two sets of tests in the [/test/](/test/) directory:			There are currently two sets of tests in the [/test/](/test/) directory:

	- [functional](/test/functional) which test the functionality of			- [functional](/test/functional) which test the functionality of
	bitcoind and bitcoin-qt by interacting with them through the RPC and P2P			ecashd and ecash-qt by interacting with them through the RPC and P2P
	interfaces.			interfaces.
	- [util](/test/util) which tests the bitcoin utilities, currently only			- [util](/test/util) which tests the ecash utilities, currently only
	bitcoin-tx.			ecash-tx.

	The util tests are run as part of `make check` target. The functional			The util tests are run as part of `make check` target. The functional
	tests are run by the Teamcity continuous build process whenever a diff is			tests are run by the Teamcity continuous build process whenever a diff is
	created or updated on Phabricator. Both sets of tests can also be run locally.			created or updated on Phabricator. Both sets of tests can also be run locally.

	# Running functional tests locally			# Running functional tests locally

	Build for your system first. Be sure to enable wallet, utils and daemon when			Build for your system first. Be sure to enable wallet, utils and daemon when
	▲ Show 20 Lines • Show All 44 Lines • ▼ Show 20 Lines

	The individual tests and the test_runner harness have many command-line			The individual tests and the test_runner harness have many command-line
	options. Run `test_runner.py -h` to see them all.			options. Run `test_runner.py -h` to see them all.

	#### Troubleshooting and debugging test failures			#### Troubleshooting and debugging test failures

	##### Resource contention			##### Resource contention

	The P2P and RPC ports used by the bitcoind nodes-under-test are chosen to make			The P2P and RPC ports used by the ecashd nodes-under-test are chosen to make
	conflicts with other processes unlikely. However, if there is another bitcoind			conflicts with other processes unlikely. However, if there is another ecashd
	process running on the system (perhaps from a previous test which hasn't successfully			process running on the system (perhaps from a previous test which hasn't successfully
	killed all its bitcoind nodes), then there may be a port conflict which will			killed all its ecashd nodes), then there may be a port conflict which will
	cause the test to fail. It is recommended that you run the tests on a system			cause the test to fail. It is recommended that you run the tests on a system
	where no other bitcoind processes are running.			where no other ecashd processes are running.

	On linux, the test_framework will warn if there is another			On linux, the test_framework will warn if there is another
	bitcoind process running when the tests are started.			ecashd process running when the tests are started.

	If there are zombie bitcoind processes after test failure, you can kill them			If there are zombie ecashd processes after test failure, you can kill them
	by running the following commands. **Note that these commands will kill all			by running the following commands. **Note that these commands will kill all
	bitcoind processes running on the system, so should not be used if any non-test			ecashd processes running on the system, so should not be used if any non-test
	bitcoind processes are being run.**			ecashd processes are being run.**

	```bash			```bash
	killall bitcoind			killall ecashd
	```			```

	or			or

	```bash			```bash
	pkill -9 bitcoind			pkill -9 ecashd
	```			```

	##### Data directory cache			##### Data directory cache

	A pre-mined blockchain with 200 blocks is generated the first time a			A pre-mined blockchain with 200 blocks is generated the first time a
	functional test is run and is stored in test/cache. This speeds up			functional test is run and is stored in test/cache. This speeds up
	test startup times since new blockchains don't need to be generated for			test startup times since new blockchains don't need to be generated for
	each test. However, the cache may get into a bad state, in which case			each test. However, the cache may get into a bad state, in which case
	tests will fail. If this happens, remove the cache directory (and make			tests will fail. If this happens, remove the cache directory (and make
	sure bitcoind processes are stopped as above):			sure ecashd processes are stopped as above):

	```bash			```bash
	rm -rf cache			rm -rf cache
	killall bitcoind			killall ecashd
	```			```

	##### Test logging			##### Test logging

	The tests contain logging at different levels (debug, info, warning, etc). By			The tests contain logging at different levels (debug, info, warning, etc). By
	default:			default:

	- When run through the test_runner harness, all logs are written to			- When run through the test_runner harness, all logs are written to
	`test_framework.log` and no logs are output to the console.			`test_framework.log` and no logs are output to the console.
	- When run directly, all logs are written to `test_framework.log` and INFO			- When run directly, all logs are written to `test_framework.log` and INFO
	level and above are output to the console.			level and above are output to the console.
	- When run on Travis, no logs are output to the console. However, if a test			- When run on Travis, no logs are output to the console. However, if a test
	fails, the `test_framework.log` and bitcoind `debug.log`s will all be dumped			fails, the `test_framework.log` and ecashd `debug.log`s will all be dumped
	to the console to help troubleshooting.			to the console to help troubleshooting.

	To change the level of logs output to the console, use the `-l` command line			To change the level of logs output to the console, use the `-l` command line
	argument.			argument.

	`test_framework.log` and bitcoind `debug.log`s can be combined into a single			`test_framework.log` and ecashd `debug.log`s can be combined into a single
	aggregate log by running the `combine_logs.py` script. The output can be plain			aggregate log by running the `combine_logs.py` script. The output can be plain
	text, colorized text or html. For example:			text, colorized text or html. For example:

	```			```
	combine_logs.py -c <test data directory> \| less -r			combine_logs.py -c <test data directory> \| less -r
	```			```

	will pipe the colorized logs from the test into less.			will pipe the colorized logs from the test into less.
	Show All 10 Lines

	A python debugger can be attached to tests at any point. Just add the line:			A python debugger can be attached to tests at any point. Just add the line:

	```py			```py
	import pdb; pdb.set_trace()			import pdb; pdb.set_trace()
	```			```

	anywhere in the test. You will then be able to inspect variables, as well as			anywhere in the test. You will then be able to inspect variables, as well as
	call methods that interact with the bitcoind nodes-under-test.			call methods that interact with the ecashd nodes-under-test.

	If further introspection of the bitcoind instances themselves becomes			If further introspection of the ecashd instances themselves becomes
	necessary, this can be accomplished by first setting a pdb breakpoint			necessary, this can be accomplished by first setting a pdb breakpoint
	at an appropriate location, running the test to that point, then using			at an appropriate location, running the test to that point, then using
	`gdb` (or `lldb` on macOS) to attach to the process and debug.			`gdb` (or `lldb` on macOS) to attach to the process and debug.

	For instance, to attach to `self.node[1]` during a run you can get			For instance, to attach to `self.node[1]` during a run you can get
	the pid of the node within `pdb`.			the pid of the node within `pdb`.

	```			```
	(pdb) self.node[1].process.pid			(pdb) self.node[1].process.pid
	```			```

	Alternatively, you can find the pid by inspecting the temp folder for the specific test			Alternatively, you can find the pid by inspecting the temp folder for the specific test
	you are running. The path to that folder is printed at the beginning of every			you are running. The path to that folder is printed at the beginning of every
	test run:			test run:

	```bash			```bash
	2017-06-27 14:13:56.686000 TestFramework (INFO): Initializing test directory /tmp/user/1000/testo9vsdjo3			2017-06-27 14:13:56.686000 TestFramework (INFO): Initializing test directory /tmp/user/1000/testo9vsdjo3
	```			```

	Use the path to find the pid file in the temp folder:			Use the path to find the pid file in the temp folder:

	```bash			```bash
	cat /tmp/user/1000/testo9vsdjo3/node1/regtest/bitcoind.pid			cat /tmp/user/1000/testo9vsdjo3/node1/regtest/ecashd.pid
	```			```

	Then you can use the pid to start `gdb`:			Then you can use the pid to start `gdb`:

	```bash			```bash
	gdb /home/example/bitcoind <pid>			gdb /home/example/ecashd <pid>
	```			```

	Note: gdb attach step may require `sudo`. To get rid of this, you can run:			Note: gdb attach step may require `sudo`. To get rid of this, you can run:

	```bash			```bash
	echo 0 \| sudo tee /proc/sys/kernel/yama/ptrace_scope			echo 0 \| sudo tee /proc/sys/kernel/yama/ptrace_scope
	```			```

	▲ Show 20 Lines • Show All 128 Lines • ▼ Show 20 Lines

	#### Using the P2P interface			#### Using the P2P interface

	- `messages.py` contains all the definitions for objects that pass			- `messages.py` contains all the definitions for objects that pass
	over the network (`CBlock`, `CTransaction`, etc, along with the network-level			over the network (`CBlock`, `CTransaction`, etc, along with the network-level
	wrappers for them, `msg_block`, `msg_tx`, etc).			wrappers for them, `msg_block`, `msg_tx`, etc).

	- P2P tests have two threads. One thread handles all network communication			- P2P tests have two threads. One thread handles all network communication
	with the bitcoind(s) being tested in a callback-based event loop; the other			with the ecashd(s) being tested in a callback-based event loop; the other
	implements the test logic.			implements the test logic.

	- `P2PConnection` is the class used to connect to a bitcoind. `P2PInterface`			- `P2PConnection` is the class used to connect to a ecashd. `P2PInterface`
	contains the higher level logic for processing P2P payloads and connecting to			contains the higher level logic for processing P2P payloads and connecting to
	the Bitcoin Core node application logic. For custom behaviour, subclass the			the Bitcoin Core node application logic. For custom behaviour, subclass the
	P2PInterface object and override the callback methods.			P2PInterface object and override the callback methods.

	- Can be used to write tests where specific P2P protocol behavior is tested.			- Can be used to write tests where specific P2P protocol behavior is tested.
	Examples tests are `p2p-acceptblock.py`, `p2p-compactblocks.py`.			Examples tests are `p2p-acceptblock.py`, `p2p-compactblocks.py`.

	#### Prototyping tests			#### Prototyping tests
	Show All 13 Lines

	#### [test_framework/test_framework.py](/test/functional/test_framework/test_framework.py)			#### [test_framework/test_framework.py](/test/functional/test_framework/test_framework.py)
	Base class for functional tests.			Base class for functional tests.

	#### [test_framework/util.py](/test/functional/test_framework/util.py)			#### [test_framework/util.py](/test/functional/test_framework/util.py)
	Generally useful functions.			Generally useful functions.

	#### [test_framework/p2p.py](/test/functional/test_framework/p2p.py)			#### [test_framework/p2p.py](/test/functional/test_framework/p2p.py)
	Test objects for interacting with a bitcoind node over the p2p interface.			Test objects for interacting with a ecashd node over the p2p interface.

	#### [test_framework/script.py](/test/functional/test_framework/script.py)			#### [test_framework/script.py](/test/functional/test_framework/script.py)
	Utilities for manipulating transaction scripts (originally from python-bitcoinlib)			Utilities for manipulating transaction scripts (originally from python-bitcoinlib)

	#### [test_framework/key.py](/test/functional/test_framework/key.py)			#### [test_framework/key.py](/test/functional/test_framework/key.py)
	Test-only secp256k1 elliptic curve implementation			Test-only secp256k1 elliptic curve implementation

	#### [test_framework/blocktools.py](/test/functional/test_framework/blocktools.py)			#### [test_framework/blocktools.py](/test/functional/test_framework/blocktools.py)
	Helper functions for creating blocks and transactions.			Helper functions for creating blocks and transactions.