[Chronik] Plugins: Load plugins based on plugins.toml in datadir
Summary:
Allow the user to specify a plugins.toml file in the datadir, which will load Python plugins at runtime.
Plugins are supposed to be placed in a plugins directory and then loaded if specified in the plugins.toml, however, any module that can be found on the PYTHONPATH can be loaded this way. This allows users to both install plugins globally (e.g. using pip install), but also have datadir-specific plugins.
This mean normal python import rules apply, i.e. if the plugin is just one file, it's fine to have a myplugin.py in the plugins folder, or if more complex handling is required, a directory with a __init__.py works too, as well as .egg files, and others.
The approach using the plugins.toml file has been picked for these reasons:
- The alternative would be automatic discovery, e.g. using the approaches listed here. However, it would mean that plugins cannot be disabled easily, other than deleting/moving the plugin; also, if users install plugins globally (e.g. using pip install), they will be shared with all instances, which might not be desirable.
- It allows us to configure plugins individually (using TOML), e.g. to enable/disable a certain feature of a plugin, or to specify a token ID of a plugin, etc.
- It allows having some plugins enabled for testnet/regtest but not for mainnet.
A plugin has to define a plugin class, the naming convention is to PascalCase the module name and append Plugin (e.g. my_demo -> MyDemoPlugin), and this can be overridden by setting the "class" config of the plugin. Currently, the init function of a plugin has to take a config object; for now this will always be the empty dict but in the future we can provide config params from the plugins.toml.
The specification for plugins can be found here.
Test Plan: ./test/functional/test_runner.py chronik_plugins_setup
Reviewers: Fabien, #bitcoin_abc
Reviewed By: Fabien, #bitcoin_abc
Differential Revision: https://reviews.bitcoinabc.org/D16219