Paths

Table of Contentst

Differential D14278

[electrum] fix the temp datadir removal for functional tests
ClosedPublic
Actions

Authored by PiRK on Jul 18 2023, 12:24.

Details

Reviewers

Fabien

Group Reviewers

Restricted Project

Commits

rABC2fa4ac16d543: [electrum] fix the temp datadir removal for functional tests

Summary

The temporary data dir is not necessarily located in /tmp, it depends on the OS and access permissions.
Checking for _datadir.startswith("/tmp") caused the tests to fail on systems with the tmp dir in /var/tmp.

Also move the filesystem operations out of start/stop_ec_daemon, so that it becomes more clear that the function creating the directory is responsible for it's deletion, and that no intermediate operation is done to mutate the path (clarifies that we don't delete something else than the tmp directory).

Also remove some assertions that _datadir is not None after _datadir = tempfile.mkdtemp(), because mkdtemp never returns None, it errors on failure. Keep an assertion in start_ec_daemon to make it clear that this need to be called after make_electrum_data_dir

Test Plan

pytest electrum/electrumabc/tests/regtest

Diff Detail

Repository

rABC Bitcoin ABC

Branch

regtest

Lint

Lint Passed

Unit

No Test Coverage

Build Status

Buildable 24529
Build 48657: Build Diff	electrum-tests
Build 48656: arc lint + arc unit

Event Timeline

PiRK created this revision.Jul 18 2023, 12:24

Herald added a reviewer: Restricted Project. · View Herald TranscriptJul 18 2023, 12:24

PiRK requested review of this revision.Jul 18 2023, 12:24

Harbormaster completed remote builds in B24529: Diff 41491.Jul 18 2023, 12:26

Fabien accepted this revision.Jul 18 2023, 12:57

This revision is now accepted and ready to land.Jul 18 2023, 12:57

Closed by commit rABC2fa4ac16d543: [electrum] fix the temp datadir removal for functional tests (authored by PiRK). · Explain WhyJul 18 2023, 13:03

This revision was automatically updated to reflect the committed changes.

PiRK added a commit: rABC2fa4ac16d543: [electrum] fix the temp datadir removal for functional tests.

@bot electrum-functional-tests

Build Bitcoin ABC Diffs / Diff Testing (electrum-functional-tests) failed.

Failed tests logs:

====== electrumabc.tests.regtest.test_rpc_misc.test_getunusedaddress ======
test setup failed
docker_compose_command = 'docker-compose'
docker_compose_file = '/work/electrum/electrumabc/tests/regtest/docker-compose.yml'
docker_compose_project_name = 'pytest19', docker_setup = 'up --build -d'
docker_cleanup = 'down -v'

    @pytest.fixture(scope="session")
    def docker_services(
        docker_compose_command,
        docker_compose_file,
        docker_compose_project_name,
        docker_setup,
        docker_cleanup,
    ):
        """Start all services from a docker compose file (`docker-compose up`).
        After test are finished, shutdown all services (`docker-compose down`)."""
    
>       with get_docker_services(
            docker_compose_command,
            docker_compose_file,
            docker_compose_project_name,
            docker_setup,
            docker_cleanup,
        ) as docker_service:

/usr/local/lib/python3.9/dist-packages/pytest_docker/plugin.py:221: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3.9/contextlib.py:117: in __enter__
    return next(self.gen)
/usr/local/lib/python3.9/dist-packages/pytest_docker/plugin.py:199: in get_docker_services
    docker_compose.execute(docker_setup)
/usr/local/lib/python3.9/dist-packages/pytest_docker/plugin.py:129: in execute
    return execute(command)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

command = 'docker-compose -f "/work/electrum/electrumabc/tests/regtest/docker-compose.yml" -p "pytest19" up --build -d'
success_codes = (0,)

    def execute(command, success_codes=(0,)):
        """Run a shell command."""
        try:
            output = subprocess.check_output(command, stderr=subprocess.STDOUT, shell=True)
            status = 0
        except subprocess.CalledProcessError as error:
            output = error.output or b""
            status = error.returncode
            command = error.cmd
    
        if status not in success_codes:
>           raise Exception(
                'Command {} returned {}: """{}""".'.format(
                    command, status, output.decode("utf-8")
                )
            )
E           Exception: Command docker-compose -f "/work/electrum/electrumabc/tests/regtest/docker-compose.yml" -p "pytest19" up --build -d returned 127: """/bin/sh: 1: docker-compose: not found
E           """.

/usr/local/lib/python3.9/dist-packages/pytest_docker/plugin.py:24: Exception
====== electrumabc.tests.regtest.test_rpc_misc.test_getservers ======
test setup failed
docker_compose_command = 'docker-compose'
docker_compose_file = '/work/electrum/electrumabc/tests/regtest/docker-compose.yml'
docker_compose_project_name = 'pytest19', docker_setup = 'up --build -d'
docker_cleanup = 'down -v'

    @pytest.fixture(scope="session")
    def docker_services(
        docker_compose_command,
        docker_compose_file,
        docker_compose_project_name,
        docker_setup,
        docker_cleanup,
    ):
        """Start all services from a docker compose file (`docker-compose up`).
        After test are finished, shutdown all services (`docker-compose down`)."""
    
>       with get_docker_services(
            docker_compose_command,
            docker_compose_file,
            docker_compose_project_name,
            docker_setup,
            docker_cleanup,
        ) as docker_service:

/usr/local/lib/python3.9/dist-packages/pytest_docker/plugin.py:221: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3.9/contextlib.py:117: in __enter__
    return next(self.gen)
/usr/local/lib/python3.9/dist-packages/pytest_docker/plugin.py:199: in get_docker_services
    docker_compose.execute(docker_setup)
/usr/local/lib/python3.9/dist-packages/pytest_docker/plugin.py:129: in execute
    return execute(command)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

command = 'docker-compose -f "/work/electrum/electrumabc/tests/regtest/docker-compose.yml" -p "pytest19" up --build -d'
success_codes = (0,)

    def execute(command, success_codes=(0,)):
        """Run a shell command."""
        try:
            output = subprocess.check_output(command, stderr=subprocess.STDOUT, shell=True)
            status = 0
        except subprocess.CalledProcessError as error:
            output = error.output or b""
            status = error.returncode
            command = error.cmd
    
        if status not in success_codes:
>           raise Exception(
                'Command {} returned {}: """{}""".'.format(
                    command, status, output.decode("utf-8")
                )
            )
E           Exception: Command docker-compose -f "/work/electrum/electrumabc/tests/regtest/docker-compose.yml" -p "pytest19" up --build -d returned 127: """/bin/sh: 1: docker-compose: not found
E           """.

/usr/local/lib/python3.9/dist-packages/pytest_docker/plugin.py:24: Exception
====== electrumabc.tests.regtest.test_rpc_misc.test_balance ======
test setup failed
docker_compose_command = 'docker-compose'
docker_compose_file = '/work/electrum/electrumabc/tests/regtest/docker-compose.yml'
docker_compose_project_name = 'pytest19', docker_setup = 'up --build -d'
docker_cleanup = 'down -v'

    @pytest.fixture(scope="session")
    def docker_services(
        docker_compose_command,
        docker_compose_file,
        docker_compose_project_name,
        docker_setup,
        docker_cleanup,
    ):
        """Start all services from a docker compose file (`docker-compose up`).
        After test are finished, shutdown all services (`docker-compose down`)."""
    
>       with get_docker_services(
            docker_compose_command,
            docker_compose_file,
            docker_compose_project_name,
            docker_setup,
            docker_cleanup,
        ) as docker_service:

/usr/local/lib/python3.9/dist-packages/pytest_docker/plugin.py:221: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3.9/contextlib.py:117: in __enter__
    return next(self.gen)
/usr/local/lib/python3.9/dist-packages/pytest_docker/plugin.py:199: in get_docker_services
    docker_compose.execute(docker_setup)
/usr/local/lib/python3.9/dist-packages/pytest_docker/plugin.py:129: in execute
    return execute(command)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

command = 'docker-compose -f "/work/electrum/electrumabc/tests/regtest/docker-compose.yml" -p "pytest19" up --build -d'
success_codes = (0,)

    def execute(command, success_codes=(0,)):
        """Run a shell command."""
        try:
            output = subprocess.check_output(command, stderr=subprocess.STDOUT, shell=True)
            status = 0
        except subprocess.CalledProcessError as error:
            output = error.output or b""
            status = error.returncode
            command = error.cmd
    
        if status not in success_codes:
>           raise Exception(
                'Command {} returned {}: """{}""".'.format(
                    command, status, output.decode("utf-8")
                )
            )
E           Exception: Command docker-compose -f "/work/electrum/electrumabc/tests/regtest/docker-compose.yml" -p "pytest19" up --build -d returned 127: """/bin/sh: 1: docker-compose: not found
E           """.

/usr/local/lib/python3.9/dist-packages/pytest_docker/plugin.py:24: Exception
====== electrumabc.tests.regtest.test_rpc_payment_request.test_addrequest ======
test setup failed
docker_compose_command = 'docker-compose'
docker_compose_file = '/work/electrum/electrumabc/tests/regtest/docker-compose.yml'
docker_compose_project_name = 'pytest19', docker_setup = 'up --build -d'
docker_cleanup = 'down -v'

    @pytest.fixture(scope="session")
    def docker_services(
        docker_compose_command,
        docker_compose_file,
        docker_compose_project_name,
        docker_setup,
        docker_cleanup,
    ):
        """Start all services from a docker compose file (`docker-compose up`).
        After test are finished, shutdown all services (`docker-compose down`)."""
    
>       with get_docker_services(
            docker_compose_command,
            docker_compose_file,
            docker_compose_project_name,
            docker_setup,
            docker_cleanup,
        ) as docker_service:

/usr/local/lib/python3.9/dist-packages/pytest_docker/plugin.py:221: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3.9/contextlib.py:117: in __enter__
    return next(self.gen)
/usr/local/lib/python3.9/dist-packages/pytest_docker/plugin.py:199: in get_docker_services
    docker_compose.execute(docker_setup)
/usr/local/lib/python3.9/dist-packages/pytest_docker/plugin.py:129: in execute
    return execute(command)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

command = 'docker-compose -f "/work/electrum/electrumabc/tests/regtest/docker-compose.yml" -p "pytest19" up --build -d'
success_codes = (0,)

    def execute(command, success_codes=(0,)):
        """Run a shell command."""
        try:
            output = subprocess.check_output(command, stderr=subprocess.STDOUT, shell=True)
            status = 0
        except subprocess.CalledProcessError as error:
            output = error.output or b""
            status = error.returncode
            command = error.cmd
    
        if status not in success_codes:
>           raise Exception(
                'Command {} returned {}: """{}""".'.format(
                    command, status, output.decode("utf-8")
                )
            )
E           Exception: Command docker-compose -f "/work/electrum/electrumabc/tests/regtest/docker-compose.yml" -p "pytest19" up --build -d returned 127: """/bin/sh: 1: docker-compose: not found
E           """.

/usr/local/lib/python3.9/dist-packages/pytest_docker/plugin.py:24: Exception

Each failure log is accessible here:
electrumabc.tests.regtest.test_rpc_misc.test_getunusedaddress
electrumabc.tests.regtest.test_rpc_misc.test_getservers
electrumabc.tests.regtest.test_rpc_misc.test_balance
electrumabc.tests.regtest.test_rpc_payment_request.test_addrequest

Revision Contents
Changeset List

Path

Size

electrum/

electrumabc/

tests/

regtest/

util.py

50 lines

Diff 41491

View Options

electrum/electrumabc/tests/regtest/util.py

[electrum] fix the temp datadir removal for functional testsClosedPublicActions