Page MenuHomePhabricator

Add symlink to most recent functional test failure directory
ClosedPublic

Authored by roqqit on Tue, Dec 10, 00:50.

Details

Reviewers
Fabien
PiRK
Group Reviewers
Restricted Owners Package(Owns No Changed Paths)
Restricted Project
Commits
rABCe9d7a6220c01: Add symlink to most recent functional test failure directory
Summary

When iterating on a change or writing a new test, a common dev process goes like this:

  1. Make a change.
  2. Run ./test/functional/test_runner.py mytest It fails.
  3. Scroll up in the terminal and copy the test data directory.
  4. Paste it back into the command line for inspection.

This process is often repeated until the test is green. The smaller the terminal, the more annoying it is.

This patch introduces a very simple solution: a symlink to the last failing test's data directory. This makes it still useful if you are running multiple tests, but the best use case is when you need to repeat the process multiple times to nail down a fix for a particular test.

Test Plan

Introduce assert() into some test.

./test/functional/test_runner.py thattest.py
ls -al lastfailure
./test/functional/combine_logs.py lastfailure
./test/functional/test_runner.py thattest.py
head lastfailure/test_framework.log  # verify the symlink updated by looking at the test directory name

Diff Detail

Repository
rABC Bitcoin ABC
Branch
lastfailurepath
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 31596
Build 62689: Build Diffbuild-without-wallet · build-clang-tidy · build-clang · build-debug · build-diff
Build 62688: arc lint + arc unit

Event Timeline

Owners added a reviewer: Restricted Owners Package.Tue, Dec 10, 00:50
roqqit requested review of this revision.Tue, Dec 10, 00:50
Fabien requested changes to this revision.Tue, Dec 10, 08:11
Fabien added a subscriber: Fabien.
Fabien added inline comments.
doc/functional-tests.md
170

This can be slightly improved by putting the symlink in the build directory, so it can be accessed by ./lastfailure

test/functional/test_runner.py
635

you probably want to gate this for unix only, I don't think this will work on windows

This revision now requires changes to proceed.Tue, Dec 10, 08:11
test/functional/test_runner.py
635

https://docs.python.org/3/library/os.html#os.symlink

Note

On newer versions of Windows 10, unprivileged accounts can create symlinks if Developer Mode is enabled. When Developer Mode is not available/enabled, the SeCreateSymbolicLinkPrivilege privilege is required, or the process must be run as an administrator.

OSError is raised when the function is called by an unprivileged user.

so maybe call this in a with contextlib.suppress(OSError): context.

Not sure what the behavior is on older Windows machines

roqqit planned changes to this revision.Tue, Dec 10, 16:51
roqqit added inline comments.
doc/functional-tests.md
170

good idea. it makes it more discoverable too

test/functional/test_runner.py
635

The docs do suggest this could work on Windows. I will read more about contextlib.suppress.

  • move lastfailure to top of build_dir
  • ignore OSError exceptions on windows (didn't use contextlib.suppress because I don't think we should ignore it on Linux)
Fabien added inline comments.
doc/functional-tests.md
164 ↗(On Diff #51521)

this needs to be updated

This revision is now accepted and ready to land.Tue, Dec 10, 19:31