Page MenuHomePhabricator

use git filter-repo instead of git filter-branch for the secp256k1 repository
ClosedPublic

Authored by PiRK on Jun 29 2023, 09:43.

Details

Summary

filter-branch is deprecated according to the official git documentation. They recommend using filter-repo instead, which is an external tool that can be used as a git plugin.

The resulting command is more readable, and the process takes much less time (a few seconds instead of 12 hours). It also fixes the issue encountered with the current script not being able to handle whitespaces (see D14143 for an alternative fix to this issue).

Note that the history rewritten by the previous script does not rebase properly on this newly rewritten history.
To make it work, I had to rewrite the entire history of the remote secp256k1 repository:

  • run the script a first time by inserting a exit 0 line just before git pull github master --rebase
  • force push the result of the newly created local secp256k1 to the remote github repository

This needs to be done only once.

Test Plan

Script tested by replacing Bitcoin-ABC/secp256k1.git with PiRK/secp256k1.git and rewritting the history of the remote repo as described above.

Commit extra changes to some cmake/* file or src/secp256k1/* file, commit some unrelated changes to only unrelated files, and rerun the script to check that it adds the proper commits as expected.

pierre@simak:~/dev/test4/bitcoin-abc$ contrib/extract-secp256k1.sh
...
Completely finished after 1.47 seconds.
From github.com:PiRK/secp256k1
 * branch            master     -> FETCH_HEAD
 * [new branch]      master     -> github/master
Current branch master is up to date.
pierre@simak:~/dev/test4/bitcoin-abc$ cd secp256k1/
pierre@simak:~/dev/test4/bitcoin-abc/secp256k1$ git log --pretty=oneline
6fda76d425667ffb22799695d60eea82968ef24b (HEAD -> master) touching stuff in src/secp256k1 and electrum/
2441b09fc64af747fef844430e9802f94906c5ce spamming cmake/FindJemalloc
3879721c67a95f29a45c4fedd65b556a0a1f0541 [cmake] move make_link function to a new MakeLink module
b85a96a724edcdce51f70f4d2b82d2202d902d59 (tag: v0.27.8, github/master) guix: use proper compiler in cmake toolchain
...

Diff Detail

Repository
rABC Bitcoin ABC
Branch
git-filter-repo
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 24357
Build 48323: Build Diff
Build 48322: arc lint + arc unit

Event Timeline

PiRK requested review of this revision.Jun 29 2023, 09:43
PiRK planned changes to this revision.Jun 29 2023, 15:16
Fabien requested changes to this revision.Jun 29 2023, 15:16
Fabien added a subscriber: Fabien.

afaict: it doesn't work

This script works but we need to reset master to the state before the repo got unfiltered files

contrib/extract-secp256k1.sh
20 ↗(On Diff #41105)

This will cause the script to fail on CI, you can revert

PiRK edited the test plan for this revision. (Show Details)

rebase and use https for git remote add

This revision is now accepted and ready to land.Jul 5 2023, 07:17