qa: Require `--exclude` for each excluded test
What changed, and why it matters
This commit is a quality-assurance cleanup for Bitcoin Core's test runner. It changes how tests are excluded from a comma-separated list to one test per `--exclude` flag. There is no security relevance: it does not touch consensus code, networking, wallet handling, or any production behavior. It only affects internal test tooling and CI scripts.
No security action needed. Treat as a normal QA/test-infrastructure change.
Security signals we found
No strong security signals were identified.
Evidence from the diff
The patch modifies test/functional/test_runner.py so the --exclude argument uses action='append' instead of accepting a comma-separated string. It updates CI scripts to pass --exclude once per test and adds a warning if a user still passes a comma in an --exclude value. This is a developer-experience and maintainability change with no runtime security impact.
Changed components
test/functional/test_runner.py.github/workflows/ci.ymlci/test/00_setup_env_native_valgrind.shci/test/00_setup_env_s390x.shInspect captured patch +19 / −10
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 5e5b0c48..3f0e953f 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -501,11 +501,13 @@ jobs:
- name: Run functional tests
env:
- # TODO: Fix the excluded test and re-enable it.
- # feature_unsupported_utxo_db.py fails on windows because of emojis in the test data directory
- EXCLUDE: '--exclude wallet_multiwallet.py,feature_unsupported_utxo_db.py'
TEST_RUNNER_EXTRA: ${{ github.event_name != 'pull_request' && '--extended' || '' }}
- run: py -3 test/functional/test_runner.py --jobs $NUMBER_OF_PROCESSORS --ci --quiet --tmpdirprefix="$RUNNER_TEMP" --combinedlogslen=99999999 --timeout-factor=$TEST_RUNNER_TIMEOUT_FACTOR $EXCLUDE $TEST_RUNNER_EXTRA
+ run: |
+ py -3 test/functional/test_runner.py --jobs $NUMBER_OF_PROCESSORS --ci --quiet --tmpdirprefix="$RUNNER_TEMP" --combinedlogslen=99999999 --timeout-factor=$TEST_RUNNER_TIMEOUT_FACTOR $TEST_RUNNER_EXTRA \
+ `# feature_unsupported_utxo_db.py fails on Windows because of emojis in the test data directory.` \
+ --exclude feature_unsupported_utxo_db.py \
+ `# See https://github.com/bitcoin/bitcoin/issues/31409.` \
+ --exclude wallet_multiwallet.py
ci-matrix:
name: ${{ matrix.name }}
diff --git a/ci/test/00_setup_env_native_valgrind.sh b/ci/test/00_setup_env_native_valgrind.sh
index 884bc395..3e23a580 100755
--- a/ci/test/00_setup_env_native_valgrind.sh
+++ b/ci/test/00_setup_env_native_valgrind.sh
@@ -13,7 +13,7 @@ export PIP_PACKAGES="--break-system-packages pycapnp"
export USE_VALGRIND=1
export NO_DEPENDS=1
# bind tests excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547
-export TEST_RUNNER_EXTRA="--exclude rpc_bind,feature_bind_extra"
+export TEST_RUNNER_EXTRA="--exclude rpc_bind --exclude feature_bind_extra"
export GOAL="install"
# TODO enable GUI
export BITCOIN_CONFIG="\
diff --git a/ci/test/00_setup_env_s390x.sh b/ci/test/00_setup_env_s390x.sh
index b7d23fbf..93031dcc 100755
--- a/ci/test/00_setup_env_s390x.sh
+++ b/ci/test/00_setup_env_s390x.sh
@@ -11,7 +11,8 @@ export PACKAGES="python3-zmq"
export CONTAINER_NAME=ci_s390x
export CI_IMAGE_NAME_TAG="mirror.gcr.io/ubuntu:24.04"
export CI_IMAGE_PLATFORM="linux/s390x"
-export TEST_RUNNER_EXTRA="--exclude rpc_bind,feature_bind_extra" # Excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547
+# bind tests excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547
+export TEST_RUNNER_EXTRA="--exclude rpc_bind --exclude feature_bind_extra"
export RUN_FUNCTIONAL_TESTS=true
export GOAL="install"
export BITCOIN_CONFIG="\
diff --git a/test/functional/test_runner.py b/test/functional/test_runner.py
index 7701f949..8c508029 100755
--- a/test/functional/test_runner.py
+++ b/test/functional/test_runner.py
@@ -400,7 +400,7 @@ def main():
parser.add_argument('--combinedlogslen', '-c', type=int, default=0, metavar='n', help='On failure, print a log (of length n lines) to the console, combined from the test framework and all test nodes.')
parser.add_argument('--coverage', action='store_true', help='generate a basic coverage report for the RPC interface')
parser.add_argument('--ci', action='store_true', help='Run checks and code that are usually only enabled in a continuous integration environment')
- parser.add_argument('--exclude', '-x', help='specify a comma-separated-list of scripts to exclude.')
+ parser.add_argument('--exclude', '-x', action='append', help='specify a script to exclude. Can be specified multiple times. The .py extension is optional.')
parser.add_argument('--extended', action='store_true', help='run the extended test suite in addition to the basic tests')
parser.add_argument('--help', '-h', '-?', action='store_true', help='print help text and exit')
parser.add_argument('--jobs', '-j', type=int, default=4, help='how many test scripts to run in parallel. Default=4.')
@@ -493,7 +493,7 @@ def main():
if args.exclude:
def print_warning_missing_test(test_name):
- print("{}WARNING!{} Test '{}' not found in current test list. Check the --exclude list.".format(BOLD[1], BOLD[0], test_name))
+ print("{}WARNING!{} Test '{}' not found in current test list. Check the --exclude options.".format(BOLD[1], BOLD[0], test_name))
if fail_on_warn:
sys.exit(1)
@@ -501,10 +501,16 @@ def main():
if not exclude_list:
print_warning_missing_test(exclude_test)
for exclude_item in exclude_list:
+ print("Excluding %s" % exclude_item)
test_list.remove(exclude_item)
- exclude_tests = [test.strip() for test in args.exclude.split(",")]
- for exclude_test in exclude_tests:
+ for exclude_test in args.exclude:
+ if ',' in exclude_test:
+ print("{}WARNING!{} --exclude '{}' contains a comma. Use --exclude for each test.".format(BOLD[1], BOLD[0], exclude_test))
+ if fail_on_warn:
+ sys.exit(1)
+
+ for exclude_test in args.exclude:
# A space in the name indicates it has arguments such as "rpc_bind.py --ipv4"
if ' ' in exclude_test:
remove_tests([test for test in test_list if test.replace('.py', '') == exclude_test.replace('.py', '')])
Why this scored 15/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.