ci: reduce sanitizer/valgrind worker count to avoid runner OOM
What the developer wrote
Authored by Níckolas Goline
ci: reduce sanitizer/valgrind worker count to avoid runner OOM
The ASan/UBSan and Valgrind shards intermittently have their hosted
runner killed mid-run with "The runner has received a shutdown signal",
surfaced as a mass test cancellation. The VM itself is being
terminated, most likely from memory exhaustion: `-n $(nproc)+1` runs 5
parallel node-clusters and ASan roughly triples RSS, peaking past the
16GB hosted-runner ceiling on heavy shards (deterministic repro at ~10%
on ASan/UBSan 3/6 due to the fixed --test-group-random-seed=42).
Drop these two jobs to `-n $(nproc)` (one fewer worker) to lower peak
memory, and add an `if: failure()` step that dumps OOM-related dmesg and
kernel-log lines so the next failure records the actual cause.
Changelog-None
What changed, and why it matters
This change only adjusts the project's automated testing setup. It reduces the number of parallel test workers in two CI jobs to stop GitHub's hosted test machines from running out of memory, and adds logging to confirm the cause if they still fail. It does not change the Core Lightning software that users run, and there is no security issue here.
No security action needed. Treat as a normal CI infrastructure maintenance commit.
Security signals we found
No strong security signals were identified.
Evidence from the diff
The commit modifies .github/workflows/ci.yaml. It changes the pytest -n argument from $(($(nproc) + 1)) to $(nproc) for the Valgrind and ASan/UBSan CI shards, and adds a failure-only step that dumps dmesg/journalctl OOM-related lines. This is an infrastructure reliability fix for CI runner memory exhaustion; no application code is touched.
Changed components
.github/workflows/ci.yamlInspect captured patch +16 / −2
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
index 5d5419cf..f7eeb8cc 100644
--- a/.github/workflows/ci.yaml
+++ b/.github/workflows/ci.yaml
@@ -706,7 +706,14 @@ jobs:
TEST_DEBUG: 1
PYTEST_OPTS: ${{ env.PYTEST_OPTS_BASE }} --test-group-random-seed=42
run: |
- VALGRIND=1 sg wireshark "uv run eatmydata pytest tests/ -n $(($(nproc) + 1)) ${PYTEST_OPTS} --test-group=${{ matrix.GROUP }} --test-group-count=${{ matrix.GROUP_COUNT }}"
+ # -n $(nproc) rather than $(nproc)+1: one fewer parallel node-cluster
+ # keeps peak RSS under the 16GB hosted-runner ceiling.
+ VALGRIND=1 sg wireshark "uv run eatmydata pytest tests/ -n $(nproc) ${PYTEST_OPTS} --test-group=${{ matrix.GROUP }} --test-group-count=${{ matrix.GROUP_COUNT }}"
+ - name: Report OOM kills
+ if: failure()
+ run: |
+ echo "=== dmesg (OOM) ==="; sudo dmesg | grep -i -E 'oom|out of memory|killed process' || echo "no OOM lines in dmesg"
+ echo "=== kernel log ==="; sudo journalctl -k --no-pager | tail -n 50 || true
- name: Upload test results
if: always()
uses: actions/upload-artifact@v7
@@ -770,7 +777,14 @@ jobs:
env:
PYTEST_OPTS: ${{ env.PYTEST_OPTS_BASE }} --test-group-random-seed=42
run: |
- sg wireshark "uv run eatmydata pytest tests/ -n $(($(nproc) + 1)) ${PYTEST_OPTS} --test-group=${{ matrix.GROUP }} --test-group-count=${{ matrix.GROUP_COUNT }}"
+ # -n $(nproc) rather than $(nproc)+1: one fewer parallel node-cluster
+ # keeps ASan peak RSS under the 16GB hosted-runner ceiling.
+ sg wireshark "uv run eatmydata pytest tests/ -n $(nproc) ${PYTEST_OPTS} --test-group=${{ matrix.GROUP }} --test-group-count=${{ matrix.GROUP_COUNT }}"
+ - name: Report OOM kills
+ if: failure()
+ run: |
+ echo "=== dmesg (OOM) ==="; sudo dmesg | grep -i -E 'oom|out of memory|killed process' || echo "no OOM lines in dmesg"
+ echo "=== kernel log ==="; sudo journalctl -k --no-pager | tail -n 50 || true
- name: Upload test results
if: always()
uses: actions/upload-artifact@v7
Why this scored 15/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.