← Watch feed
android build: cache p4a recipe downloads even for FRESH_CLONE builds
Public commit record
81/100 · Strong
What the developer wrote
Authored by SomberNight
android build: cache p4a recipe downloads even for FRESH_CLONE builds
I propose we cache some downloads even for "fresh clone" builds (that have `ELECBUILD_COMMIT` env var set), with an opt-out reusing the existing `ELECBUILD_NOCACHE` env var. Currently `ELECBUILD_NOCACHE` only opts-out from the docker build cache, now I would reuse it for p4a recipe downloads.
`qt-everywhere-src-6.10.2.tar.xz` [0] is 1.3 GiB, and we download it twice per arch.
(once for the qt6 recipe, once for the hostqt6 recipe)
It seems to me, download.qt.io is heavily rate-limiting downloads from some IPs.
When doing a release, we build 3 archs, so we download Qt six times.
With this patch, we would only download it twice.
(note: I see no need to have separate caches per arch atm.)
Also, when doing multiple builds to debug reproducibility, with this patch, only the first attempt needs to download Qt (twice). Subsequent attempts hit the cache.
---
note: the git tag is part of the cache key out of cache-staleness paranoia. It is not the full commit though, as that would kill the reproducibility-debugging use case.
[0]: https://github.com/spesmilo/python-for-android/blob/8c0fcc9ef2e559918ca96ecde6e09fe521bb1427/pythonforandroid/recipes/qt6/__init__.py#L11
✓ Specific, descriptive subject✓ Names a concrete action or component✓ Provides detailed explanatory context✓ Links an issue, advisory, or supporting reference
What changed, and why it matters
This change is a build-system optimization for Electrum's Android release builds. It caches large Qt source downloads between builds so they are not repeatedly downloaded from the internet. There is no security vulnerability here.
Recommended action
No security action needed. This is a benign build optimization.
Security signals we found
No strong security signals were identified.
Evidence from the diff
The patch modifies contrib/android/build.sh to reuse a single p4a (python-for-android) package download cache across architectures and across repeated release builds, controlled by the existing ELECBUILD_NOCACHE opt-out. It mounts the cache directory into the Docker container at the per-arch package paths. The change also broadens a chown operation from the fresh clone directory to its parent base directory. No code handling secrets, cryptography, network protocols, or user data is changed.
Changed components
contrib/android/build.shInspect captured patch +17 / −6
diff --git a/contrib/android/build.sh b/contrib/android/build.sh
index a6aec1e..d0357f2 100755
--- a/contrib/android/build.sh
+++ b/contrib/android/build.sh
@@ -32,9 +32,10 @@ mkdir -p "${PROJECT_ROOT}/.buildozer_$1"
ln -s ".buildozer_$1" ${PROJECT_ROOT}/.buildozer
DOCKER_BUILD_FLAGS=""
+DOCKER_RUN_FLAGS=""
if [ ! -z "$ELECBUILD_NOCACHE" ] ; then
info "ELECBUILD_NOCACHE is set. forcing rebuild of docker image."
- DOCKER_BUILD_FLAGS="--pull --no-cache"
+ DOCKER_BUILD_FLAGS="$DOCKER_BUILD_FLAGS --pull --no-cache"
fi
if [ -z "$ELECBUILD_COMMIT" ] ; then # local dev build
@@ -51,21 +52,31 @@ docker build \
# maybe do fresh clone
if [ ! -z "$ELECBUILD_COMMIT" ] ; then
info "ELECBUILD_COMMIT=$ELECBUILD_COMMIT. doing fresh clone and git checkout."
- FRESH_CLONE=${FRESH_CLONE:-"/var/tmp/electrum_build/android/fresh_clone/electrum"}
+ FRESH_CLONE_BASE=${FRESH_CLONE_BASE:-"/var/tmp/electrum_build/android"}
+ FRESH_CLONE="$FRESH_CLONE_BASE/fresh_clone/electrum"
rm -rf "$FRESH_CLONE" 2>/dev/null || ( info "we need sudo to rm prev FRESH_CLONE." && sudo rm -rf "$FRESH_CLONE" )
umask 0022
git clone "$PROJECT_ROOT" "$FRESH_CLONE"
cd "$FRESH_CLONE"
git checkout "$ELECBUILD_COMMIT"
PROJECT_ROOT_OR_FRESHCLONE_ROOT="$FRESH_CLONE"
+ if [ -z "$ELECBUILD_NOCACHE" ] ; then
+ info "ELECBUILD_NOCACHE is not set. mounting p4a download cache."
+ GIT_TAG="$(git describe --abbrev=0)"
+ mkdir -p "$FRESH_CLONE_BASE/downloads_cache/$GIT_TAG/p4a_packages"
+ mkdir -p "$FRESH_CLONE"/.buildozer/android/platform/build-{armeabi-v7a,arm64-v8a,x86,x86_64}/packages
+ DOCKER_RUN_FLAGS="$DOCKER_RUN_FLAGS -v $FRESH_CLONE_BASE/downloads_cache/$GIT_TAG/p4a_packages:/home/user/wspace/electrum/.buildozer/android/platform/build-armeabi-v7a/packages"
+ DOCKER_RUN_FLAGS="$DOCKER_RUN_FLAGS -v $FRESH_CLONE_BASE/downloads_cache/$GIT_TAG/p4a_packages:/home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/packages"
+ DOCKER_RUN_FLAGS="$DOCKER_RUN_FLAGS -v $FRESH_CLONE_BASE/downloads_cache/$GIT_TAG/p4a_packages:/home/user/wspace/electrum/.buildozer/android/platform/build-x86/packages"
+ DOCKER_RUN_FLAGS="$DOCKER_RUN_FLAGS -v $FRESH_CLONE_BASE/downloads_cache/$GIT_TAG/p4a_packages:/home/user/wspace/electrum/.buildozer/android/platform/build-x86_64/packages"
+ fi
else
info "not doing fresh clone."
fi
-DOCKER_RUN_FLAGS=""
if [[ "$3" == "release" ]] ; then
info "'release' mode selected. mounting ~/.keystore inside container."
- DOCKER_RUN_FLAGS="-v $HOME/.keystore:/home/user/.keystore"
+ DOCKER_RUN_FLAGS="$DOCKER_RUN_FLAGS -v $HOME/.keystore:/home/user/.keystore"
fi
if sh -c ": >/dev/tty" >/dev/null 2>/dev/null; then
info "/dev/tty is available and usable"
@@ -77,8 +88,8 @@ mkdir --parents "$PROJECT_ROOT_OR_FRESHCLONE_ROOT"/.buildozer/.gradle
# check uid and maybe chown. see #8261
if [ ! -z "$ELECBUILD_COMMIT" ] ; then # fresh clone (reproducible build)
if [ $(id -u) != "1000" ] || [ $(id -g) != "1000" ] ; then
- info "need to chown -R FRESH_CLONE dir. prompting for sudo."
- sudo chown -R 1000:1000 "$FRESH_CLONE"
+ info "need to chown -R FRESH_CLONE_BASE dir. prompting for sudo."
+ sudo chown -R 1000:1000 "$FRESH_CLONE_BASE"
fi
fi
docker run --rm \
Risk score
Our methodology →Why this scored 15/100
Human-validated context
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
No validated notes yet.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.