build: migrate from verde to jade_builder
What changed, and why it matters
This commit is a routine build-system housekeeping change. It swaps the Docker image used for automated builds from 'verde' to 'jade_builder', updates the ESP-IDF framework version from 5.4.0 to 5.4.3, adjusts paths to the ESP-IDF installation, and refactors several CI helper scripts. There is no change to the wallet firmware code that handles private keys, transactions, or user data, and no security vulnerability is introduced or fixed in the diff itself.
No security action required. Treat as normal build-maintenance commit. If reviewing for supply-chain assurance, verify the new blockstream/jade_builder image and ESP-IDF 5.4.3 toolchain are built from known, audited sources, but that verification is outside the scope of this code diff.
Security signals we found
No strong security signals were identified.
Evidence from the diff
The commit migrates the project’s CI/build environment from the blockstream/verde image to blockstream/jade_builder, updates idf version references in dependencies.lock files from 5.4.0 to 5.4.3, and normalizes ESP-IDF activation from $HOME/esp/esp-idf/export.sh to /opt/esp/idf/export.sh. It also removes virtualenv setup steps where the new image already provides dependencies, extracts chip-specific flashing logic into new tools/initial_flash_esp32.sh and tools/initial_flash_esp32s3.sh scripts, and updates the ASAN library path for gcc 13. No source code affecting cryptographic operations, memory safety, or protocol handling is modified.
Changed components
CI/CD configuration (.gitlab-ci.yml, gitlab/*.yml)Build helper scripts (ci_flash.sh, ota_delta_ci.sh, main/qemu/*.sh, tools/*.sh)ESP-IDF dependency lock files (dependencies.lock.esp32, dependencies.lock.esp32s3)Inspect captured patch +124 / −153
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 92c5f9c..8277e7b 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -14,7 +14,7 @@ stages:
- build_diy
- docker_build
-image: blockstream/verde@sha256:9171360dae48d3fba7c0f36ab5eb45e2529d178a603e89197cb08fb123ca619e
+image: blockstream/jade_builder@sha256:6937ea8808b89fe3510af6e156da4495d313c0ec7d3370108896ed50b605d237
include:
- gitlab/prod_fw.yml
@@ -50,9 +50,8 @@ include:
- $CI_JOB_NAME/sdkconfig*
- $CI_JOB_NAME/bsdiff.*
script:
- - . $HOME/esp/esp-idf/export.sh
+ - pushd /opt/esp/idf && . ./export.sh && popd
- idf.py all size-components size
- - source /venv/bin/activate
- ./tools/fwprep.py build/jade.bin build
- ./tools/mkindex.py build $(basename $(ls build/*_fw.bin) | cut -d'_' -f1)
- cp sdkconfig sdkconfig.defaults build/
diff --git a/ci_flash.sh b/ci_flash.sh
index 754f006..56d9908 100755
--- a/ci_flash.sh
+++ b/ci_flash.sh
@@ -2,7 +2,7 @@
set -eo pipefail
if [[ -z ${JADESERIALPORT} ]]; then
- echo "Serial port \"${JADESERIALPORT}\" isn't valid, using defaults"
+ echo "JADESERIALPORT not set, defaulting..."
if [ "$(uname)" == "Darwin" ]; then
JADESERIALPORT=/dev/cu.SLAB_USBtoUART
elif [ -c /dev/ttyUSB0 ]; then
@@ -10,74 +10,39 @@ if [[ -z ${JADESERIALPORT} ]]; then
else
JADESERIALPORT=/dev/ttyACM0
fi
- echo "Serial port set to default \"${JADESERIALPORT}\""
fi
+echo "Using JADESERIALPORT ${JADESERIALPORT}"
-TARGET_CHIP=${1:-esp32}
-BUILD_DIR=build
-SKIP_ARGS=$2
+TARGET_CHIP=${1}
+SKIP_ARGS=${2}
-if [ "$TARGET_CHIP" = "esp32" ]; then
- python ${IDF_PATH}/components/esptool_py/esptool/esptool.py --chip ${TARGET_CHIP} --port ${JADESERIALPORT} --baud 2000000 --before default_reset erase_flash
- python ${IDF_PATH}/components/esptool_py/esptool/esptool.py --chip ${TARGET_CHIP} --port ${JADESERIALPORT} --baud 2000000 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size detect 0xE000 ${BUILD_DIR}/ota_data_initial.bin 0x1000 ${BUILD_DIR}/bootloader/bootloader.bin 0x10000 ${BUILD_DIR}/jade.bin 0x9000 ${BUILD_DIR}/partition_table/partition-table.bin
-else
- python ${IDF_PATH}/components/esptool_py/esptool/esptool.py --chip ${TARGET_CHIP} --port ${JADESERIALPORT} --baud 460800 --before default_reset erase_flash
- python ${IDF_PATH}/components/esptool_py/esptool/esptool.py --chip ${TARGET_CHIP} --port ${JADESERIALPORT} --baud 460800 --before=default_reset --after=hard_reset write_flash --flash_mode dio --flash_freq 80m --flash_size 8MB 0x0 ${BUILD_DIR}/bootloader/bootloader.bin 0x20000 ${BUILD_DIR}/jade.bin 0x8000 ${BUILD_DIR}/partition_table/partition-table.bin 0x1a000 ${BUILD_DIR}/ota_data_initial.bin
+if [ -z "$IDF_PATH" ]; then
+ get_idf
fi
+./tools/initial_flash_${TARGET_CHIP}.sh ${JADESERIALPORT}
sleep 1
-
-if fgrep -qs "CONFIG_APPTRACE_GCOV_ENABLE=y" ${BUILD_DIR}/sdkconfig sdkconfig; then
- cleanup() {
- killall -9 openocd || true
- }
-
- trap cleanup EXIT ERR
- killall -9 openocd || true
- # idf.py openocd already needs to do reconfigure but in background could mess things up
- idf.py reconfigure
- idf.py openocd &
-
- # this takes a while on CI because it will want to rebuild things to start openocd
- # Wait up to 20 minutes (1200 seconds) for openocd to start
- for i in {1..1200}; do
- if pgrep openocd > /dev/null; then
- echo "openocd is running"
- break
- fi
- if ! jobs %% >/dev/null; then
- echo "idf.py openocd failed"
- exit 1
- fi
- sleep 1
- done
-
- if ! pgrep openocd > /dev/null; then
- echo "openocd did not start within 20 minutes"
- exit 1
- fi
-fi
-
-if [ -r ~/venv3/bin/activate ]; then
- # Assume we are running under the CI: pinserver requirements are already installed
- source ~/venv3/bin/activate
+if [ -f /.dockerenv ]; then
+ # Running under docker/the CI: main requirements are
+ # already installed into the idf environment.
+ # Just install the pinserver requirements
+ pip install -r pinserver/requirements.txt
else
- # Install and activate a local venv
- if [ ! -r ./venv3/bin/activate ]; then
- virtualenv -p python3 venv3
- fi
- source ./venv3/bin/activate
+ # Running locally: set up a virtualenv
+ virtualenv -p python3 venv3
+ source venv3/bin/activate
+ pip install --require-hashes -r requirements.txt
pip install -r pinserver/requirements.txt
fi
-pip install --require-hashes -r requirements.txt
-# NOTE: tools/fwprep.py should have run in the build step and produced the compressed firmware file
if [ ! -x /usr/bin/bt-agent ]; then
echo "bt-agent not available, skipping bluetooth OTA"
SKIP_ARGS="--skipble"
fi
-FW_FULL=$(ls ${BUILD_DIR}/*_fw.bin)
+
+# NOTE: tools/fwprep.py should have run in the build step and produced the compressed firmware file
+FW_FULL=$(ls build/*_fw.bin)
python jade_ota.py --push-mnemonic --log=INFO --serialport=${JADESERIALPORT} --fwfile=${FW_FULL} ${SKIP_ARGS}
sleep 5
@@ -85,10 +50,6 @@ python -c "from jadepy import JadeAPI; jade = JadeAPI.create_serial(device=\"${J
python test_jade.py --log=INFO --serialport=${JADESERIALPORT} ${SKIP_ARGS}
-# check if gcov is enabled and run collection tool
-if fgrep -qs "CONFIG_APPTRACE_GCOV_ENABLE=y" ${BUILD_DIR}/sdkconfig sdkconfig; then
- ./tools/gcov/generate_report.sh
- killall -9 openocd || true
+if [ ! -f /.dockerenv ]; then
+ deactivate
fi
-
-deactivate
diff --git a/dependencies.lock.esp32 b/dependencies.lock.esp32
index 4136b33..7f168c9 100644
--- a/dependencies.lock.esp32
+++ b/dependencies.lock.esp32
@@ -101,7 +101,7 @@ dependencies:
idf:
source:
type: idf
- version: 5.4.0
+ version: 5.4.3
direct_dependencies:
- espressif/button
- espressif/cbor
diff --git a/dependencies.lock.esp32s3 b/dependencies.lock.esp32s3
index 10661a9..fb4d18c 100644
--- a/dependencies.lock.esp32s3
+++ b/dependencies.lock.esp32s3
@@ -153,7 +153,7 @@ dependencies:
idf:
source:
type: idf
- version: 5.4.0
+ version: 5.4.3
direct_dependencies:
- espressif/button
- espressif/cbor
diff --git a/gitlab/dev_fw.yml b/gitlab/dev_fw.yml
index 4f44126..144a0a9 100644
--- a/gitlab/dev_fw.yml
+++ b/gitlab/dev_fw.yml
@@ -8,39 +8,39 @@
build_dev_jade:
extends: .build_dev_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade --dev
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade --dev
build_dev_jade_noradio:
extends: .build_dev_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade --dev --noradio
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade --dev --noradio
build_dev_jade_v1_1:
extends: .build_dev_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v1_1 --dev
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v1_1 --dev
build_dev_jade_v1_1_noradio:
extends: .build_dev_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v1_1 --dev --noradio
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v1_1 --dev --noradio
build_dev_jade_v2:
extends: .build_dev_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v2 --dev
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v2 --dev
build_dev_jade_v2_noradio:
extends: .build_dev_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v2 --dev --noradio
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v2 --dev --noradio
build_dev_jade_v2c:
extends: .build_dev_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v2c --dev
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v2c --dev
build_dev_jade_v2c_noradio:
extends: .build_dev_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v2c --dev --noradio
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v2c --dev --noradio
diff --git a/gitlab/flash.yml b/gitlab/flash.yml
index f632360..ff0b9c8 100644
--- a/gitlab/flash.yml
+++ b/gitlab/flash.yml
@@ -109,7 +109,7 @@ flash_jade_ota_delta_v1_1_ci:
variables:
IDF_TOOLS_PATH: "/home/gitlab-runner/.idf/54"
script:
- - . $HOME/esp/esp-idf-5.4/export.sh
+ - pushd $HOME/esp/esp-idf-5.4 && . ./export.sh && popd
- mv $(echo $CI_JOB_NAME | sed "s/^${CI_JOB_STAGE}/build_test/") build
- cp build/sdkconfig ./
- ./ci_flash.sh esp32s3 --skipble
@@ -136,7 +136,7 @@ flash_jade_ota_delta_v2_ci:
- job: flash_jade_v2_noradio_ci
artifacts: false
script:
- - . $HOME/esp/esp-idf-5.4/export.sh
+ - pushd $HOME/esp/esp-idf-5.4 && . ./export.sh && popd
- mv build_test_jade_v2_ci build
- mv build_test_jade_v2_noradio_ci build_noradio
- ./ota_delta_ci.sh esp32s3 --skipble
diff --git a/gitlab/prod_fw.yml b/gitlab/prod_fw.yml
index fea4ec3..f6d9809 100644
--- a/gitlab/prod_fw.yml
+++ b/gitlab/prod_fw.yml
@@ -8,39 +8,39 @@
build_prod_jade:
extends: .build_prod_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade
build_prod_jade_noradio:
extends: .build_prod_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade --noradio
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade --noradio
build_prod_jade_v1_1:
extends: .build_prod_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v1_1
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v1_1
build_prod_jade_v1_1_noradio:
extends: .build_prod_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v1_1 --noradio
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v1_1 --noradio
build_prod_jade_v2:
extends: .build_prod_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v2
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v2
build_prod_jade_v2_noradio:
extends: .build_prod_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v2 --noradio
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v2 --noradio
build_prod_jade_v2c:
extends: .build_prod_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v2c
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v2c
build_prod_jade_v2c_noradio:
extends: .build_prod_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v2c --noradio
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v2c --noradio
diff --git a/gitlab/python.yml b/gitlab/python.yml
index ff0ba96..806a1ce 100644
--- a/gitlab/python.yml
+++ b/gitlab/python.yml
@@ -16,10 +16,5 @@ build_jadepy_wheel:
GIT_SUBMODULE_STRATEGY: none
script:
- cd $CI_PROJECT_DIR
- - apt install -yqq python3-venv
- - virtualenv -p python3 venv
- - source venv/bin/activate
- pip install build
- python -m build
- - deactivate
- - rm -rf venv
diff --git a/gitlab/stage_release.sh b/gitlab/stage_release.sh
index 5c8cf85..5aa6fff 100755
--- a/gitlab/stage_release.sh
+++ b/gitlab/stage_release.sh
@@ -11,8 +11,7 @@ export LABEL=${2:-${1}} # use SHA if tag name not given/empty
export UPLOAD_DIR="release/staging/upload"
export STAGING_DIR="release/staging/$LABEL"
-. $HOME/esp/esp-idf/export.sh
-. /venv/bin/activate
+pushd /opt/esp/idf && . ./export.sh && popd
# prod builds
mkdir -p $STAGING_DIR
diff --git a/gitlab/test.yml b/gitlab/test.yml
index 7cf9b39..81d9d1b 100644
--- a/gitlab/test.yml
+++ b/gitlab/test.yml
@@ -6,7 +6,7 @@ test_bip85_rsa_key_gen:
tags: [ ga ]
stage: test
script:
- - . $HOME/esp/esp-idf/export.sh
+ - pushd /opt/esp/idf && . ./export.sh && popd
- (cd tools/bip85_rsa_key_gen && ./test.sh)
- git diff --exit-code
@@ -14,7 +14,7 @@ test_format:
tags: [ ga ]
stage: test
script:
- - . $HOME/esp/esp-idf/export.sh
+ - pushd /opt/esp/idf && . ./export.sh && popd
- ./format.sh
- idf.py reconfigure
- git diff --exit-code
diff --git a/gitlab/test_fw.yml b/gitlab/test_fw.yml
index cf57412..017a7a3 100644
--- a/gitlab/test_fw.yml
+++ b/gitlab/test_fw.yml
@@ -16,54 +16,54 @@
build_test_jade_ci:
extends: .build_test_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade --dev --debug --ci
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade --dev --debug --ci
build_test_jade_noradio_ci:
extends: .build_test_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade --dev --debug --ci --noradio
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade --dev --debug --ci --noradio
build_test_jade_v1_1_ci:
extends: .build_test_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v1_1 --dev --debug --ci
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v1_1 --dev --debug --ci
build_test_jade_v1_1_noradio_ci:
extends: .build_test_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v1_1 --dev --debug --ci --noradio
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v1_1 --dev --debug --ci --noradio
build_test_jade_v2_ci:
extends: .build_test_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v2 --dev --debug --ci --jtag
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v2 --dev --debug --ci --jtag
build_test_jade_v2_noradio_ci:
extends: .build_test_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v2 --dev --debug --ci --jtag --noradio
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v2 --dev --debug --ci --jtag --noradio
build_test_jade_v2c_ci:
extends: .build_test_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v2c --dev --debug --ci --jtag
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v2c --dev --debug --ci --jtag
build_test_jade_v2c_noradio_ci:
extends: .build_test_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh jade_v2c --dev --debug --ci --jtag --noradio
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh jade_v2c --dev --debug --ci --jtag --noradio
build_test_qemu:
extends: .build_test_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh qemu --dev --ci
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh qemu --dev --ci
build_test_qemu_psram:
extends: .build_test_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh qemu --dev --ci --psram
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh qemu --dev --ci --psram
build_test_qemu_psram_unamalgamated:
extends: .build_test_template
before_script:
- - . $HOME/esp/esp-idf/export.sh && ./tools/switch_to.sh qemu --dev --ci --psram --unamalgamated
+ - pushd /opt/esp/idf && . ./export.sh && popd && ./tools/switch_to.sh qemu --dev --ci --psram --unamalgamated
diff --git a/gitlab/test_libjade.yml b/gitlab/test_libjade.yml
index a7f72bf..36e79e1 100644
--- a/gitlab/test_libjade.yml
+++ b/gitlab/test_libjade.yml
@@ -1,10 +1,8 @@
.libjade_test_template:
tags: [ ga ]
before_script:
- - . $HOME/esp/esp-idf/export.sh
+ - pushd /opt/esp/idf && . ./export.sh && popd
- ./tools/switch_to.sh jade --dev --noradio
- - source /venv/bin/activate
- - pip install -r requirements.txt
- pip install -r pinserver/requirements.txt
- pip install .
@@ -23,7 +21,7 @@ test_libjade_sanitize:
- ./libjade/make_libjade.sh Sanitize
- export ASAN_OPTIONS=symbolize=1,detect_leaks=0
- export UBSAN_OPTIONS=print_stacktrace=1
- - export ASAN_SO=/usr/lib/gcc/x86_64-linux-gnu/12/libasan.so
+ - export ASAN_SO=/usr/lib/gcc/x86_64-linux-gnu/13/libasan.so
- export LD_LIBRARY_PATH=$PWD/build_linux/libjade
- export SOCKET_LINK=$PWD/tmp_socket
- echo "----------------------- STANDARD TESTS ------------------------"
diff --git a/main/qemu/docker_test.sh b/main/qemu/docker_test.sh
index 19b5d49..a85e162 100755
--- a/main/qemu/docker_test.sh
+++ b/main/qemu/docker_test.sh
@@ -1,7 +1,7 @@
#!/usr/bin/env bash
set -e
-. /root/esp/esp-idf/export.sh
+pushd /opt/esp/idf && . ./export.sh && popd
# Build the qemu variant of the firmware
cd /jade
@@ -9,11 +9,6 @@ rm -fr sdkconfig
cp configs/sdkconfig_qemu.defaults sdkconfig.defaults
idf.py all
-# Setup the python environment
-virtualenv -p python3 /venv3
-source /venv3/bin/activate
-pip install -r requirements.txt
-
# Build the bsdiff tool in the 'tools' directory
gcc -O2 -DBSDIFF_EXECUTABLE -o tools/bsdiff components/esp32_bsdiff/bsdiff.c
diff --git a/main/qemu/make-flash-img.sh b/main/qemu/make-flash-img.sh
index 4baa210..e4aaad0 100755
--- a/main/qemu/make-flash-img.sh
+++ b/main/qemu/make-flash-img.sh
@@ -3,7 +3,7 @@ set -eo pipefail
FLASH_IMAGE_FILE=/flash_image.bin
-. /root/esp/esp-idf/export.sh
+pushd /opt/esp/idf && . ./export.sh && popd
esptool.py --chip esp32 merge_bin --fill-flash-size 4MB -o ${FLASH_IMAGE_FILE} \
--flash_mode dio --flash_freq 40m --flash_size 4MB \
diff --git a/main/qemu/qemu_ci_flash.sh b/main/qemu/qemu_ci_flash.sh
index 5f6e3b6..836a62b 100755
--- a/main/qemu/qemu_ci_flash.sh
+++ b/main/qemu/qemu_ci_flash.sh
@@ -1,7 +1,7 @@
#!/bin/bash
set -eo pipefail
-. $HOME/esp/esp-idf/export.sh
+pushd /opt/esp/idf && . ./export.sh && popd
# this is the minimum to run qemu
# build/qemu-system-xtensa -nographic \
@@ -32,7 +32,6 @@ qemu-system-xtensa -nographic \
sleep 4
-source /venv/bin/activate
git submodule update --init --depth 1 pinserver
pip install --require-hashes -r requirements.txt -r pinserver/requirements.txt
diff --git a/main/qemu/qemu_gdb.sh b/main/qemu/qemu_gdb.sh
index ee21e41..86ad5c1 100755
--- a/main/qemu/qemu_gdb.sh
+++ b/main/qemu/qemu_gdb.sh
@@ -2,7 +2,7 @@
set -eo pipefail
-. $HOME/esp/esp-idf/export.sh
+pushd /opt/esp/idf && . ./export.sh && popd
pkill -f qemu-system-xtensa | true
diff --git a/main/qemu/qemu_reboot.sh b/main/qemu/qemu_reboot.sh
index c286b5d..4608272 100755
--- a/main/qemu/qemu_reboot.sh
+++ b/main/qemu/qemu_reboot.sh
@@ -2,7 +2,7 @@
set -eo pipefail
-. $HOME/esp/esp-idf/export.sh
+pushd /opt/esp/idf && . ./export.sh && popd
pkill -f qemu-system-xtensa || true
diff --git a/main/qemu/qemu_run.sh b/main/qemu/qemu_run.sh
index d9d49d5..972faa5 100755
--- a/main/qemu/qemu_run.sh
+++ b/main/qemu/qemu_run.sh
@@ -2,7 +2,7 @@
#
set -eo pipefail
-. $HOME/esp/esp-idf/export.sh
+pushd /opt/esp/idf && . ./export.sh && popd
pkill -f qemu-system-xtensa | true
diff --git a/main/qemu/run_emulator.sh b/main/qemu/run_emulator.sh
index f07f600..a86dd09 100755
--- a/main/qemu/run_emulator.sh
+++ b/main/qemu/run_emulator.sh
@@ -26,7 +26,7 @@ cmd="cp -r /jade /jade_cpy && cd /jade_cpy"
cmd+=" && cp configs/${config_file} sdkconfig.defaults"
# comment this out to make subsequent builds faster
#cmd+=" && rm -fr build sdkconfig"
-cmd+=" && . /root/esp/esp-idf/export.sh && idf.py build"
+cmd+=" && pushd /opt/esp/idf && . ./export.sh && popd && idf.py build"
cmd+=" && ./main/qemu/make-flash-img.sh && ./main/qemu/qemu_run.sh"
docker run --rm -v $PWD:/jade \
diff --git a/ota_delta_ci.sh b/ota_delta_ci.sh
index 5e2c7c9..7528055 100755
--- a/ota_delta_ci.sh
+++ b/ota_delta_ci.sh
@@ -2,7 +2,7 @@
set -eo pipefail
if [[ -z ${JADESERIALPORT} ]]; then
- echo "Serial port \"${JADESERIALPORT}\" isn't valid, using defaults"
+ echo "JADESERIALPORT not set, defaulting..."
if [ "$(uname)" == "Darwin" ]; then
JADESERIALPORT=/dev/cu.SLAB_USBtoUART
elif [ -c /dev/ttyUSB0 ]; then
@@ -10,39 +10,32 @@ if [[ -z ${JADESERIALPORT} ]]; then
else
JADESERIALPORT=/dev/ttyACM0
fi
- echo "Serial port set to default \"${JADESERIALPORT}\""
fi
+echo "Using JADESERIALPORT ${JADESERIALPORT}"
-TARGET_CHIP=${1:-esp32}
-BUILD_DIR=build
-SKIP_ARGS=$2
+TARGET_CHIP=${1}
+SKIP_ARGS=${2}
-# Reset the device and then flash the ble-enabled variant
-if [ "$TARGET_CHIP" = "esp32" ]; then
- python ${IDF_PATH}/components/esptool_py/esptool/esptool.py --chip ${TARGET_CHIP} --port ${JADESERIALPORT} --baud 2000000 --before default_reset erase_flash
- python ${IDF_PATH}/components/esptool_py/esptool/esptool.py --chip ${TARGET_CHIP} --port ${JADESERIALPORT} --baud 2000000 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size detect 0xE000 ${BUILD_DIR}/ota_data_initial.bin 0x1000 ${BUILD_DIR}/bootloader/bootloader.bin 0x10000 ${BUILD_DIR}/jade.bin 0x9000 ${BUILD_DIR}/partition_table/partition-table.bin
-else
- python ${IDF_PATH}/components/esptool_py/esptool/esptool.py --chip ${TARGET_CHIP} --port ${JADESERIALPORT} --baud 460800 --before default_reset erase_flash
- python ${IDF_PATH}/components/esptool_py/esptool/esptool.py --chip ${TARGET_CHIP} --port ${JADESERIALPORT} --baud 460800 --before=default_reset --after=hard_reset write_flash --flash_mode dio --flash_freq 80m --flash_size 8MB 0x0 ${BUILD_DIR}/bootloader/bootloader.bin 0x20000 ${BUILD_DIR}/jade.bin 0x8000 ${BUILD_DIR}/partition_table/partition-table.bin 0x1a000 ${BUILD_DIR}/ota_data_initial.bin
+if [ -z "$IDF_PATH" ]; then
+ get_idf
fi
+./tools/initial_flash_${TARGET_CHIP}.sh ${JADESERIALPORT}
sleep 1
-# Setup the python environment
-if [ -r ~/venv3/bin/activate ]; then
- # Assume we are running under the CI: pinserver requirements are already installed
- source ~/venv3/bin/activate
+if [ -f /.dockerenv ]; then
+ # Running under docker/the CI: main requirements are
+ # already installed into the idf environment.
+ # Just install the pinserver requirements
+ pip install -r pinserver/requirements.txt
else
- # Install and activate a local venv
- if [ ! -r ./venv3/bin/activate ]; then
- virtualenv -p python3 venv3
- fi
- source ./venv3/bin/activate
+ # Running locally: set up a virtualenv
+ virtualenv -p python3 venv3
+ source venv3/bin/activate
+ pip install --require-hashes -r requirements.txt
pip install -r pinserver/requirements.txt
fi
-pip install --require-hashes -r requirements.txt
-# NOTE: tools/fwprep.py should have run in the build step and produced the compressed firmware file
if [ ! -x /usr/bin/bt-agent ]; then
echo "bt-agent not available, skipping bluetooth OTA"
SKIP_ARGS="--skipble"
@@ -65,33 +58,41 @@ FW_BLE=$(ls build/*_ble_*_fw.bin)
FW_NORADIO=$(ls build_noradio/*_noradio_*_fw.bin)
# Make four patches between radio and ble firmware variants
-PATCHDIR=patches
-mkdir -p ${PATCHDIR}
+# into the patches/ dir.
+mkdir -p patches
+
+function cleanup {
+ local ret=$?
+ rm -rf ./tools/bsdiff patches
+ exit $ret
+}
+trap cleanup EXIT
-./tools/mkpatch.py ${FW_NORADIO} ${FW_NORADIO} ${PATCHDIR} --force
-./tools/mkpatch.py ${FW_BLE} ${FW_BLE} ${PATCHDIR} --force
-./tools/mkpatch.py ${FW_NORADIO} ${FW_BLE} ${PATCHDIR} --force # makes both directions
+./tools/mkpatch.py ${FW_NORADIO} ${FW_NORADIO} patches --force
+./tools/mkpatch.py ${FW_BLE} ${FW_BLE} patches --force
+./tools/mkpatch.py ${FW_NORADIO} ${FW_BLE} patches --force # makes both directions
+rm -f ./tools/bsdiff
sleep 2
# first we ota to noradio via ble (or serial if skipping ble)
# NOTE: the filename is of the pattern: 'final-from-base' - hence noradio*ble*patch.bin
-FW_PATCH=$(ls ${PATCHDIR}/*_noradio_*_ble*_patch.bin)
+FW_PATCH=$(ls patches/*_noradio_*_ble*_patch.bin)
cp "${FW_NORADIO}.hash" "${FW_PATCH}.hash"
python jade_ota.py --log=INFO ${SKIP_SERIAL} ${SKIP_ARGS} --serialport=${JADESERIALPORT} --fwfile=${FW_PATCH}
# then we test the same exact firmware via serial
-FW_PATCH=$(ls ${PATCHDIR}/*_noradio_*_noradio*_patch.bin)
+FW_PATCH=$(ls patches/*_noradio_*_noradio*_patch.bin)
python jade_ota.py --log=INFO --skipble --serialport=${JADESERIALPORT} --fwfile=${FW_PATCH}
sleep 2
# then we test from noradio to ble via serial
# NOTE: the filename is of the pattern: 'final-from-base' - hence ble*noradio*patch.bin
-FW_PATCH=$(ls ${PATCHDIR}/*_ble_*_noradio*_patch.bin)
+FW_PATCH=$(ls patches/*_ble_*_noradio*_patch.bin)
cp "${FW_BLE}.hash" "${FW_PATCH}.hash"
python jade_ota.py --log=INFO --skipble --serialport=${JADESERIALPORT} --fwfile=${FW_PATCH}
sleep 2
# finally we test the same exact firmware via ble (or serial if skipping ble)
-FW_PATCH=$(ls ${PATCHDIR}/*_ble_*_ble*_patch.bin)
+FW_PATCH=$(ls patches/*_ble_*_ble*_patch.bin)
python jade_ota.py --log=INFO ${SKIP_SERIAL} ${SKIP_ARGS} --serialport=${JADESERIALPORT} --fwfile=${FW_PATCH}
sleep 2
diff --git a/tools/check_default_configs.sh b/tools/check_default_configs.sh
index 949e204..66c06c7 100755
--- a/tools/check_default_configs.sh
+++ b/tools/check_default_configs.sh
@@ -2,7 +2,7 @@
set -eo pipefail
if [ -f /.dockerenv ]; then
- . ${HOME}/esp/esp-idf/export.sh
+ pushd /opt/esp/idf && . ./export.sh && popd
fi
export IDF_COMPONENT_API_CACHE_EXPIRATION_MINUTES=500
diff --git a/tools/initial_flash_esp32.sh b/tools/initial_flash_esp32.sh
new file mode 100755
index 0000000..23a15fa
--- /dev/null
+++ b/tools/initial_flash_esp32.sh
@@ -0,0 +1,12 @@
+#!/usr/bin/env bash
+# Perform an initial flash of an esp32 device
+# ./tools/initial_flash_esp32.sh <serialport>
+set -eo pipefail
+
+if [ -z "$IDF_PATH" ]; then
+ get_idf
+fi
+
+python ${IDF_PATH}/components/esptool_py/esptool/esptool.py --chip esp32 --port ${1} --baud 2000000 --before default_reset erase_flash
+
+python ${IDF_PATH}/components/esptool_py/esptool/esptool.py --chip esp32 --port ${1} --baud 2000000 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size detect 0xE000 build/ota_data_initial.bin 0x1000 build/bootloader/bootloader.bin 0x10000 build/jade.bin 0x9000 build/partition_table/partition-table.bin
diff --git a/tools/initial_flash_esp32s3.sh b/tools/initial_flash_esp32s3.sh
new file mode 100755
index 0000000..e64d033
--- /dev/null
+++ b/tools/initial_flash_esp32s3.sh
@@ -0,0 +1,12 @@
+#!/usr/bin/env bash
+# Perform an initial flash of an esp32s3 device
+# ./tools/initial_flash_esp32s3.sh <serialport>
+set -eo pipefail
+
+if [ -z "$IDF_PATH" ]; then
+ get_idf
+fi
+
+python ${IDF_PATH}/components/esptool_py/esptool/esptool.py --chip esp32s3 --port ${1} --baud 460800 --before default_reset erase_flash
+
+python ${IDF_PATH}/components/esptool_py/esptool/esptool.py --chip esp32s3 --port ${1} --baud 460800 --before=default_reset --after=hard_reset write_flash --flash_mode dio --flash_freq 80m --flash_size 8MB 0x0 build/bootloader/bootloader.bin 0x20000 build/jade.bin 0x8000 build/partition_table/partition-table.bin 0x1a000 build/ota_data_initial.bin
Why this scored 15/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.