Merge bitcoin/bitcoin#36747: Update libmultiprocess subtree to add nonunix platform support and fix race conditions
What changed, and why it matters
This is a routine subtree update that pulls in a newer version of the libmultiprocess inter-process communication library used by Bitcoin Core. The commit message and code changes explicitly describe several bug fixes for race conditions, use-after-free, and file-descriptor leaks in the IPC layer. It also raises the minimum required Cap'n Proto version because older versions have a known memory-safety vulnerability (CVE-2022-46149). The changes are defensive: they fix bugs rather than introduce new attack surface, but the bugs being fixed are in security-relevant areas (cross-process communication, threading, and process spawning).
Treat this as a security-relevant maintenance update. Users building Bitcoin Core with multiprocess support should ensure Cap'n Proto is at least 0.9.2 (or 1.0+ when using Clang). Operators running multiprocess node/GUI should upgrade to the release containing this commit. No immediate emergency response is warranted, but the race-condition fixes should be included in release notes and backported to maintained branches that ship multiprocess builds.
Security signals we found
Fixes data race between IPC worker threads and disconnects (Antithesis)
Fixes use-after-free on socket disconnect before ConnectStream connects
Fixes FD_CLOEXEC race where SpawnProcess descriptors could leak into other forked processes
Fixes connection/resource leaks and hangs when client construct() throws
Bumps minimum Cap'n Proto version to 0.9.2 (1.0 with Clang) and rejects CVE-2022-46149-affected versions
Adds compile-time guard against KJ_NO_EXCEPTIONS=1
Replaces raw FD socketpair with kj AsyncIoStream for portable, reference-counted stream ownership
Adds shutdownWrite in Connection destructor to ensure clean cross-process teardown
Evidence from the diff
The merge commit updates src/ipc/libmultiprocess from v13 to v14. Notable security-relevant changes visible in the diff include: (1) replacing raw Unix socketpair FDs with kj::AsyncIoStream for cross-platform IPC and to avoid FD lifetime races; (2) adding cleanup in ProxyClientBase when construct() throws, preventing connection leaks/hangs; (3) initializing call_context.getResults() on the event-loop thread to avoid a data race with RpcConnectionState::disconnect() during abrupt remote disconnect (libmultiprocess#348); (4) clearing FD_CLOEXEC in the child after fork to prevent SpawnProcess FDs from leaking into concurrently forked processes (libmultiprocess#311); (5) reporting exec failures back to the parent in SpawnProcess (libmultiprocess#312); (6) bumping minimum Cap’n Proto to 0.9.2 and rejecting 0.9.0/0.9.1 for CVE-2022-46149 and pre-1.0 versions with Clang 16+; (7) compile-time error if KJ_NO_EXCEPTIONS is set, because exception-less KJ would let server-side exceptions crash the process. The diff also shows broad modernization (C++20 concepts, requires clauses, std::ranges::copy instead of memcpy) and CI/build improvements.
Changed components
src/ipc/libmultiprocess (subtree)Bitcoin Core IPC / multiprocess modeCap'n Proto dependency version policyCMake build configuration for IPCInspect captured patch +1713 / −581
### CMakeLists.txt
@@ -515,6 +515,18 @@ try_append_cxx_flags("-Wtrailing-whitespace=any" TARGET warn_interface SKIP_LINK
try_append_cxx_flags("-Wunused-parameter" TARGET warn_interface SKIP_LINK
IF_CHECK_PASSED "-Wno-unused-parameter"
)
+# Lambda attributes (e.g. [[noreturn]]) before the parameter list are a C++23
+# extension intentionally used in C++20 builds for compatibility with
+# -Wmissing-noreturn. Suppress the extension warning rather than removing the
+# attributes or working around them.
+try_append_cxx_flags("-Wc++23-lambda-attributes" TARGET warn_interface SKIP_LINK
+ IF_CHECK_PASSED "-Wno-c++23-lambda-attributes"
+)
+# Older clang versions (e.g. clang-17) do not recognize -Wc++23-lambda-attributes
+# but emit the same diagnostic under the broader -Wc++23-extensions flag instead.
+try_append_cxx_flags("-Wc++23-extensions" TARGET warn_interface SKIP_LINK
+ IF_CHECK_PASSED "-Wno-c++23-extensions"
+)
configure_file(cmake/script/Coverage.cmake Coverage.cmake USE_SOURCE_PERMISSIONS COPYONLY)
configure_file(cmake/script/CoverageFuzz.cmake CoverageFuzz.cmake USE_SOURCE_PERMISSIONS COPYONLY)
### doc/dependencies.md
@@ -36,7 +36,7 @@ Bitcoin Core requires one of the following compilers.
| Dependency | Releases | Minimum required |
| --- | --- | --- |
-| [Cap'n Proto](../depends/packages/native_capnp.mk) ([IPC](multiprocess.md)) | [link](https://capnproto.org/) | [0.7.1](https://github.com/bitcoin/bitcoin/pull/33241) |
+| [Cap'n Proto](../depends/packages/native_capnp.mk) ([IPC](multiprocess.md)) | [link](https://capnproto.org/) | [0.9.2](https://github.com/bitcoin/bitcoin/pull/36747) (1.0 with Clang) |
| [libmultiprocess](../depends/packages/native_libmultiprocess.mk) ([IPC](multiprocess.md)) | [link](https://github.com/bitcoin-core/libmultiprocess/tags) | [v7.0-pre1](https://github.com/bitcoin/bitcoin/pull/33517) |
| Python (scripts, tests) | [link](https://www.python.org) | [3.10](https://github.com/bitcoin/bitcoin/pull/30527) |
| [Qt](../depends/packages/qt.mk) (gui) | [link](https://download.qt.io/archive/qt/) | [6.2](https://github.com/bitcoin/bitcoin/pull/30997) |
### src/ipc/libmultiprocess/.clang-tidy
@@ -1,28 +1,58 @@
Checks: '
-*,
bugprone-argument-comment,
+bugprone-dangling-handle,
+bugprone-inaccurate-erase,
+bugprone-integer-division,
+bugprone-lambda-function-name,
bugprone-move-forwarding-reference,
+bugprone-nondeterministic-pointer-iteration-order,
+bugprone-raw-memory-call-on-non-trivial-type,
+bugprone-return-const-ref-from-parameter,
+bugprone-signed-char-misuse,
+bugprone-sizeof-container,
+bugprone-sizeof-expression,
+bugprone-spuriously-wake-up-functions,
bugprone-string-constructor,
-bugprone-use-after-move,
-bugprone-lambda-function-name,
+bugprone-string-literal-with-embedded-nul,
+bugprone-stringview-nullptr,
+bugprone-suspicious-memory-comparison,
+bugprone-suspicious-stringview-data-usage,
+bugprone-swapped-arguments,
+bugprone-undefined-memory-manipulation,
bugprone-unhandled-self-assignment,
-misc-unused-using-decls,
+bugprone-unused-raii,
+bugprone-unused-return-value,
+bugprone-use-after-move,
+bugprone-virtual-near-miss,
+fuchsia-header-anon-namespaces,
+misc-definitions-in-headers,
misc-no-recursion,
+misc-unused-using-decls,
+modernize-avoid-bind,
modernize-deprecated-headers,
+modernize-make-shared,
+modernize-make-unique,
+modernize-unary-static-assert,
+modernize-use-default,
modernize-use-default-member-init,
modernize-use-emplace,
modernize-use-equals-default,
modernize-use-noexcept,
modernize-use-nullptr,
+modernize-use-scoped-lock,
modernize-use-starts-ends-with,
+modernize-use-transparent-functors,
performance-*,
-performance-avoid-endl,
-performance-enum-size,
-performance-inefficient-string-concatenation,
-performance-no-int-to-ptr,
-performance-noexcept-move-constructor,
-performance-unnecessary-value-param,
+readability-avoid-const-params-in-decls,
readability-const-return-type,
+readability-container-contains,
readability-redundant-declaration,
readability-redundant-string-init,
clang-analyzer-core.*,
@@ -35,6 +65,10 @@ CheckOptions:
- key: modernize-deprecated-headers.CheckHeaderFile
value: false
- key: performance-move-const-arg.CheckTriviallyCopyableMove
- value: false
+ value: false # Disabled, to allow the bugprone-use-after-move rule on trivially copyable types, to catch logic bugs
- key: bugprone-unhandled-self-assignment.WarnOnlyIfThisHasSuspiciousField
value: false
+ - key: bugprone-unused-return-value.CheckedReturnTypes
+ value: '^::std::error_code$;^::std::error_condition$;^::std::errc$;^::std::expected$;^::util::Result$;^::util::Expected$'
+ - key: bugprone-unused-return-value.AllowCastToVoid
+ value: true # Can be removed with C++26 once the _ placeholder exists.
### src/ipc/libmultiprocess/.github/workflows/bitcoin-core-ci.yml
@@ -18,7 +18,8 @@ concurrency:
env:
BITCOIN_REPO: bitcoin/bitcoin
- LLVM_VERSION: 22
+ BITCOIN_CORE_REF: refs/heads/master
+ LLVM_VERSION: 23
LIBCXX_DIR: /tmp/libcxx-build/
jobs:
@@ -36,13 +37,13 @@ jobs:
functional_test_runs: 20
nproc_multiplier: 2
functional_timeout_factor: 40
- runner: ubuntu-24.04
+ runner: ubuntu-26.04
apt-llvm: true
packages: >-
ccache
- clang-22
- llvm-22
- libclang-rt-22-dev
+ clang-23
+ llvm-23
+ libclang-rt-23-dev
libboost-dev
libsqlite3-dev
libcapnp-dev
@@ -75,13 +76,14 @@ jobs:
steps:
- name: Checkout Bitcoin Core
- uses: actions/checkout@v4
+ uses: actions/checkout@v7
with:
repository: ${{ env.BITCOIN_REPO }}
+ ref: ${{ env.BITCOIN_CORE_REF }}
fetch-depth: 1
- name: Checkout libmultiprocess
- uses: actions/checkout@v4
+ uses: actions/checkout@v7
with:
path: _libmultiprocess
@@ -113,7 +115,7 @@ jobs:
- name: Restore ccache
id: ccache-restore
- uses: actions/cache/restore@v4
+ uses: actions/cache/restore@v6
with:
path: ${{ env.CCACHE_DIR }}
key: ccache-${{ matrix.name }}-${{ github.ref }}-${{ github.sha }}
@@ -153,7 +155,7 @@ jobs:
run: _libmultiprocess/ci/scripts/bitcoin_core_ci.sh run_ipc_functional_tests "${{ matrix.functional_test_runs }}" "${{ matrix.functional_timeout_factor }}"
- name: Save ccache
- uses: actions/cache/save@v4
+ uses: actions/cache/save@v6
if: github.ref == 'refs/heads/master' || steps.ccache-restore.outputs.cache-hit != 'true'
with:
path: ${{ env.CCACHE_DIR }}
@@ -191,13 +193,14 @@ jobs:
steps:
- name: Checkout Bitcoin Core
- uses: actions/checkout@v4
+ uses: actions/checkout@v7
with:
repository: ${{ env.BITCOIN_REPO }}
+ ref: ${{ env.BITCOIN_CORE_REF }}
fetch-depth: 1
- name: Checkout libmultiprocess
- uses: actions/checkout@v4
+ uses: actions/checkout@v7
with:
path: _libmultiprocess
@@ -212,7 +215,7 @@ jobs:
- name: Restore instrumented libc++ cache
id: libcxx-cache
- uses: actions/cache@v4
+ uses: actions/cache@v6
with:
path: ${{ env.LIBCXX_DIR }}
key: libcxx-Thread-llvmorg-${{ env.LLVM_VERSION }}.1.0
@@ -227,7 +230,7 @@ jobs:
- name: Restore depends cache
id: depends-cache
- uses: actions/cache/restore@v4
+ uses: actions/cache/restore@v6
with:
path: |
depends/built
@@ -239,7 +242,7 @@ jobs:
run: _libmultiprocess/ci/scripts/bitcoin_core_ci.sh build_depends_without_ipc
- name: Save depends cache
- uses: actions/cache/save@v4
+ uses: actions/cache/save@v6
if: steps.depends-cache.outputs.cache-hit != 'true'
with:
path: |
@@ -255,7 +258,7 @@ jobs:
- name: Restore ccache
id: ccache-restore
- uses: actions/cache/restore@v4
+ uses: actions/cache/restore@v6
with:
path: ${{ env.CCACHE_DIR }}
key: ccache-TSan-${{ github.ref }}-${{ github.sha }}
@@ -292,7 +295,7 @@ jobs:
run: _libmultiprocess/ci/scripts/bitcoin_core_ci.sh run_ipc_functional_tests "${{ matrix.functional_test_runs }}" "${{ matrix.functional_timeout_factor }}"
- name: Save ccache
- uses: actions/cache/save@v4
+ uses: actions/cache/save@v6
if: github.ref == 'refs/heads/master' || steps.ccache-restore.outputs.cache-hit != 'true'
with:
path: ${{ env.CCACHE_DIR }}
### src/ipc/libmultiprocess/.github/workflows/ci.yml
@@ -17,12 +17,18 @@ jobs:
# Test all supported releases.
# See https://www.netbsd.org/releases/.
include:
- - release: 9.4
- capnproto-cppflags: 'CPPFLAGS="-DKJ_NO_EXCEPTIONS=0 -DKJ_USE_KQUEUE=0"'
- - release: 10.1
+ # Due to a bug in Cap'n Proto, GCC compilers older
+ # than 14.x require `-DKJ_NO_EXCEPTIONS=0` when
+ # processing `capnproto/c++/src/kj/common.h`.
+ # See https://github.com/capnproto/capnproto/pull/2756/changes/28fa1a6753514d8de56875ec2222fa0e1c3a04f7.
+ - release: '10.1'
capnproto-cppflags: 'CPPFLAGS="-DKJ_NO_EXCEPTIONS=0"'
+ env_vars: 'CXXFLAGS="-DKJ_NO_EXCEPTIONS=0"'
+ - release: '11.0'
+ capnproto-cppflags: 'CPPFLAGS="-DKJ_NO_EXCEPTIONS=0"'
+ env_vars: 'CXXFLAGS="-DKJ_NO_EXCEPTIONS=0"'
steps:
- - uses: actions/checkout@v6
+ - uses: actions/checkout@v7
- name: Start NetBSD VM
uses: vmactions/netbsd-vm@v1
@@ -44,9 +50,9 @@ jobs:
pkgsrc/devel/libtool-base \
pkgsrc/devel/pkgconf \
pkgsrc/devel/zlib \
- `# gcc15 is referenced here because the pkgsrc framework requires lang/gcc15/version.mk to exist` \
+ `# gcc16 is referenced here because the pkgsrc framework requires lang/gcc16/version.mk to exist` \
`# during the "make install" step below, even though we compile our project with gcc14.` \
- pkgsrc/lang/gcc15 \
+ pkgsrc/lang/gcc16 \
pkgsrc/mk \
pkgsrc/pkgtools \
pkgsrc/security/openssl \
@@ -63,7 +69,7 @@ jobs:
- name: Run CI script
run: |
cd ${{ github.workspace }}
- CI_CONFIG="ci/configs/netbsd.bash" bash ci/scripts/ci.sh
+ CI_CONFIG="ci/configs/netbsd.bash" ${{ matrix.env_vars }} bash ci/scripts/ci.sh
build-openbsd:
runs-on: ubuntu-latest
@@ -72,7 +78,7 @@ jobs:
run:
shell: openbsd {0}
steps:
- - uses: actions/checkout@v5
+ - uses: actions/checkout@v7
- name: Start OpenBSD VM
uses: vmactions/openbsd-vm@v1
@@ -94,13 +100,13 @@ jobs:
run:
shell: freebsd {0}
steps:
- - uses: actions/checkout@v5
+ - uses: actions/checkout@v7
- name: Start FreeBSD VM
uses: vmactions/freebsd-vm@v1
with:
prepare: |
- pkg install -y cmake ninja bash capnproto
+ pkg install -y cmake ninja bash capnproto git
sync: 'rsync'
copyback: false
@@ -114,7 +120,7 @@ jobs:
name: build • macos
steps:
- - uses: actions/checkout@v5
+ - uses: actions/checkout@v7
- name: Install dependencies
env:
@@ -131,23 +137,23 @@ jobs:
runs-on: ubuntu-latest
env:
- NIXPKGS_CHANNEL: nixos-25.05
- NIX_EXTRA_CONFIG: |
+ CI_NIXPKGS_CHANNEL: nixos-26.05
+ CI_NIX_CONFIG: |
keep-env-derivations = true
keep-outputs = true
- NIX_EXTRA_CONFIG_ACT: |
+ CI_NIX_CONFIG_ACT: |
sandbox = false
filter-syscalls = false
strategy:
fail-fast: false
matrix:
- config: [default, llvm, gnu32, sanitize, olddeps]
+ config: [default, llvm, gnu32, sanitize, olddeps, newdeps]
name: build • ${{ matrix.config }}
steps:
- - uses: actions/checkout@v5
+ - uses: actions/checkout@v7
- name: Determine CI configuration
id: config
@@ -158,13 +164,12 @@ jobs:
- name: Install Nix
uses: cachix/install-nix-action@v31 # 2025-05-27, from https://github.com/cachix/install-nix-action/tags
with:
- nix_path: nixpkgs=https://github.com/NixOS/nixpkgs/archive/${{ steps.config.outputs.nixpkgs_rev }}.tar.gz
# Act executes inside an unprivileged container (Docker or Podman),
# so KVM support isn't available.
enable_kvm: "${{ github.actor != 'nektos/act' }}"
extra_nix_config: |
- ${{ env.NIX_EXTRA_CONFIG }}
- ${{ github.actor == 'nektos/act' && env.NIX_EXTRA_CONFIG_ACT || '' }}
+ ${{ env.CI_NIX_CONFIG }}
+ ${{ github.actor == 'nektos/act' && env.CI_NIX_CONFIG_ACT || '' }}
- name: Cache Nix store
if: steps.config.outputs.cache_nix_store == 'true'
### src/ipc/libmultiprocess/CMakeLists.txt
@@ -13,7 +13,7 @@ endif()
include("cmake/compat_find.cmake")
find_package(Threads REQUIRED)
-find_package(CapnProto 0.7 QUIET NO_MODULE)
+find_package(CapnProto 0.9 NO_MODULE)
if(NOT CapnProto_FOUND)
message(FATAL_ERROR
"Cap'n Proto is required but was not found.\n"
@@ -35,9 +35,7 @@ set(CAPNPROTO_CLANG_INCOMPATIBLE FALSE)
# https://github.com/capnproto/capnproto/blob/master/security-advisories/2022-11-30-0-pointer-list-bounds.md
# https://capnproto.org/news/2022-11-30-CVE-2022-46149-security-advisory.html
# https://dwrensha.github.io/capnproto-rust/2022/11/30/out_of_bounds_memory_access_bug.html
-if(CapnProto_VERSION STREQUAL "0.7.0"
- OR CapnProto_VERSION STREQUAL "0.8.0"
- OR CapnProto_VERSION STREQUAL "0.9.0"
+if(CapnProto_VERSION STREQUAL "0.9.0"
OR CapnProto_VERSION STREQUAL "0.9.1"
OR CapnProto_VERSION STREQUAL "0.10.0"
OR CapnProto_VERSION STREQUAL "0.10.1"
@@ -66,7 +64,7 @@ if(CAPNPROTO_CVE_AFFECTED OR CAPNPROTO_CLANG_INCOMPATIBLE)
string(APPEND RESOLUTION_OPTIONS " - Upgrade to Cap'n Proto version 1.0 or newer (recommended)\n")
if(CAPNPROTO_CVE_AFFECTED AND NOT CAPNPROTO_CLANG_INCOMPATIBLE)
- string(APPEND RESOLUTION_OPTIONS " - Upgrade to a patched minor version (0.7.1, 0.8.1, 0.9.2, 0.10.3, or later)\n")
+ string(APPEND RESOLUTION_OPTIONS " - Upgrade to a patched minor version (0.9.2, 0.10.3, or later)\n")
elseif(CAPNPROTO_CLANG_INCOMPATIBLE AND NOT CAPNPROTO_CVE_AFFECTED)
string(APPEND RESOLUTION_OPTIONS " - Use GCC instead of Clang\n")
endif()
@@ -98,7 +96,12 @@ if(MP_ENABLE_IWYU)
if(NOT IWYU_EXECUTABLE)
message(FATAL_ERROR "MP_ENABLE_IWYU is ON but include-what-you-use was not found.")
endif()
- set(CMAKE_CXX_INCLUDE_WHAT_YOU_USE "${IWYU_EXECUTABLE};-Xiwyu;--error")
+ # Pass -w so CXXFLAGS meant for the compiler (-Werror -Wpedantic, GCC-only -W
+ # options) do not turn clang warnings into errors in the IWYU pass. Clang
+ # stops after 19 errors, and IWYU then exits nonzero without checking includes.
+ # Run IWYU through a wrapper so its diagnostics are shown when it fails
+ # without suggesting includes (see cmake/iwyu-wrapper.sh).
+ set(CMAKE_CXX_INCLUDE_WHAT_YOU_USE "${CMAKE_CURRENT_SOURCE_DIR}/cmake/iwyu-wrapper.sh;${IWYU_EXECUTABLE};-Xiwyu;--error;-w")
if(DEFINED ENV{IWYU_MAPPING_FILE})
list(APPEND CMAKE_CXX_INCLUDE_WHAT_YOU_USE "-Xiwyu" "--mapping_file=$ENV{IWYU_MAPPING_FILE}")
endif()
@@ -176,9 +179,11 @@ set(MP_PUBLIC_HEADERS
include/mp/type-struct.h
include/mp/type-threadmap.h
include/mp/type-tuple.h
+ include/mp/type-unordered-set.h
include/mp/type-vector.h
include/mp/type-void.h
- include/mp/util.h)
+ include/mp/util.h
+ include/mp/version.h)
add_library(multiprocess STATIC
${MP_PROXY_SRCS}
${MP_PUBLIC_HEADERS}
@@ -204,11 +209,21 @@ add_executable(mpgen src/mp/gen.cpp $<TARGET_OBJECTS:mputil>)
add_executable(Libmultiprocess::mpgen ALIAS mpgen)
target_include_directories(mpgen PRIVATE $<BUILD_INTERFACE:${CMAKE_CURRENT_BINARY_DIR}/include>)
target_include_directories(mpgen PUBLIC $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include> $<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>)
-target_link_libraries(mpgen PRIVATE CapnProto::capnp)
-target_link_libraries(mpgen PRIVATE CapnProto::capnp-rpc)
target_link_libraries(mpgen PRIVATE CapnProto::capnpc)
+target_link_libraries(mpgen PRIVATE CapnProto::capnp)
target_link_libraries(mpgen PRIVATE CapnProto::kj)
+target_link_libraries(mpgen PRIVATE CapnProto::kj-async)
target_link_libraries(mpgen PRIVATE Threads::Threads)
+# Use CAPNP_EXECUTABLE/CAPNPC_CXX_EXECUTABLE cmake variables set by
+# CapnProtoConfig.cmake rather than reading the CapnProto::capnp_tool
+# IMPORTED_LOCATION directly. On Debian/Ubuntu, IMPORTED_LOCATION is broken
+# by a multiarch cmake packaging bug (see
+# https://github.com/bitcoin-core/libmultiprocess/issues/328), but
+# CapnProtoConfig.cmake works around it by hardcoding the correct paths. On
+# other platforms the variables hold generator expressions that
+# target_compile_definitions evaluates correctly at generation time.
+target_compile_definitions(mpgen PRIVATE CAPNP_EXECUTABLE=\"${CAPNP_EXECUTABLE}\")
+target_compile_definitions(mpgen PRIVATE CAPNPC_CXX_EXECUTABLE=\"${CAPNPC_CXX_EXECUTABLE}\")
set_target_properties(mpgen PROPERTIES
INSTALL_RPATH_USE_LINK_PATH TRUE)
set_target_properties(mpgen PROPERTIES
### src/ipc/libmultiprocess/ci/README.md
@@ -21,6 +21,7 @@ CI_CONFIG=ci/configs/llvm.bash ci/scripts/run.sh
CI_CONFIG=ci/configs/gnu32.bash ci/scripts/run.sh
CI_CONFIG=ci/configs/sanitize.bash ci/scripts/run.sh
CI_CONFIG=ci/configs/olddeps.bash ci/scripts/run.sh
+CI_CONFIG=ci/configs/newdeps.bash ci/scripts/run.sh
```
By default CI jobs will reuse their build directories. `CI_CLEAN=1` can be specified to delete them before running instead.
### src/ipc/libmultiprocess/ci/configs/default.bash
@@ -1,5 +1,5 @@
CI_DESC="CI job using default libraries and tools, and running IWYU"
CI_DIR=build-default
-export CXXFLAGS="-Werror -Wall -Wextra -Wpedantic -Wno-unused-parameter"
+export CXXFLAGS="-Werror -Wall -Wextra -Wpedantic -Wundef -Wunused -Wunused-const-variable -Wextra-semi -Wmissing-declarations -Wmissing-noreturn -Wtrailing-whitespace"
CMAKE_ARGS=(-DMP_ENABLE_IWYU=ON)
BUILD_ARGS=(-k)
### src/ipc/libmultiprocess/ci/configs/freebsd.bash
@@ -1,5 +1,5 @@
CI_DESC="CI config for FreeBSD"
CI_DIR=build-freebsd
-export CXXFLAGS="-Werror -Wall -Wextra -Wpedantic -Wno-unused-parameter"
+export CXXFLAGS="-Werror -Wall -Wextra -Wpedantic -Wunused -Wunused-const-variable -Wno-c++23-lambda-attributes"
CMAKE_ARGS=(-G Ninja)
BUILD_ARGS=(-k 0)
### src/ipc/libmultiprocess/ci/configs/gnu32.bash
@@ -6,6 +6,6 @@ NIX_ARGS=(
--arg minimal true
--arg crossPkgs 'import <nixpkgs> { crossSystem = { config = "i686-unknown-linux-gnu"; }; }'
)
-export CXXFLAGS="-Werror -Wall -Wextra -Wpedantic -Wno-unused-parameter"
+export CXXFLAGS="-Werror -Wall -Wextra -Wpedantic -Wunused -Wunused-const-variable"
CMAKE_ARGS=(-G Ninja)
BUILD_ARGS=(-k 0)
### src/ipc/libmultiprocess/ci/configs/llvm.bash
@@ -2,7 +2,7 @@ CI_DESC="CI job using LLVM-based libraries and tools (clang, libc++, clang-tidy,
CI_DIR=build-llvm
NIX_ARGS=(--arg enableLibcxx true)
export CXX=clang++
-export CXXFLAGS="-Werror -Wall -Wextra -Wpedantic -Wthread-safety -Wno-unused-parameter"
+export CXXFLAGS="-Werror -Wall -Wextra -Wextra-semi -Wmissing-prototypes -Wpedantic -Wundef -Wthread-safety -Wunused -Wunused-const-variable -Wno-c++23-lambda-attributes"
CMAKE_ARGS=(
-G Ninja
-DMP_ENABLE_CLANG_TIDY=ON
### src/ipc/libmultiprocess/ci/configs/macos.bash
@@ -1,5 +1,5 @@
CI_DESC="CI config for macOS"
CI_DIR=build-macos
-export CXXFLAGS="-Werror -Wall -Wextra -Wpedantic -Wno-unused-parameter"
+export CXXFLAGS="-Werror -Wall -Wextra -Wpedantic -Wunused -Wunused-const-variable -Wno-c++23-lambda-attributes"
CMAKE_ARGS=(-G Ninja -DMP_ENABLE_CLANG_TIDY=ON)
BUILD_ARGS=(-k 0)
### src/ipc/libmultiprocess/ci/configs/netbsd.bash
@@ -1,11 +1,5 @@
CI_DESC="CI config for NetBSD"
CI_DIR=build-netbsd
-export CXXFLAGS="-Werror -Wall -Wextra -Wpedantic -Wno-unused-parameter"
-# Hardcode GCC 14, since default GCC versions installed by NetBSD are older
-# and may not be compatible with libmultiprocess. GCC 14 was chosen because
-# it's the latest compiler available on all versions of NetBSD that we test.
-# Note that the GCC version specified here must match the version specified
-# in pkg_add in ci.yml.
-export CXX="/usr/pkg/gcc14/bin/g++"
+export CXXFLAGS="-Werror -Wall -Wextra -Wpedantic -Wunused -Wunused-const-variable ${CXXFLAGS:-}"
CMAKE_ARGS=(-G Ninja)
BUILD_ARGS=(-k 0)
### src/ipc/libmultiprocess/ci/configs/newdeps.bash
@@ -0,0 +1,6 @@
+CI_DESC="CI job using newest Cap'n Proto and cmake versions"
+CI_DIR=build-newdeps
+export CXXFLAGS="-Werror -Wall -Wextra -Wpedantic -Wunused -Wunused-const-variable -Wno-error=array-bounds"
+CAPNP_CHECKOUT=master # This is the v1.x release branch
+NIX_ARGS=(--argstr capnprotoVersion "none" --argstr cmakeVersion "4.3.4")
+BUILD_ARGS=(-k)
### src/ipc/libmultiprocess/ci/configs/olddeps.bash
@@ -1,5 +1,8 @@
CI_DESC="CI job using old Cap'n Proto and cmake versions"
CI_DIR=build-olddeps
-export CXXFLAGS="-Werror -Wall -Wextra -Wpedantic -Wno-unused-parameter -Wno-error=array-bounds"
-NIX_ARGS=(--argstr capnprotoVersion "0.7.1" --argstr cmakeVersion "3.12.4")
+# Pin olddeps to an older Nixpkgs channel, to be able to test an older GCC.
+CI_NIXPKGS_CHANNEL=nixos-25.05
+export CXXFLAGS="-Werror -Wall -Wextra -Wpedantic -Wunused -Wunused-const-variable -Wno-error=array-bounds"
+CAPNP_CHECKOUT=v0.9.2 # Use a checkout to compile it with the selected GCC
+NIX_ARGS=(--argstr capnprotoVersion "none" --argstr cmakeVersion "3.12.4" --argstr gccVersion "10")
BUILD_ARGS=(-k)
### src/ipc/libmultiprocess/ci/configs/openbsd.bash
@@ -1,5 +1,8 @@
CI_DESC="CI config for OpenBSD"
CI_DIR=build-openbsd
-export CXXFLAGS="-Werror -Wall -Wextra -Wpedantic -Wno-unused-parameter"
+# The -Wno-c2y-extensions flag can be dropped once the change
+# from commit 3724713e82456ce7d66d1f377539dd2e142d969a becomes
+# available in Clang.
+export CXXFLAGS="-Werror -Wall -Wextra -Wpedantic -Wunused -Wunused-const-variable -Wno-c++23-lambda-attributes -Wno-c2y-extensions"
CMAKE_ARGS=(-G Ninja)
BUILD_ARGS=(-k 0)
### src/ipc/libmultiprocess/ci/configs/sanitize.bash
@@ -2,7 +2,7 @@ CI_DESC="CI job running ThreadSanitizer"
CI_DIR=build-sanitize
NIX_ARGS=(--arg enableLibcxx true --argstr libcxxSanitizers "Thread" --argstr capnprotoSanitizers "thread")
export CXX=clang++
-export CXXFLAGS="-ggdb -Werror -Wall -Wextra -Wpedantic -Wthread-safety -Wno-unused-parameter -fsanitize=thread"
+export CXXFLAGS="-ggdb -Werror -Wall -Wextra -Wpedantic -Wthread-safety -Wunused -Wunused-const-variable -fsanitize=thread -Wno-c++23-lambda-attributes"
CMAKE_ARGS=()
-BUILD_ARGS=(-k -j4)
+BUILD_ARGS=(-k)
BUILD_TARGETS=(mptest)
### src/ipc/libmultiprocess/ci/scripts/bitcoin_core_ci.sh
@@ -15,7 +15,7 @@ replace_subtree() {
}
add_llvm_apt_repository() {
- curl -s "https://apt.llvm.org/llvm-snapshot.gpg.key" | sudo tee "/etc/apt/trusted.gpg.d/apt.llvm.org.asc" > /dev/null
+ curl --fail --silent --show-error --location "https://apt.llvm.org/llvm-snapshot.gpg.key" | sudo tee "/etc/apt/trusted.gpg.d/apt.llvm.org.asc" > /dev/null
source /etc/os-release
echo "deb http://apt.llvm.org/${VERSION_CODENAME}/ llvm-toolchain-${VERSION_CODENAME}-${LLVM_VERSION} main" | sudo tee "/etc/apt/sources.list.d/llvm.list"
sudo apt-get update
### src/ipc/libmultiprocess/ci/scripts/ci.sh
@@ -21,10 +21,43 @@ cmake --version
cmake_ver=$(cmake --version | awk '/version/{print $3; exit}')
ver_ge() { [ "$(printf '%s\n' "$2" "$1" | sort -V | head -n1)" = "$2" ]; }
+# If CAPNP_CHECKOUT was requested, clone and install requested Cap'n Proto branch or tag
+capnp_prefix=
+if [ -n "${CAPNP_CHECKOUT-}" ]; then
+ capnp_prefix="$PWD/capnp-install"
+ [ -e "capnp" ] || git clone -b "${CAPNP_CHECKOUT}" "https://github.com/capnproto/capnproto" capnp
+ mkdir -p capnp/build
+ (
+ cd capnp/build
+ git --no-pager log -1 || true
+ CXXFLAGS="-std=c++20" cmake .. "-DCMAKE_INSTALL_PREFIX=${capnp_prefix}" -DBUILD_TESTING=OFF -DWITH_OPENSSL=OFF -DWITH_ZLIB=OFF
+ cmake --build .
+ cmake --build . --target install # Replacement for modern "cmake --install .".
+ )
+ export CMAKE_PREFIX_PATH="${capnp_prefix}:${CMAKE_PREFIX_PATH-}"
+fi
+
src_dir=$PWD
mkdir -p "$CI_DIR"
cd "$CI_DIR"
-cmake "$src_dir" "${CMAKE_ARGS[@]+"${CMAKE_ARGS[@]}"}"
+export CMAKE_BUILD_PARALLEL_LEVEL="$(nproc)"
+git --no-pager log -1 || true
+cmake_args=("${CMAKE_ARGS[@]+"${CMAKE_ARGS[@]}"}")
+if ! cmake "$src_dir" "${cmake_args[@]}"; then
+ # If cmake failed, try it again with debug options.
+ # Could add --trace / --trace-expand here too but they are very verbose.
+ cmake_args+=(--debug-output --debug-trycompile)
+ if ver_ge "$cmake_ver" "3.16"; then cmake_args+=(--log-level=DEBUG); fi
+ if ver_ge "$cmake_ver" "3.17"; then cmake_args+=(--debug-find); fi
+ cmake "$src_dir" "${cmake_args[@]}" || : "cmake exited with $?"
+ if ver_ge "$cmake_ver" "3.26"; then
+ cat CMakeFiles/CMakeConfigureLog.yaml || true
+ else
+ cat CMakeFiles/CMakeError.log CMakeFiles/CMakeOutput.log || true
+ fi
+ find . -ls || true
+ false
+fi
if ver_ge "$cmake_ver" "3.15"; then
cmake --build . -t "${BUILD_TARGETS[@]}" -- "${BUILD_ARGS[@]+"${BUILD_ARGS[@]}"}"
else
### src/ipc/libmultiprocess/ci/scripts/config.sh
@@ -17,8 +17,8 @@ source "${SCRIPT_DIR}/ci_helpers.sh"
[ "${CI_CONFIG+x}" ] && source "$CI_CONFIG"
# Resolve the nixpkgs channel to a specific revision for use in cache keys.
-if [[ -n "${NIXPKGS_CHANNEL:-}" ]]; then
- rev="$(curl --fail --location --silent --show-error "https://channels.nixos.org/${NIXPKGS_CHANNEL}/git-revision")"
+if [[ -n "${CI_NIXPKGS_CHANNEL:-}" ]]; then
+ rev="$(curl --fail --location --silent --show-error "https://channels.nixos.org/${CI_NIXPKGS_CHANNEL}/git-revision")"
test -n "${rev}"
write_output_var nixpkgs_rev "${rev}"
fi
### src/ipc/libmultiprocess/ci/scripts/run.sh
@@ -10,14 +10,16 @@ set -o errexit -o nounset -o pipefail -o xtrace
[ "${CI_CONFIG+x}" ] && source "$CI_CONFIG"
-nix develop --ignore-environment --keep CI_CONFIG --keep CI_CLEAN "${NIX_ARGS[@]+"${NIX_ARGS[@]}"}" -f shell.nix --command ci/scripts/ci.sh
+nix_args=("${NIX_ARGS[@]+${NIX_ARGS[@]}}")
+if [[ -n "${CI_NIXPKGS_CHANNEL:-}" ]]; then
+ nix_args+=(-I "nixpkgs=channel:${CI_NIXPKGS_CHANNEL}")
+fi
+nix develop "${nix_args[@]}" --ignore-environment --keep CI_CONFIG --keep CI_CLEAN -f shell.nix --command ci/scripts/ci.sh
# Create a GC root for the shell closure so the cache-nix-action save step
# does not garbage-collect it.
if [ -n "${CI_CACHE_NIX_STORE-}" ]; then
- nix-build shell.nix \
- -o "$CI_DIR/gcroot" \
- "${NIX_ARGS[@]+"${NIX_ARGS[@]}"}"
+ nix-build shell.nix "${nix_args[@]}" -o "$CI_DIR/gcroot"
# Verify the closure is complete so the cache-nix-action save step has
# everything it needs.
nix-store --query --requisites "$CI_DIR/gcroot" >/dev/null
### src/ipc/libmultiprocess/cmake/TargetCapnpSources.cmake
@@ -83,7 +83,7 @@ function(target_capnp_sources target include_prefix)
add_custom_command(
OUTPUT ${capnp_file}.c++ ${capnp_file}.h ${capnp_file}.proxy-client.c++ ${capnp_file}.proxy-types.h ${capnp_file}.proxy-server.c++ ${capnp_file}.proxy-types.c++ ${capnp_file}.proxy.h
COMMAND ${MPGEN_BINARY} ${CMAKE_CURRENT_SOURCE_DIR} ${include_prefix} ${CMAKE_CURRENT_SOURCE_DIR}/${capnp_file} ${TCS_IMPORT_PATHS} ${mp_include_dir}
- DEPENDS ${capnp_file}
+ DEPENDS ${capnp_file} ${MPGEN_BINARY}
VERBATIM
)
# Skip linting for capnp-generated files but keep it for mpgen-generated ones
### src/ipc/libmultiprocess/cmake/compat_config.cmake
@@ -15,45 +15,3 @@ endif()
if (NOT DEFINED CAPNPC_OUTPUT_DIR)
set(CAPNPC_OUTPUT_DIR "${CMAKE_CURRENT_BINARY_DIR}")
endif()
-
-# CMake target definitions for backwards compatibility with Ubuntu bionic
-# capnproto 0.6.1 package (https://packages.ubuntu.com/bionic/libcapnp-dev)
-# https://github.com/bitcoin-core/libmultiprocess/issues/27
-
-if (NOT DEFINED CAPNP_LIB_CAPNPC AND DEFINED CAPNP_LIB_CAPNP-RPC)
- string(REPLACE "-rpc" "c" CAPNP_LIB_CAPNPC "${CAPNP_LIB_CAPNP-RPC}")
-endif()
-
-if (NOT DEFINED CapnProto_capnpc_IMPORTED_LOCATION AND DEFINED CapnProto_capnp-rpc_IMPORTED_LOCATION)
- string(REPLACE "-rpc" "c" CapnProto_capnpc_IMPORTED_LOCATION "${CapnProto_capnp-rpc_IMPORTED_LOCATION}")
-endif()
-
-if (NOT TARGET CapnProto::capnp AND DEFINED CAPNP_LIB_CAPNP)
- add_library(CapnProto::capnp SHARED IMPORTED)
- set_target_properties(CapnProto::capnp PROPERTIES IMPORTED_LOCATION "${CAPNP_LIB_CAPNP}")
-endif()
-
-if (NOT TARGET CapnProto::capnpc AND DEFINED CAPNP_LIB_CAPNPC)
- add_library(CapnProto::capnpc SHARED IMPORTED)
- set_target_properties(CapnProto::capnpc PROPERTIES IMPORTED_LOCATION "${CAPNP_LIB_CAPNPC}")
-endif()
-
-if (NOT TARGET CapnProto::capnpc AND DEFINED CapnProto_capnpc_IMPORTED_LOCATION)
- add_library(CapnProto::capnpc SHARED IMPORTED)
- set_target_properties(CapnProto::capnpc PROPERTIES IMPORTED_LOCATION "${CapnProto_capnpc_IMPORTED_LOCATION}")
-endif()
-
-if (NOT TARGET CapnProto::capnp-rpc AND DEFINED CAPNP_LIB_CAPNP-RPC)
- add_library(CapnProto::capnp-rpc SHARED IMPORTED)
- set_target_properties(CapnProto::capnp-rpc PROPERTIES IMPORTED_LOCATION "${CAPNP_LIB_CAPNP-RPC}")
-endif()
-
-if (NOT TARGET CapnProto::kj AND DEFINED CAPNP_LIB_KJ)
- add_library(CapnProto::kj SHARED IMPORTED)
- set_target_properties(CapnProto::kj PROPERTIES IMPORTED_LOCATION "${CAPNP_LIB_KJ}")
-endif()
-
-if (NOT TARGET CapnProto::kj-async AND DEFINED CAPNP_LIB_KJ-ASYNC)
- add_library(CapnProto::kj-async SHARED IMPORTED)
- set_target_properties(CapnProto::kj-async PROPERTIES IMPORTED_LOCATION "${CAPNP_LIB_KJ-ASYNC}")
-endif()
### src/ipc/libmultiprocess/cmake/iwyu-wrapper.sh
@@ -0,0 +1,22 @@
+#!/usr/bin/env bash
+# Run include-what-you-use for CMAKE_CXX_INCLUDE_WHAT_YOU_USE and make sure its
+# output is shown when it fails. Usage: iwyu-wrapper.sh IWYU [ARGS...]
+#
+# "cmake -E __run_co_compile" only prints IWYU output that contains "should add
+# these lines:" or "should remove these lines:" (HandleIWYU in CMake's
+# Source/cmcmd.cxx). If IWYU fails for another reason, such as a clang error,
+# the build stops with a bare "Error 1". In that case this script appends a line
+# containing the marker text, so CMake prints the clang diagnostics too.
+#
+# This works around https://gitlab.kitware.com/cmake/cmake/-/issues/28020 and
+# can be removed once CMake prints IWYU output whenever IWYU fails.
+set -o nounset
+iwyu=$1
+shift
+out=$("$iwyu" "$@" 2>&1)
+ret=$?
+printf '%s\n' "$out" >&2
+if [ "$ret" -ne 0 ] && ! grep -q -e 'should add these lines:' -e 'should remove these lines:' <<<"$out"; then
+ printf 'include-what-you-use failed with exit status %s before suggesting includes (marker for CMake: "should add these lines:"); see the diagnostics above.\n' "$ret" >&2
+fi
+exit "$ret"
### src/ipc/libmultiprocess/cmake/pthread_checks.cmake
@@ -12,17 +12,25 @@ cmake_push_check_state()
set(CMAKE_REQUIRED_LIBRARIES Threads::Threads)
check_cxx_source_compiles("
#include <pthread.h>
- int main(int argc, char** argv)
+ int main()
{
char thread_name[16];
return pthread_getname_np(pthread_self(), thread_name, sizeof(thread_name));
}"
HAVE_PTHREAD_GETNAME_NP)
+check_cxx_source_compiles("
+ #include <pthread.h>
+ int main()
+ {
+ return pthread_setname_np(pthread_self(), \"%s\", (void*)\"name\");
+ }"
+ HAVE_PTHREAD_SETNAME_NP_3ARG)
+
check_cxx_source_compiles("
#include <cstdint>
#include <pthread.h>
- int main(int argc, char** argv)
+ int main()
{
uint64_t tid;
pthread_threadid_np(nullptr, &tid);
@@ -33,7 +41,7 @@ check_cxx_source_compiles("
check_cxx_source_compiles("
#include <pthread.h>
#include <pthread_np.h>
- int main(int argc, char** argv)
+ int main()
{
return pthread_getthreadid_np();
}"
### src/ipc/libmultiprocess/doc/design.md
@@ -120,7 +120,7 @@ sequenceDiagram
participant PMT as ProxyMethodTraits
participant Impl as Actual C++ Method
- serverInvoke->>SF1: SF1::invoke
+ serverInvoke->>SF1: SF1::invoke
SF1->>SF2: SF2::invoke
SF2->>SR: SR::invoke
SR->>SC: SC::invoke
@@ -167,7 +167,7 @@ Thread mapping enables each client thread to have a dedicated server thread proc
Thread mapping is initialized by defining an interface method with a `ThreadMap` parameter and/or response. The example below adds `ThreadMap` to the `construct` method because libmultiprocess calls the `construct` method automatically.
```capnp
-interface InitInterface $Proxy.wrap("Init") {
+interface InitInterface $Proxy.wrap("Init") {
construct @0 (threadMap: Proxy.ThreadMap) -> (threadMap :Proxy.ThreadMap);
}
```
### src/ipc/libmultiprocess/doc/install.md
@@ -1,6 +1,6 @@
# libmultiprocess Installation
-Installation currently requires Cap'n Proto 0.7 or higher:
+Installation currently requires Cap'n Proto 0.9 or higher:
```sh
apt install libcapnp-dev capnproto
### src/ipc/libmultiprocess/doc/usage.md
@@ -12,6 +12,12 @@ The `*.capnp` data definition files are consumed by the _libmultiprocess_ code g
The `ProxyServer` objects help translate IPC requests from a socket to method calls on a local object. The `ProxyServer` objects are just used internally by the `mp::ServeStream(loop, socket, wrapped_object)` and `mp::ListenConnections(loop, socket, wrapped_object[, max_connections])` functions, and aren't exposed externally. The `ProxyClient` classes are exposed, and returned from the `mp::ConnectStream(loop, socket)` function and meant to be used directly. The classes implement methods described in `.capnp` definitions, and whenever any method is called, a request with the method arguments is sent over the associated IPC connection, and the corresponding `wrapped_object` method on the other end of the connection is called, with the `ProxyClient` method blocking until it returns and forwarding back any return value to the `ProxyClient` method caller.
+## Event loop
+
+Making connections and IPC calls requires an `mp::EventLoop` instance running in its own dedicated thread. The event loop is reference-counted, and `EventLoop::loop()` returns when the last `mp::EventLoopRef` is released. Therefore, any code that needs to use the loop across multiple calls should maintain its own `EventLoopRef`. Connection objects and proxy objects automatically hold a reference for as long as they exist.
+
+For a typical setup, see [example.cpp](../example/example.cpp).
+
## Example
A simple interface description can be found at [test/mp/test/foo.capnp](../test/mp/test/foo.capnp), implementation in [test/mp/test/foo.h](../test/mp/test/foo.h), and usage in [test/mp/test/test.cpp](../test/mp/test/test.cpp).
### src/ipc/libmultiprocess/doc/versions.md
@@ -7,9 +7,15 @@ Library versions are tracked with simple
Versioning policy is described in the [version.h](../include/mp/version.h)
include.
-## v13
+## v14
+- Adds support for nonunix platforms, making API changes that are not backwards compatible ([#274](https://github.com/bitcoin-core/libmultiprocess/pull/274)).
- Current unstable version.
+## [v13.0](https://github.com/bitcoin-core/libmultiprocess/commits/v13.0)
+- Fixes a `memcpy` ubsan warning by using `std::ranges::copy` ([#305](https://github.com/bitcoin-core/libmultiprocess/pull/305)).
+- Misc: map serialization round-trip test coverage ([#297](https://github.com/bitcoin-core/libmultiprocess/pull/297)) and version bump ([#306](https://github.com/bitcoin-core/libmultiprocess/pull/306)).
+- Used in Bitcoin Core 32.x and newer releases with [#35720](https://github.com/bitcoin/bitcoin/pull/35720).
+
## [v12.0](https://github.com/bitcoin-core/libmultiprocess/commits/v12.0)
- Adds an optional `max_connections` parameter to `ListenConnections` ([#269](https://github.com/bitcoin-core/libmultiprocess/pull/269)).
- Used in Bitcoin Core 32.x and newer releases with [#35684](https://github.com/bitcoin/bitcoin/pull/35684).
@@ -24,7 +30,7 @@ include.
- Fixes a rare mptest hang on musl builds caused by a lost wakeup bug in `Waiter` ([#295](https://github.com/bitcoin-core/libmultiprocess/pull/295)).
- Fixes a race condition in a log print detected by TSan ([#286](https://github.com/bitcoin-core/libmultiprocess/pull/286)).
- Build improvements: makes `target_capnp_sources` work correctly when libmultiprocess is used as a CMake subproject ([#289](https://github.com/bitcoin-core/libmultiprocess/pull/289)), adds `mp_headers` target for better lint tool support ([#291](https://github.com/bitcoin-core/libmultiprocess/pull/291)), and fixes compatibility with recent Nix and CMake 4.0 ([#238](https://github.com/bitcoin-core/libmultiprocess/pull/238)).
-- Test, CI, documentation, and minor code improvements: design document corrections ([#278](https://github.com/bitcoin-core/libmultiprocess/pull/278)), field constant comments ([#279](https://github.com/bitcoin-core/libmultiprocess/pull/279)), clang-tidy fix ([#292](https://github.com/bitcoin-core/libmultiprocess/pull/292)), new smoke test for double-precision float values ([#294](https://github.com/bitcoin-core/libmultiprocess/pull/294)), new test for recursive async IPC calls ([#301](https://github.com/bitcoin-core/libmultiprocess/pull/301)), removal of libevent from Core CI builds ([#299](https://github.com/bitcoin-core/libmultiprocess/pull/299)), and rename of `EventLoop::m_num_clients` to `m_num_refs` ([#302](https://github.com/bitcoin-core/libmultiprocess/pull/302)).
+- Test, CI, documentation, and minor code improvements: design document corrections ([#278](https://github.com/bitcoin-core/libmultiprocess/pull/278)), field constant comments ([#279](https://github.com/bitcoin-core/libmultiprocess/pull/279)), clang-tidy fix ([#292](https://github.com/bitcoin-core/libmultiprocess/pull/292)), new smoke test for double-precision float values ([#294](https://github.com/bitcoin-core/libmultiprocess/pull/294)), new test for recursive async IPC calls ([#301](https://github.com/bitcoin-core/libmultiprocess/pull/301)), removal of libevent from Core CI builds ([#299](https://github.com/bitcoin-core/libmultiprocess/pull/299)), rename of `EventLoop::m_num_clients` to `m_num_refs` ([#302](https://github.com/bitcoin-core/libmultiprocess/pull/302)), and version bump ([#270](https://github.com/bitcoin-core/libmultiprocess/pull/270)).
- Used in Bitcoin Core 32.x and newer releases with [#35661](https://github.com/bitcoin/bitcoin/pull/35661).
## [v10.0](https://github.com/bitcoin-core/libmultiprocess/commits/v10.0)
### src/ipc/libmultiprocess/example/calculator.cpp
@@ -6,8 +6,7 @@
#include <init.capnp.h>
#include <init.capnp.proxy.h> // NOLINT(misc-include-cleaner) // IWYU pragma: keep
-#include <charconv>
-#include <cstring>
+#include <cstring> // IWYU pragma: keep
#include <fstream>
#include <functional>
#include <iostream>
@@ -16,9 +15,9 @@
#include <kj/memory.h>
#include <memory>
#include <mp/proxy-io.h>
+#include <mp/util.h>
#include <stdexcept>
#include <string>
-#include <system_error>
#include <utility>
class CalculatorImpl : public Calculator
@@ -51,14 +50,10 @@ int main(int argc, char** argv)
std::cout << "Usage: mpcalculator <fd>\n";
return 1;
}
- int fd;
- if (std::from_chars(argv[1], argv[1] + strlen(argv[1]), fd).ec != std::errc{}) {
- std::cerr << argv[1] << " is not a number or is larger than an int\n";
- return 1;
- }
+ mp::SocketId socket{mp::StartSpawned(argv[1])};
mp::EventLoop loop("mpcalculator", LogPrint);
std::unique_ptr<Init> init = std::make_unique<InitImpl>();
- mp::ServeStream<InitInterface>(loop, fd, *init);
+ mp::ServeStream<InitInterface>(loop, mp::MakeStream(loop, socket), *init);
loop.loop();
return 0;
}
### src/ipc/libmultiprocess/example/example.cpp
@@ -10,29 +10,29 @@
#include <fstream>
#include <future>
#include <iostream>
-#include <kj/async.h>
#include <kj/common.h>
#include <memory>
+#include <mp/proxy.h>
#include <mp/proxy-io.h>
#include <mp/util.h>
#include <stdexcept>
#include <string>
#include <thread>
#include <tuple>
+#include <utility>
#include <vector>
namespace fs = std::filesystem;
static auto Spawn(mp::EventLoop& loop, const std::string& process_argv0, const std::string& new_exe_name)
{
- int pid;
- const int fd = mp::SpawnProcess(pid, [&](int fd) -> std::vector<std::string> {
+ const auto [pid, socket] = mp::SpawnProcess([&](mp::SpawnConnectInfo info) -> std::vector<std::string> {
fs::path path = process_argv0;
path.remove_filename();
path.append(new_exe_name);
- return {path.string(), std::to_string(fd)};
+ return {path.string(), std::move(info)};
});
- return std::make_tuple(mp::ConnectStream<InitInterface>(loop, fd), pid);
+ return std::make_tuple(mp::ConnectStream<InitInterface>(loop, mp::MakeStream(loop, socket)), pid);
}
static void LogPrint(mp::LogMessage log_data)
@@ -55,6 +55,7 @@ int main(int argc, char** argv)
loop.loop();
});
mp::EventLoop* loop = promise.get_future().get();
+ mp::EventLoopRef loop_ref{*loop};
auto [printer_init, printer_pid] = Spawn(*loop, argv[0], "mpprinter");
auto [calc_init, calc_pid] = Spawn(*loop, argv[0], "mpcalculator");
@@ -71,6 +72,7 @@ int main(int argc, char** argv)
mp::WaitProcess(calc_pid);
printer_init.reset();
mp::WaitProcess(printer_pid);
+ loop_ref.reset();
loop_thread.join();
std::cout << "Bye!" << std::endl;
return 0;
### src/ipc/libmultiprocess/example/init.h
@@ -14,7 +14,7 @@ class Init
public:
virtual ~Init() = default;
virtual std::unique_ptr<Printer> makePrinter() { return nullptr; }
- virtual std::unique_ptr<Calculator> makeCalculator(std::unique_ptr<Printer> printer) { return nullptr; }
+ virtual std::unique_ptr<Calculator> makeCalculator(std::unique_ptr<Printer>) { return nullptr; }
};
#endif // EXAMPLE_INIT_H
### src/ipc/libmultiprocess/example/printer.cpp
@@ -7,18 +7,17 @@
#include <init.capnp.h>
#include <init.capnp.proxy.h> // NOLINT(misc-include-cleaner) // IWYU pragma: keep
-#include <charconv>
-#include <cstring>
+#include <cstring> // IWYU pragma: keep
#include <fstream>
#include <iostream>
#include <kj/async.h>
#include <kj/common.h>
#include <kj/memory.h>
#include <memory>
#include <mp/proxy-io.h>
+#include <mp/util.h>
#include <stdexcept>
#include <string>
-#include <system_error>
class PrinterImpl : public Printer
{
@@ -44,14 +43,10 @@ int main(int argc, char** argv)
std::cout << "Usage: mpprinter <fd>\n";
return 1;
}
- int fd;
- if (std::from_chars(argv[1], argv[1] + strlen(argv[1]), fd).ec != std::errc{}) {
- std::cerr << argv[1] << " is not a number or is larger than an int\n";
- return 1;
- }
+ mp::SocketId socket{mp::StartSpawned(argv[1])};
mp::EventLoop loop("mpprinter", LogPrint);
std::unique_ptr<Init> init = std::make_unique<InitImpl>();
- mp::ServeStream<InitInterface>(loop, fd, *init);
+ mp::ServeStream<InitInterface>(loop, mp::MakeStream(loop, socket), *init);
loop.loop();
return 0;
}
### src/ipc/libmultiprocess/include/mp/config.h.in
@@ -10,6 +10,7 @@
#cmakedefine HAVE_KJ_FILESYSTEM
#cmakedefine HAVE_PTHREAD_GETNAME_NP @HAVE_PTHREAD_GETNAME_NP@
+#cmakedefine HAVE_PTHREAD_SETNAME_NP_3ARG @HAVE_PTHREAD_SETNAME_NP_3ARG@
#cmakedefine HAVE_PTHREAD_THREADID_NP @HAVE_PTHREAD_THREADID_NP@
#cmakedefine HAVE_PTHREAD_GETTHREADID_NP @HAVE_PTHREAD_GETTHREADID_NP@
### src/ipc/libmultiprocess/include/mp/proxy-io.h
@@ -12,15 +12,18 @@
#include <capnp/rpc-twoparty.h>
+#include <any>
#include <assert.h>
#include <algorithm>
#include <condition_variable>
#include <cstdlib>
#include <functional>
#include <kj/function.h>
+#include <kj/io.h>
#include <map>
#include <memory>
#include <optional>
+#include <ranges>
#include <sstream>
#include <string>
#include <thread>
@@ -213,6 +216,9 @@ class Logger
std::string LongThreadName(const char* exe_name);
+//! Wrap a socket file descriptor as an async stream, taking ownership of the fd.
+Stream MakeStream(EventLoop&loop, SocketId socket);
+
//! Event loop implementation.
//!
//! Cap'n Proto threading model is very simple: all I/O operations are
@@ -229,8 +235,8 @@ std::string LongThreadName(const char* exe_name);
//! assumes it will only be accessed from one thread. So all this code needs to
//! actually run on one thread, and the EventLoop::loop() method is the entry point for
//! this thread. ProxyClient and ProxyServer objects that use other threads and
-//! need to perform I/O operations post to this thread using EventLoop::post()
-//! and EventLoop::sync() methods.
+//! need to perform I/O operations post to this thread using the
+//! EventLoop::sync() method.
//!
//! Specifically, because ProxyClient methods can be called from arbitrary
//! threads, and ProxyServer methods can run on arbitrary threads, ProxyClient
@@ -263,16 +269,7 @@ class EventLoop
//! Run function on event loop thread. Does not return until function completes.
//! Must be called while the loop() function is active.
- void post(kj::Function<void()> fn);
-
- //! Wrapper around EventLoop::post that takes advantage of the
- //! fact that callable will not go out of scope to avoid requirement that it
- //! be copyable.
- template <typename Callable>
- void sync(Callable&& callable)
- {
- post(std::forward<Callable>(callable));
- }
+ void sync(kj::FunctionParam<void()> fn);
//! Register cleanup function to run on asynchronous worker thread without
//! blocking the event loop thread.
@@ -294,6 +291,12 @@ class EventLoop
//! Check if loop should exit.
bool done() const MP_REQUIRES(m_mutex);
+ //! Type of m_incoming_connections list.
+ using Connections = std::list<Connection>;
+
+ //! View of incoming connections yielding Connection& for each entry.
+ auto incomingConnections() { return std::views::all(m_incoming_connections); }
+
//! Process name included in thread names so combined debug output from
//! multiple processes is easier to understand.
const char* m_exe_name;
@@ -305,17 +308,18 @@ class EventLoop
//! method has not been called.
std::thread m_async_thread;
- //! Callback function to run on event loop thread during post() or sync() call.
- kj::Function<void()>* m_post_fn MP_GUARDED_BY(m_mutex) = nullptr;
+ //! Callback function to run on event loop thread during sync() call.
+ kj::FunctionParam<void()>* m_sync_fn MP_GUARDED_BY(m_mutex) = nullptr;
//! Callback functions to run on async thread.
std::optional<CleanupList> m_async_fns MP_GUARDED_BY(m_mutex);
- //! Pipe read handle used to wake up the event loop thread.
- int m_wait_fd = -1;
+ //! Socket pair used to post and wait for wakeups to the event loop thread.
+ kj::Own<kj::AsyncIoStream> m_wait_stream;
+ kj::Own<kj::AsyncIoStream> m_post_stream;
- //! Pipe write handle used to wake up the event loop thread.
- int m_post_fd = -1;
+ //! Synchronous writer used to write to m_post_stream.
+ kj::Own<kj::OutputStream> m_post_writer;
//! Number of EventLoopRef instances referencing this event loop. This is a
//! sum of the number of client and server objects (Connection, ProxyClient,
@@ -340,7 +344,7 @@ class EventLoop
std::unique_ptr<kj::TaskSet> m_task_set;
//! List of connections.
- std::list<Connection> m_incoming_connections;
+ Connections m_incoming_connections;
//! Logging options
LogOptions m_log_opts;
@@ -369,6 +373,12 @@ class EventLoop
//! Hook called on the event loop thread when a client has disconnected.
std::function<void()> testing_hook_disconnected;
+
+ //! Miscellaneous testing hook. Called from various places with an
+ //! argument identifying the call site (typically a string literal), so
+ //! tests can control timing or inject behavior at specific points without
+ //! requiring a dedicated hook for each one.
+ std::function<void(std::any)> testing_hook_misc;
};
//! Single element task queue used to handle recursive capnp calls. (If the
@@ -451,7 +461,7 @@ class Connection
//! Capability::Client handles owned by ProxyClient objects), then schedules
//! asynchronous cleanup functions to run in a worker thread (to run
//! destructors of m_impl instances owned by ProxyServer objects).
- ~Connection();
+ ~Connection() noexcept(false);
//! Register synchronous cleanup function to run on event loop thread (with
//! access to capnp thread local variables) when disconnect() is called.
@@ -587,7 +597,28 @@ ProxyClientBase<Interface, Impl>::ProxyClientBase(typename Interface::Client cli
});
}
});
- Sub::construct(*this);
+ // If construct() fails, run the cleanup functions before rethrowing,
+ // because ~ProxyClientBase will not run for an object whose constructor
+ // threw, and the connection would otherwise be leaked.
+ try {
+ Sub::construct(*this);
+ } catch (...) {
+ MP_LOG(*m_context.loop, Log::Debug) << "Cleaning up " << CxxTypeName(*this) << " " << this << " after construct() failure";
+ CleanupRun(m_context.cleanup_fns);
+ throw;
+ }
+
+ // If this client owns the connection, delete the connection on disconnect.
+ if (destroy_connection) {
+ m_context.loop->sync([&] {
+ EventLoop& loop = *m_context.loop;
+ Connection* connection = m_context.connection;
+ connection->onDisconnect([&loop, connection] {
+ MP_LOG(loop, Log::Warning) << "IPC client: unexpected network disconnect.";
+ delete connection;
+ });
+ });
+ }
}
template <typename Interface, typename Impl>
@@ -689,7 +720,7 @@ using ConnThread = ConnThreads::iterator;
// inserted bool.
std::tuple<ConnThread, bool> SetThread(GuardedRef<ConnThreads> threads, Connection* connection, const std::function<Thread::Client()>& make_thread);
-//! The thread_local ThreadContext g_thread_context struct provides information
+//! The thread_local ThreadContext struct (see CurrentThread()) provides information
//! about individual threads and a way of communicating between them. Because
//! it's a thread local struct, each ThreadContext instance is initialized by
//! the thread that owns it.
@@ -824,51 +855,84 @@ kj::Promise<T> ProxyServer<Thread>::post(Fn&& fn)
return ret;
}
-//! Given stream file descriptor, make a new ProxyClient object to send requests
-//! over the stream. Also create a new Connection object embedded in the
-//! client that is freed when the client is closed.
+//! Given a stream, make a new ProxyClient object to send requests over it.
+//! Also create a new Connection object embedded in the client that is freed
+//! when the client is closed.
+//!
+//! If the init interface declares a construct() method, creating the client
+//! calls it, so this function may block making an IPC call and may throw if
+//! the call fails.
+//!
+//! If destroy_connection is false, the returned client does not take
+//! ownership of the connection, and the caller is responsible for
+//! disconnecting and freeing it (accessible as client->m_context.connection)
+//! whenever it is done with it.
template <typename InitInterface>
-std::unique_ptr<ProxyClient<InitInterface>> ConnectStream(EventLoop& loop, int fd)
+std::unique_ptr<ProxyClient<InitInterface>> ConnectStream(EventLoop& loop, Stream stream, bool destroy_connection = true)
{
typename InitInterface::Client init_client(nullptr);
std::unique_ptr<Connection> connection;
loop.sync([&] {
- auto stream =
- loop.m_io_context.lowLevelProvider->wrapSocketFd(fd, kj::LowLevelAsyncIoProvider::TAKE_OWNERSHIP);
connection = std::make_unique<Connection>(loop, kj::mv(stream));
init_client = connection->m_rpc_system->bootstrap(ServerVatId().vat_id).castAs<InitInterface>();
- Connection* connection_ptr = connection.get();
- connection->onDisconnect([&loop, connection_ptr] {
- MP_LOG(loop, Log::Warning) << "IPC client: unexpected network disconnect.";
- delete connection_ptr;
- });
});
- return std::make_unique<ProxyClient<InitInterface>>(
- kj::mv(init_client), connection.release(), /* destroy_connection= */ true);
+ return std::make_unique<ProxyClient<InitInterface>>(kj::mv(init_client), connection.release(), destroy_connection);
}
//! Given stream and init objects, construct a new ProxyServer object that
//! handles requests from the stream by calling the init object. Embed the
-//! ProxyServer in a Connection object that is stored and erased if
-//! disconnected. This should be called from the event loop thread.
+//! ProxyServer in a Connection object that is stored in
+//! loop.m_incoming_connections. This should be called from the event loop
+//! thread. Returns the new ProxyServer along with an iterator to its
+//! Connection.
+//!
+//! If destroy_connection is false, the connection is not erased
+//! automatically when the peer disconnects, and the caller is responsible
+//! for erasing loop.m_incoming_connections at the returned iterator
+//! whenever it is done with the connection.
template <typename InitInterface, typename InitImpl, typename OnDisconnect>
-void _Serve(EventLoop& loop, kj::Own<kj::AsyncIoStream>&& stream, InitImpl& init, OnDisconnect&& on_disconnect)
+std::pair<ProxyServer<InitInterface>*, EventLoop::Connections::iterator> _Serve(EventLoop& loop,
+ kj::Own<kj::AsyncIoStream>&& stream,
+ std::shared_ptr<InitImpl> init,
+ OnDisconnect&& on_disconnect,
+ bool destroy_connection = true)
{
+ ProxyServer<InitInterface>* server = nullptr;
loop.m_incoming_connections.emplace_front(loop, kj::mv(stream), [&](Connection& connection) {
- // Disable deleter so proxy server object doesn't attempt to delete the
- // init implementation when the proxy client is destroyed or
- // disconnected.
- return kj::heap<ProxyServer<InitInterface>>(std::shared_ptr<InitImpl>(&init, [](InitImpl*){}), connection);
+ auto proxy_server = kj::heap<ProxyServer<InitInterface>>(std::move(init), connection);
+ server = proxy_server;
+ return capnp::Capability::Client(kj::mv(proxy_server));
});
auto it = loop.m_incoming_connections.begin();
MP_LOG(loop, Log::Info) << "IPC server: socket connected.";
if (loop.testing_hook_connected) loop.testing_hook_connected();
- it->onDisconnect([&loop, it, on_disconnect = std::forward<OnDisconnect>(on_disconnect)]() mutable {
+ it->onDisconnect([&loop, it, on_disconnect = std::forward<OnDisconnect>(on_disconnect), destroy_connection]() mutable {
MP_LOG(loop, Log::Info) << "IPC server: socket disconnected.";
- loop.m_incoming_connections.erase(it);
+ if (destroy_connection) loop.m_incoming_connections.erase(it);
on_disconnect();
if (loop.testing_hook_disconnected) loop.testing_hook_disconnected();
});
+ return {server, it};
+}
+
+//! Overload of _Serve that takes a reference to the init object instead of a
+//! shared_ptr. ProxyServer objects use shared_ptr's internally to optionally
+//! take ownership of interfaces being served, and free them when clients are
+//! no longer using them. Some libmultiprocess callers take advantage of this
+//! and pass shared_ptr's directly. But other callers that do not give away
+//! ownership are not required to use shared_ptr, so this overload converts
+//! references they pass into shared_ptr's with empty deleters. This prevents
+//! the ProxyServer object from deleting the init object when the client is
+//! disconnected.
+template <typename InitInterface, typename InitImpl, typename OnDisconnect>
+std::pair<ProxyServer<InitInterface>*, EventLoop::Connections::iterator> _Serve(EventLoop& loop,
+ kj::Own<kj::AsyncIoStream>&& stream,
+ InitImpl& init,
+ OnDisconnect&& on_disconnect,
+ bool destroy_connection = true)
+{
+ return _Serve<InitInterface>(loop, kj::mv(stream), std::shared_ptr<InitImpl>(&init, [](InitImpl*){}),
+ std::forward<OnDisconnect>(on_disconnect), destroy_connection);
}
struct Listener
@@ -905,22 +969,20 @@ void _Listen(const std::shared_ptr<Listener>& listener, EventLoop& loop, InitImp
}));
}
-//! Given stream file descriptor and an init object, handle requests on the
-//! stream by calling methods on the Init object.
+//! Given a stream and an init object, handle requests on the stream by calling
+//! methods on the Init object. See _Serve for details on the return value and
+//! the destroy_connection parameter.
template <typename InitInterface, typename InitImpl>
-void ServeStream(EventLoop& loop, int fd, InitImpl& init)
+std::pair<ProxyServer<InitInterface>*, EventLoop::Connections::iterator> ServeStream(
+ EventLoop& loop, Stream stream, InitImpl&& init, bool destroy_connection = true)
{
- _Serve<InitInterface>(
- loop,
- loop.m_io_context.lowLevelProvider->wrapSocketFd(fd, kj::LowLevelAsyncIoProvider::TAKE_OWNERSHIP),
- init,
- [] {});
+ return _Serve<InitInterface>(loop, kj::mv(stream), std::forward<InitImpl>(init), /*on_disconnect=*/ [] {}, destroy_connection);
}
-//! Given listening socket file descriptor and an init object, handle incoming
+//! Given listening socket identifier and an init object, handle incoming
//! connections and requests by calling methods on the Init object.
template <typename InitInterface, typename InitImpl>
-void ListenConnections(EventLoop& loop, int fd, InitImpl& init, std::optional<size_t> max_connections = std::nullopt)
+void ListenConnections(EventLoop& loop, SocketId fd, InitImpl& init, std::optional<size_t> max_connections = std::nullopt)
{
loop.sync([&]() {
auto listener{std::make_shared<Listener>(
@@ -935,6 +997,26 @@ extern thread_local ThreadContext g_thread_context; // NOLINT(bitcoin-nontrivial
// cannot be thread_local" which should not be a problem on modern platforms, and
// could lead to a small memory leak at worst on older ones.
+//! Return the current thread's ThreadContext.
+//!
+//! Why per-thread state is needed at all: libmultiprocess has no control over
+//! which threads the C++ application uses to call ProxyClient methods after
+//! the proxy objects are returned to it. The thread-mapping model (see
+//! "Thread Mapping" in doc/design.md) gives each application thread making
+//! IPC calls a dedicated server-side thread that executes its requests, so
+//! thread-local state and recursive mutexes work as expected across the
+//! process boundary and callbacks from the server run on the originating
+//! client thread. The client-side handles for those dedicated server threads
+//! (the ProxyClient<Thread> objects returned by ThreadMap.makeThread, stored
+//! per connection in the request_threads / callback_threads maps below) are
+//! state that must be keyed implicitly by the calling thread, and must be
+//! released when the client thread exits so the corresponding server threads
+//! are freed. A thread_local object is the C++ mechanism that provides both
+//! of these: per-thread storage plus a destructor that runs at thread exit
+//! (the C equivalent would be a pthread key destructor). This is why
+//! ThreadContext is thread_local and why its destructor is nontrivial.
+ThreadContext& CurrentThread();
+
} // namespace mp
#endif // MP_PROXY_IO_H
### src/ipc/libmultiprocess/include/mp/proxy-types.h
@@ -13,6 +13,12 @@
#include <typeindex>
#include <vector>
+// KJ exception support is required, or exceptions thrown in serverInvoke below go uncaught
+// and crash the process instead of being returned to the client as errors.
+#if defined(KJ_NO_EXCEPTIONS) && KJ_NO_EXCEPTIONS
+#error "KJ_NO_EXCEPTIONS=1 is set but libmultiprocess requires support for exceptions. Please check build settings. On NetBSD you may also need to set -DKJ_NO_EXCEPTIONS=0 -DKJ_NO_RTTI=0 explicitly (https://github.com/capnproto/capnproto/pull/2756)"
+#endif
+
namespace mp {
template <typename Value>
@@ -204,7 +210,7 @@ struct ReadDestUpdate
//! can only be used to disable ReadField() calls in certain contexts like
//! std::optional and pointer contexts.
template <typename... LocalTypes, typename Input>
-bool CustomHasField(TypeList<LocalTypes...>, InvokeContext& invoke_context, const Input& input)
+bool CustomHasField(TypeList<LocalTypes...>, InvokeContext&, const Input& input)
{
return input.has();
}
@@ -220,14 +226,14 @@ void ThrowField(TypeList<LocalType>, InvokeContext& invoke_context, Input&& inpu
{
ReadField(
TypeList<LocalType>(), invoke_context, input, ReadDestEmplace(TypeList<LocalType>(),
- [](auto&& ...args) -> const LocalType& { throw LocalType{std::forward<decltype(args)>(args)...}; }));
+ [] [[noreturn]] (auto&& ...args) -> const LocalType& { throw LocalType{std::forward<decltype(args)>(args)...}; }));
}
//! Special case for generic std::exception. It's an abstract type so it can't
//! be created directly. Rethrow as std::runtime_error so callers expecting it
//! will still catch it.
template <typename Input>
-void ThrowField(TypeList<std::exception>, InvokeContext& invoke_context, Input&& input)
+void ThrowField(TypeList<std::exception>, InvokeContext&, Input&& input)
{
auto data = input.get();
throw std::runtime_error(std::string(CharCast(data.begin()), data.size()));
@@ -241,7 +247,7 @@ void ThrowField(TypeList<std::exception>, InvokeContext& invoke_context, Input&&
//! way to represent them in Cap'n Proto and a null Data field is a convenient
//! representation.
template <typename... Values>
-bool CustomHasValue(InvokeContext& invoke_context, const Values&... value)
+bool CustomHasValue(InvokeContext&, const Values&...)
{
return true;
}
@@ -285,9 +291,28 @@ void BuildList(TypeList<LocalType>, InvokeContext& invoke_context, Output&& outp
{
auto list = output.init(value.size());
size_t i = 0;
- for (const auto& elem : value) {
- BuildField(TypeList<LocalType>(), invoke_context, ListOutput<typename decltype(list)::Builds>(list, i), elem);
- ++i;
+ // Iterate with an explicit iterator rather than a range-for loop so the
+ // value category of `*it` is passed through to BuildField unchanged. This
+ // matters for two reasons:
+ //
+ // - Elements must be passed as non-const so that BuildField can move out of
+ // them, e.g. calling unique_ptr::release() to transfer ownership of an
+ // interface pointer to the capnp server.
+ //
+ // - For proxy containers like std::vector<bool>, `*it` is a prvalue proxy
+ // object rather than a reference. A range-for loop would bind it to a
+ // named variable and demote it to an lvalue; passing `*it` directly
+ // preserves the prvalue-ness.
+ //
+ // Only move out of elements when `value` itself is an rvalue container that
+ // is about to be destroyed. When it is an lvalue reference owned by the
+ // caller, pass elements as lvalues so BuildField does not move from them.
+ for (auto it = value.begin(); it != value.end(); ++it, ++i) {
+ if constexpr (std::is_lvalue_reference_v<Value&&>) {
+ BuildField(TypeList<LocalType>(), invoke_context, ListOutput<typename decltype(list)::Builds>(list, i), *it);
+ } else {
+ BuildField(TypeList<LocalType>(), invoke_context, ListOutput<typename decltype(list)::Builds>(list, i), std::move(*it));
+ }
}
}
@@ -434,7 +459,7 @@ struct ClientException
struct BuildParams : IterateFieldsHelper<BuildParams, 0>
{
template <typename Params, typename ParamList>
- void handleField(InvokeContext& invoke_context, Params& params, ParamList)
+ void handleField(InvokeContext&, Params&, ParamList)
{
}
@@ -599,7 +624,8 @@ struct ServerExcept : Parent
//! Helper for CustomPassField below. Call Accessor::get method if it has one,
//! otherwise return capnp::Void.
template <typename Accessor, typename Message>
-decltype(auto) MaybeGet(Message&& message, decltype(Accessor::get(message))* enable = nullptr)
+decltype(auto) MaybeGet(Message&& message)
+ requires requires { Accessor::get(message); }
{
return Accessor::get(message);
}
@@ -627,7 +653,7 @@ template <typename Accessor, typename... Args>
auto PassField(Priority<2>, Args&&... args) -> decltype(CustomPassField<Accessor>(std::forward<Args>(args)...))
{
return CustomPassField<Accessor>(std::forward<Args>(args)...);
-};
+}
template <int argc, typename Accessor, typename Parent>
struct ServerField : Parent
@@ -691,9 +717,9 @@ void serverDestroy(Server& server)
template <typename ProxyClient, typename GetRequest, typename... FieldObjs>
void clientInvoke(ProxyClient& proxy_client, const GetRequest& get_request, FieldObjs&&... fields)
{
- if (!g_thread_context.waiter) {
- assert(g_thread_context.thread_name.empty());
- g_thread_context.thread_name = ThreadName(proxy_client.m_context.loop->m_exe_name);
+ if (!CurrentThread().waiter) {
+ assert(CurrentThread().thread_name.empty());
+ CurrentThread().thread_name = ThreadName(proxy_client.m_context.loop->m_exe_name);
// If next assert triggers, it means clientInvoke is being called from
// the capnp event loop thread. This can happen when a ProxyServer
// method implementation that runs synchronously on the event loop
@@ -702,13 +728,13 @@ void clientInvoke(ProxyClient& proxy_client, const GetRequest& get_request, Fiel
// run asynchronously off the event loop thread. This is easy to fix by
// just adding a 'context :Proxy.Context' argument to the capnp method
// declaration so the server method runs in a dedicated thread.
- assert(!g_thread_context.loop_thread);
- g_thread_context.waiter = std::make_unique<Waiter>();
+ assert(!CurrentThread().loop_thread);
+ CurrentThread().waiter = std::make_unique<Waiter>();
MP_LOGPLAIN(*proxy_client.m_context.loop, Log::Info)
- << "{" << g_thread_context.thread_name
+ << "{" << CurrentThread().thread_name
<< "} IPC client first request from current thread, constructing waiter";
}
- ThreadContext& thread_context{g_thread_context};
+ ThreadContext& thread_context{CurrentThread()};
std::optional<ClientInvokeContext> invoke_context; // Must outlive waiter->wait() call below
std::exception_ptr exception;
std::string kj_exception;
### src/ipc/libmultiprocess/include/mp/proxy.h
@@ -44,7 +44,9 @@ inline void CleanupRun(CleanupList& fns) {
}
}
-//! Event loop smart pointer automatically managing m_num_refs.
+//! Event loop smart pointer automatically managing m_num_refs. Hold one
+//! reference to keep the loop running, once the last reference is released
+//! EventLoop::loop() will exit.
//! If a lock pointer argument is passed, the specified lock will be used,
//! otherwise EventLoop::m_mutex will be locked when needed.
class EventLoopRef
@@ -315,21 +317,21 @@ template <typename Field, int flags>
struct Accessor : public Field
{
//! Field is present from the Cap'n Proto Params struct (client -> server).
- static const bool in = flags & FIELD_IN;
+ static constexpr bool in = (flags & FIELD_IN) != 0;
//! Field is present from the Cap'n Proto Results struct (server -> client).
- static const bool out = flags & FIELD_OUT;
+ static constexpr bool out = (flags & FIELD_OUT) != 0;
//! Field has a companion has{Name} boolean field in the Cap'n Proto struct.
//! This is used to represent optional primitive values (e.g. C++
//! std::optional<int>) because Cap'n Proto doesn't allow primitive fields to
//! be unset.
- static const bool optional = flags & FIELD_OPTIONAL;
+ static constexpr bool optional = (flags & FIELD_OPTIONAL) != 0;
//! Results field has a companion want{Name} boolean field in the Params
//! struct. Used for optional output parameters (e.g. C++ int*) and set to
//! true if the caller passed a non-null pointer and wants the result.
- static const bool requested = flags & FIELD_REQUESTED;
+ static constexpr bool requested = (flags & FIELD_REQUESTED) != 0;
//! Field is a Cap'n Proto pointer type (struct, list, text, data,
//! interface) as opposed to a primitive type (bool, int, float, enum).
- static const bool boxed = flags & FIELD_BOXED;
+ static constexpr bool boxed = (flags & FIELD_BOXED) != 0;
};
//! Wrapper around std::function for passing std::function objects between client and servers.
### src/ipc/libmultiprocess/include/mp/type-char.h
@@ -7,6 +7,10 @@
#include <mp/util.h>
+#include <algorithm>
+#include <ranges>
+#include <stdexcept>
+
namespace mp {
template <typename Output, size_t size>
void CustomBuildField(TypeList<const unsigned char*>,
@@ -16,7 +20,7 @@ void CustomBuildField(TypeList<const unsigned char*>,
Output&& output)
{
auto result = output.init(size);
- memcpy(result.begin(), value, size);
+ std::ranges::copy(value, result.begin());
}
template <size_t size, typename Input, typename ReadDest>
@@ -26,9 +30,11 @@ decltype(auto) CustomReadField(TypeList<unsigned char[size]>,
Input&& input,
ReadDest&& read_dest)
{
+ auto data = input.get();
+ if (data.size() != size) throw std::range_error("unexpected field size");
+
return read_dest.update([&](auto& value) {
- auto data = input.get();
- memcpy(value, data.begin(), size);
+ std::ranges::copy(input.get(), std::ranges::begin(value));
});
}
} // namespace mp
### src/ipc/libmultiprocess/include/mp/type-chrono.h
@@ -13,12 +13,12 @@ namespace mp {
//! Overload CustomBuildField and CustomReadField to serialize std::chrono
//! parameters and return values as numbers.
template <class Rep, class Period, typename Value, typename Output>
-void CustomBuildField(TypeList<std::chrono::duration<Rep, Period>>, Priority<1>, InvokeContext& invoke_context, Value&& value,
+void CustomBuildField(TypeList<std::chrono::duration<Rep, Period>>, Priority<1>, InvokeContext&, Value&& value,
Output&& output)
{
- static_assert(std::numeric_limits<decltype(output.get())>::lowest() <= std::numeric_limits<Rep>::lowest(),
+ static_assert(safe_less_equal(std::numeric_limits<decltype(output.get())>::lowest(), std::numeric_limits<Rep>::lowest()),
"capnp type does not have enough range to hold lowest std::chrono::duration value");
- static_assert(std::numeric_limits<decltype(output.get())>::max() >= std::numeric_limits<Rep>::max(),
+ static_assert(safe_greater_equal(std::numeric_limits<decltype(output.get())>::max(), std::numeric_limits<Rep>::max()),
"capnp type does not have enough range to hold highest std::chrono::duration value");
output.set(value.count());
}
@@ -29,6 +29,29 @@ decltype(auto) CustomReadField(TypeList<std::chrono::duration<Rep, Period>>, Pri
{
return read_dest.construct(input.get());
}
+
+//! Overload CustomBuildField and CustomReadField to serialize
+//! std::chrono::time_point parameters and return values as integer tick counts.
+//! The capnp field type must be an integer type with enough range to hold the
+//! time_since_epoch() count for the given Duration (e.g. Int64 for nanoseconds).
+template <class Clock, class Duration, typename Value, typename Output>
+void CustomBuildField(TypeList<std::chrono::time_point<Clock, Duration>>, Priority<1>, InvokeContext&,
+ Value&& value, Output&& output)
+{
+ using Rep = typename Duration::rep;
+ static_assert(safe_less_equal(std::numeric_limits<decltype(output.get())>::lowest(), std::numeric_limits<Rep>::lowest()),
+ "capnp type does not have enough range to hold lowest std::chrono::time_point value");
+ static_assert(safe_greater_equal(std::numeric_limits<decltype(output.get())>::max(), std::numeric_limits<Rep>::max()),
+ "capnp type does not have enough range to hold highest std::chrono::time_point value");
+ output.set(value.time_since_epoch().count());
+}
+
+template <class Clock, class Duration, typename Input, typename ReadDest>
+decltype(auto) CustomReadField(TypeList<std::chrono::time_point<Clock, Duration>>, Priority<1>,
+ InvokeContext& invoke_context, Input&& input, ReadDest&& read_dest)
+{
+ return read_dest.construct(std::chrono::time_point<Clock, Duration>{Duration{input.get()}});
+}
} // namespace mp
#endif // MP_PROXY_TYPE_CHRONO_H
### src/ipc/libmultiprocess/include/mp/type-context.h
@@ -12,11 +12,11 @@
namespace mp {
template <typename Output>
+ requires FieldTypeIs<Output, Context::Builder>
void CustomBuildField(TypeList<>,
Priority<1>,
ClientInvokeContext& invoke_context,
- Output&& output,
- typename std::enable_if<std::is_same<decltype(output.get()), Context::Builder>::value>::type* enable = nullptr)
+ Output&& output)
{
auto& connection = invoke_context.connection;
auto& thread_context = invoke_context.thread_context;
@@ -93,7 +93,7 @@ auto PassField(Priority<1>, TypeList<>, ServerContext& server_context, const Fn&
// call. In this case, the callbackThread value should point
// to the same thread already in the map, so there is no
// need to update the map.
- auto& thread_context = g_thread_context;
+ auto& thread_context = CurrentThread();
auto& request_threads = thread_context.request_threads;
ConnThread request_thread;
bool inserted{false};
@@ -131,6 +131,18 @@ auto PassField(Priority<1>, TypeList<>, ServerContext& server_context, const Fn&
std::tie(request_thread, inserted) = SetThread(
GuardedRef{thread_context.waiter->m_mutex, request_threads}, server.m_context.connection,
[&] { return Accessor::get(call_context.getParams()).getCallbackThread(); });
+ // Initialize the request's results struct here on the event loop
+ // thread, so later getResults() calls on the execution thread
+ // return the response capnp caches on first use instead of
+ // reading Cap'n Proto connection state, which is unsafe off the
+ // event loop thread because RpcConnectionState::disconnect()
+ // overwrites it there without synchronization on an abrupt remote
+ // disconnect (bitcoin-core/libmultiprocess#348). After this call,
+ // the only call_context state accessed by the execution thread is
+ // the params reader and the results struct allocated here; any
+ // future change accessing other call_context state needs to move
+ // that access to the event loop thread the same way.
+ call_context.getResults();
});
// If an entry was inserted into the request_threads map,
### src/ipc/libmultiprocess/include/mp/type-data.h
@@ -29,16 +29,15 @@ concept IsByteSpan =
//! blob types like uint256 or PKHash with data() and size() methods pointing to
//! bytes.
template <typename LocalType, typename Value, typename Output>
-void CustomBuildField(TypeList<LocalType>, Priority<2>, InvokeContext& invoke_context, Value&& value, Output&& output)
+void CustomBuildField(TypeList<LocalType>, Priority<2>, InvokeContext&, Value&& value, Output&& output)
requires (std::is_same_v<decltype(output.get()), ::capnp::Data::Builder> && IsByteSpan<LocalType>)
{
- auto data = std::span{value};
- auto result = output.init(data.size());
- std::ranges::copy(data, result.begin());
+ auto result = output.init(value.size());
+ std::ranges::copy(value, result.begin());
}
template <typename LocalType, typename Input, typename ReadDest>
-decltype(auto) CustomReadField(TypeList<LocalType>, Priority<2>, InvokeContext& invoke_context, Input&& input, ReadDest&& read_dest)
+decltype(auto) CustomReadField(TypeList<LocalType>, Priority<2>, InvokeContext&, Input&& input, ReadDest&& read_dest)
requires (std::is_same_v<decltype(input.get()), ::capnp::Data::Reader> && IsByteSpan<LocalType>)
{
using ByteType = decltype(std::span{std::declval<LocalType>().begin(), std::declval<LocalType>().end()})::element_type;
### src/ipc/libmultiprocess/include/mp/type-function.h
@@ -61,7 +61,7 @@ decltype(auto) CustomReadField(TypeList<std::function<FnR(FnParams...)>> types,
return read_dest.construct(ProxyCallFn<decltype(client)>{std::move(client)});
}
return read_dest.construct();
-};
+}
} // namespace mp
#endif // MP_PROXY_TYPE_FUNCTION_H
### src/ipc/libmultiprocess/include/mp/type-interface.h
@@ -21,12 +21,12 @@ kj::Own<typename Interface::Server> CustomMakeProxyServer(InvokeContext& context
}
template <typename Impl, typename Value, typename Output>
+ requires InterfaceField<Output>
void CustomBuildField(TypeList<std::unique_ptr<Impl>>,
Priority<1>,
InvokeContext& invoke_context,
Value&& value,
- Output&& output,
- typename Decay<decltype(output.get())>::Calls* enable = nullptr)
+ Output&& output)
{
if (value) {
using Interface = typename decltype(output.get())::Calls;
@@ -35,12 +35,12 @@ void CustomBuildField(TypeList<std::unique_ptr<Impl>>,
}
template <typename Impl, typename Value, typename Output>
+ requires InterfaceField<Output>
void CustomBuildField(TypeList<std::shared_ptr<Impl>>,
Priority<2>,
InvokeContext& invoke_context,
Value&& value,
- Output&& output,
- typename Decay<decltype(output.get())>::Calls* enable = nullptr)
+ Output&& output)
{
if (value) {
using Interface = typename decltype(output.get())::Calls;
@@ -49,12 +49,12 @@ void CustomBuildField(TypeList<std::shared_ptr<Impl>>,
}
template <typename Impl, typename Output>
+ requires InterfaceField<Output>
void CustomBuildField(TypeList<Impl&>,
Priority<1>,
InvokeContext& invoke_context,
Impl& value,
- Output&& output,
- typename decltype(output.get())::Calls* enable = nullptr)
+ Output&& output)
{
// Disable deleter so proxy server object doesn't attempt to delete the
// wrapped implementation when the proxy client is destroyed or
@@ -77,12 +77,12 @@ std::unique_ptr<Impl> CustomMakeProxyClient(InvokeContext& context, typename Int
}
template <typename LocalType, typename Input, typename ReadDest>
+ requires InterfaceField<Input>
decltype(auto) CustomReadField(TypeList<std::unique_ptr<LocalType>>,
Priority<1>,
InvokeContext& invoke_context,
Input&& input,
- ReadDest&& read_dest,
- typename Decay<decltype(input.get())>::Calls* enable = nullptr)
+ ReadDest&& read_dest)
{
using Interface = typename Decay<decltype(input.get())>::Calls;
if (CustomHasField(TypeList<LocalType>(), invoke_context, input)) {
@@ -93,12 +93,12 @@ decltype(auto) CustomReadField(TypeList<std::unique_ptr<LocalType>>,
}
template <typename LocalType, typename Input, typename ReadDest>
+ requires InterfaceField<Input>
decltype(auto) CustomReadField(TypeList<std::shared_ptr<LocalType>>,
Priority<1>,
InvokeContext& invoke_context,
Input&& input,
- ReadDest&& read_dest,
- typename Decay<decltype(input.get())>::Calls* enable = nullptr)
+ ReadDest&& read_dest)
{
using Interface = typename Decay<decltype(input.get())>::Calls;
if (CustomHasField(TypeList<LocalType>(), invoke_context, input)) {
### src/ipc/libmultiprocess/include/mp/type-map.h
@@ -9,9 +9,11 @@
#include <mp/type-pair.h>
#include <mp/util.h>
+#include <map>
+
namespace mp {
-template <typename KeyLocalType, typename ValueLocalType, typename Value, typename Output>
-void CustomBuildField(TypeList<std::map<KeyLocalType, ValueLocalType>>,
+template <typename KeyLocalType, typename ValueLocalType, typename Compare, typename Allocator, typename Value, typename Output>
+void CustomBuildField(TypeList<std::map<KeyLocalType, ValueLocalType, Compare, Allocator>>,
Priority<1>,
InvokeContext& invoke_context,
Value&& value,
@@ -49,8 +51,8 @@ auto EmplacePiecewiseSafe(
}
}
-template <typename KeyLocalType, typename ValueLocalType, typename Input, typename ReadDest>
-decltype(auto) CustomReadField(TypeList<std::map<KeyLocalType, ValueLocalType>>,
+template <typename KeyLocalType, typename ValueLocalType, typename Compare, typename Allocator, typename Input, typename ReadDest>
+decltype(auto) CustomReadField(TypeList<std::map<KeyLocalType, ValueLocalType, Compare, Allocator>>,
Priority<1>,
InvokeContext& invoke_context,
Input&& input,
### src/ipc/libmultiprocess/include/mp/type-message.h
@@ -13,8 +13,8 @@ namespace mp {
//! CustomBuildField overload because it only requires defining a normal
//! function, not a template function, but less flexible.
template <typename LocalType, typename Value, typename Output>
-void CustomBuildField(TypeList<LocalType>, Priority<2>, InvokeContext& invoke_context, Value&& value, Output&& output,
- decltype(CustomBuildMessage(invoke_context, value, std::move(output.get())))* enable = nullptr)
+void CustomBuildField(TypeList<LocalType>, Priority<2>, InvokeContext& invoke_context, Value&& value, Output&& output)
+ requires requires { CustomBuildMessage(invoke_context, value, std::move(output.get())); }
{
CustomBuildMessage(invoke_context, value, std::move(output.init()));
}
@@ -25,17 +25,17 @@ void CustomBuildField(TypeList<LocalType>, Priority<2>, InvokeContext& invoke_co
//! function, not a template function, but less flexible.
template <typename LocalType, typename Input, typename ReadDest>
decltype(auto) CustomReadField(TypeList<LocalType>, Priority<2>, InvokeContext& invoke_context, Input&& input,
- ReadDest&& read_dest,
- decltype(CustomReadMessage(invoke_context, input.get(),
- std::declval<LocalType&>()))* enable = nullptr)
+ ReadDest&& read_dest)
+ requires requires { CustomReadMessage(invoke_context, input.get(), std::declval<LocalType&>()); }
{
return read_dest.update([&](auto& value) { if (CustomHasField(TypeList<LocalType>(), invoke_context, input)) CustomReadMessage(invoke_context, input.get(), value); });
}
//! Helper for CustomPassField below. Call Accessor::init method if it has one,
//! otherwise do nothing.
template <typename Accessor, typename Message>
-decltype(auto) MaybeInit(Message&& message, decltype(Accessor::get(message))* enable = nullptr)
+decltype(auto) MaybeInit(Message&& message)
+ requires requires { Accessor::get(message); }
{
return Accessor::init(message);
}
### src/ipc/libmultiprocess/include/mp/type-number.h
@@ -6,51 +6,60 @@
#define MP_PROXY_TYPE_NUMBER_H
#include <mp/util.h>
+#include <utility>
namespace mp {
template <typename LocalType, typename Value>
-LocalType BuildPrimitive(InvokeContext& invoke_context,
+ requires std::is_enum_v<Value>
+LocalType BuildPrimitive(InvokeContext&,
const Value& value,
- TypeList<LocalType>,
- typename std::enable_if<std::is_enum<Value>::value>::type* enable = nullptr)
+ TypeList<LocalType>)
{
using E = std::make_unsigned_t<std::underlying_type_t<Value>>;
using T = std::make_unsigned_t<LocalType>;
- static_assert(std::numeric_limits<T>::max() >= std::numeric_limits<E>::max(), "mismatched integral/enum types");
+ static_assert(std::cmp_greater_equal(std::numeric_limits<T>::max(), std::numeric_limits<E>::max()), "mismatched integral/enum types");
return static_cast<LocalType>(value);
}
template <typename LocalType, typename Value>
-LocalType BuildPrimitive(InvokeContext& invoke_context,
+ requires (std::is_integral_v<Value> && !std::is_same_v<std::remove_cv_t<Value>, bool>)
+LocalType BuildPrimitive(InvokeContext&,
const Value& value,
- TypeList<LocalType>,
- typename std::enable_if<std::is_integral<Value>::value, int>::type* enable = nullptr)
+ TypeList<LocalType>)
{
static_assert(
- std::numeric_limits<LocalType>::lowest() <= std::numeric_limits<Value>::lowest(), "mismatched integral types");
+ std::cmp_less_equal(std::numeric_limits<LocalType>::lowest(), std::numeric_limits<Value>::lowest()), "mismatched integral types");
static_assert(
- std::numeric_limits<LocalType>::max() >= std::numeric_limits<Value>::max(), "mismatched integral types");
+ std::cmp_greater_equal(std::numeric_limits<LocalType>::max(), std::numeric_limits<Value>::max()), "mismatched integral types");
+ return value;
+}
+
+template <typename LocalType>
+LocalType BuildPrimitive(InvokeContext&, bool value, TypeList<LocalType>)
+{
+ static_assert(std::is_same_v<LocalType, bool>,
+ "capnp field type should be Bool for bool parameters. Fix the .capnp schema.");
return value;
}
template <typename LocalType, typename Value>
-LocalType BuildPrimitive(InvokeContext& invoke_context,
+ requires std::is_floating_point_v<Value>
+LocalType BuildPrimitive(InvokeContext&,
const Value& value,
- TypeList<LocalType>,
- typename std::enable_if<std::is_floating_point<Value>::value>::type* enable = nullptr)
+ TypeList<LocalType>)
{
static_assert(std::is_same<Value, LocalType>::value,
"mismatched floating point types. please fix message.capnp type declaration to match wrapped interface");
return value;
}
template <typename LocalType, typename Input, typename ReadDest>
+ requires std::is_enum_v<LocalType>
decltype(auto) CustomReadField(TypeList<LocalType>,
Priority<1>,
- InvokeContext& invoke_context,
+ InvokeContext&,
Input&& input,
- ReadDest&& read_dest,
- typename std::enable_if<std::is_enum<LocalType>::value>::type* enable = nullptr)
+ ReadDest&& read_dest)
{
// Disable clang-tidy out-of-range enum value check which triggers when
// using an enum type that does not have a 0 value. The check correctly
@@ -62,12 +71,12 @@ decltype(auto) CustomReadField(TypeList<LocalType>,
}
template <typename LocalType, typename Input, typename ReadDest>
+ requires std::is_integral_v<LocalType>
decltype(auto) CustomReadField(TypeList<LocalType>,
Priority<1>,
- InvokeContext& invoke_context,
+ InvokeContext&,
Input&& input,
- ReadDest&& read_dest,
- typename std::enable_if<std::is_integral<LocalType>::value>::type* enable = nullptr)
+ ReadDest&& read_dest)
{
auto value = input.get();
if (value < std::numeric_limits<LocalType>::min() || value > std::numeric_limits<LocalType>::max()) {
@@ -77,12 +86,12 @@ decltype(auto) CustomReadField(TypeList<LocalType>,
}
template <typename LocalType, typename Input, typename ReadDest>
+ requires std::is_floating_point_v<LocalType>
decltype(auto) CustomReadField(TypeList<LocalType>,
Priority<1>,
- InvokeContext& invoke_context,
+ InvokeContext&,
Input&& input,
- ReadDest&& read_dest,
- typename std::enable_if<std::is_floating_point<LocalType>::value>::type* enable = nullptr)
+ ReadDest&& read_dest)
{
auto value = input.get();
static_assert(std::is_same<LocalType, decltype(value)>::value, "floating point type mismatch");
### src/ipc/libmultiprocess/include/mp/type-set.h
@@ -9,8 +9,8 @@
#include <mp/util.h>
namespace mp {
-template <typename LocalType, typename Value, typename Output>
-void CustomBuildField(TypeList<std::set<LocalType>>,
+template <typename LocalType, typename Compare, typename Allocator, typename Value, typename Output>
+void CustomBuildField(TypeList<std::set<LocalType, Compare, Allocator>>,
Priority<1>,
InvokeContext& invoke_context,
Value&& value,
@@ -19,8 +19,8 @@ void CustomBuildField(TypeList<std::set<LocalType>>,
BuildList(TypeList<LocalType>(), invoke_context, output, value);
}
-template <typename LocalType, typename Input, typename ReadDest>
-decltype(auto) CustomReadField(TypeList<std::set<LocalType>>,
+template <typename LocalType, typename Compare, typename Allocator, typename Input, typename ReadDest>
+decltype(auto) CustomReadField(TypeList<std::set<LocalType, Compare, Allocator>>,
Priority<1>,
InvokeContext& invoke_context,
Input&& input,
### src/ipc/libmultiprocess/include/mp/type-string.h
@@ -7,22 +7,24 @@
#include <mp/util.h>
+#include <algorithm>
+
namespace mp {
template <typename Value, typename Output>
void CustomBuildField(TypeList<std::string>,
Priority<1>,
- InvokeContext& invoke_context,
+ InvokeContext&,
Value&& value,
Output&& output)
{
auto result = output.init(value.size());
- memcpy(result.begin(), value.data(), value.size());
+ std::ranges::copy(value, result.begin());
}
template <typename Input, typename ReadDest>
decltype(auto) CustomReadField(TypeList<std::string>,
Priority<1>,
- InvokeContext& invoke_context,
+ InvokeContext&,
Input&& input,
ReadDest&& read_dest)
{
### src/ipc/libmultiprocess/include/mp/type-struct.h
@@ -9,11 +9,11 @@
namespace mp {
template <size_t index, typename LocalType, typename Value, typename Output>
+ requires (index < ProxyType<LocalType>::fields)
void BuildOne(TypeList<LocalType> param,
InvokeContext& invoke_context,
Output&& output,
- Value&& value,
- typename std::enable_if < index<ProxyType<LocalType>::fields>::type * enable = nullptr)
+ Value&& value)
{
using Index = std::integral_constant<size_t, index>;
using Struct = typename ProxyType<LocalType>::Struct;
@@ -25,31 +25,31 @@ void BuildOne(TypeList<LocalType> param,
}
template <size_t index, typename LocalType, typename Value, typename Output>
-void BuildOne(TypeList<LocalType> param,
- InvokeContext& invoke_context,
- Output&& output,
- Value&& value,
- typename std::enable_if<index == ProxyType<LocalType>::fields>::type* enable = nullptr)
+ requires (index == ProxyType<LocalType>::fields)
+void BuildOne(TypeList<LocalType>,
+ InvokeContext&,
+ Output&&,
+ Value&&)
{
}
template <typename LocalType, typename Value, typename Output>
+ requires requires { typename ProxyType<LocalType>::Struct; }
void CustomBuildField(TypeList<LocalType> local_type,
Priority<1>,
InvokeContext& invoke_context,
Value&& value,
- Output&& output,
- typename ProxyType<LocalType>::Struct* enable = nullptr)
+ Output&& output)
{
BuildOne<0>(local_type, invoke_context, output.init(), value);
}
template <size_t index, typename LocalType, typename Input, typename Value>
+ requires (index != ProxyType<LocalType>::fields)
void ReadOne(TypeList<LocalType> param,
InvokeContext& invoke_context,
Input&& input,
- Value&& value,
- typename std::enable_if<index != ProxyType<LocalType>::fields>::type* enable = nullptr)
+ Value&& value)
{
using Index = std::integral_constant<size_t, index>;
using Struct = typename ProxyType<LocalType>::Struct;
@@ -62,21 +62,21 @@ void ReadOne(TypeList<LocalType> param,
}
template <size_t index, typename LocalType, typename Input, typename Value>
-void ReadOne(TypeList<LocalType> param,
- InvokeContext& invoke_context,
- Input& input,
- Value& value,
- typename std::enable_if<index == ProxyType<LocalType>::fields>::type* enable = nullptr)
+ requires (index == ProxyType<LocalType>::fields)
+void ReadOne(TypeList<LocalType>,
+ InvokeContext&,
+ Input&,
+ Value&)
{
}
template <typename LocalType, typename Input, typename ReadDest>
+ requires requires { typename ProxyType<LocalType>::Struct; }
decltype(auto) CustomReadField(TypeList<LocalType> param,
Priority<1>,
InvokeContext& invoke_context,
Input&& input,
- ReadDest&& read_dest,
- typename ProxyType<LocalType>::Struct* enable = nullptr)
+ ReadDest&& read_dest)
{
return read_dest.update([&](auto& value) { ReadOne<0>(param, invoke_context, input, value); });
}
### src/ipc/libmultiprocess/include/mp/type-threadmap.h
@@ -19,21 +19,21 @@ struct ProxyServer<ThreadMap> final : public virtual ThreadMap::Server
};
template <typename Output>
+ requires FieldTypeIs<Output, ThreadMap::Client>
void CustomBuildField(TypeList<>,
Priority<1>,
InvokeContext& invoke_context,
- Output&& output,
- typename std::enable_if<std::is_same<decltype(output.get()), ThreadMap::Client>::value>::type* enable = nullptr)
+ Output&& output)
{
output.set(kj::heap<ProxyServer<ThreadMap>>(invoke_context.connection));
}
template <typename Input>
+ requires FieldTypeIs<Input, ThreadMap::Client>
decltype(auto) CustomReadField(TypeList<>,
Priority<1>,
InvokeContext& invoke_context,
- Input&& input,
- typename std::enable_if<std::is_same<decltype(input.get()), ThreadMap::Client>::value>::type* enable = nullptr)
+ Input&& input)
{
invoke_context.connection.m_thread_map = input.get();
}
### src/ipc/libmultiprocess/include/mp/type-unordered-set.h
@@ -10,8 +10,8 @@
#include <unordered_set>
namespace mp {
-template <typename LocalType, typename Value, typename Output>
-void CustomBuildField(TypeList<std::unordered_set<LocalType>>,
+template <typename LocalType, typename Hash, typename KeyEqual, typename Allocator, typename Value, typename Output>
+void CustomBuildField(TypeList<std::unordered_set<LocalType, Hash, KeyEqual, Allocator>>,
Priority<1>,
InvokeContext& invoke_context,
Value&& value,
@@ -20,8 +20,8 @@ void CustomBuildField(TypeList<std::unordered_set<LocalType>>,
BuildList(TypeList<LocalType>(), invoke_context, output, value);
}
-template <typename LocalType, typename Input, typename ReadDest>
-decltype(auto) CustomReadField(TypeList<std::unordered_set<LocalType>>,
+template <typename LocalType, typename Hash, typename KeyEqual, typename Allocator, typename Input, typename ReadDest>
+decltype(auto) CustomReadField(TypeList<std::unordered_set<LocalType, Hash, KeyEqual, Allocator>>,
Priority<1>,
InvokeContext& invoke_context,
Input&& input,
### src/ipc/libmultiprocess/include/mp/type-vector.h
@@ -9,8 +9,8 @@
#include <mp/util.h>
namespace mp {
-template <typename LocalType, typename Value, typename Output>
-void CustomBuildField(TypeList<std::vector<LocalType>>,
+template <typename LocalType, typename Allocator, typename Value, typename Output>
+void CustomBuildField(TypeList<std::vector<LocalType, Allocator>>,
Priority<1>,
InvokeContext& invoke_context,
Value&& value,
@@ -19,13 +19,13 @@ void CustomBuildField(TypeList<std::vector<LocalType>>,
BuildList(TypeList<LocalType>(), invoke_context, output, value);
}
-inline static bool BuildPrimitive(InvokeContext& invoke_context, std::vector<bool>::const_reference value, TypeList<bool>)
+inline static bool BuildPrimitive(InvokeContext&, std::vector<bool>::const_reference value, TypeList<bool>)
{
return value;
}
-template <typename LocalType, typename Input, typename ReadDest>
-decltype(auto) CustomReadField(TypeList<std::vector<LocalType>>,
+template <typename LocalType, typename Allocator, typename Input, typename ReadDest>
+decltype(auto) CustomReadField(TypeList<std::vector<LocalType, Allocator>>,
Priority<1>,
InvokeContext& invoke_context,
Input&& input,
### src/ipc/libmultiprocess/include/mp/util.h
@@ -5,12 +5,15 @@
#ifndef MP_UTIL_H
#define MP_UTIL_H
+#include <array>
#include <capnp/schema.h>
#include <cassert>
#include <cstdlib>
#include <cstring>
#include <exception>
#include <functional>
+#include <kj/async-io.h>
+#include <kj/memory.h>
#include <kj/string-tree.h>
#include <mutex>
#include <string>
@@ -30,6 +33,21 @@ namespace mp {
//! Generic utility functions used by capnp code.
+// std::cmp_less_equal/cmp_greater_equal only accept integer types (C++20 §[utility.intcmp]).
+// These wrappers fall back to regular comparison for floating-point types.
+template <typename A, typename B>
+constexpr bool safe_less_equal(A a, B b)
+{
+ if constexpr (std::is_floating_point_v<A> || std::is_floating_point_v<B>) return a <= b;
+ else return std::cmp_less_equal(a, b);
+}
+template <typename A, typename B>
+constexpr bool safe_greater_equal(A a, B b)
+{
+ if constexpr (std::is_floating_point_v<A> || std::is_floating_point_v<B>) return a >= b;
+ else return std::cmp_greater_equal(a, b);
+}
+
//! Type holding a list of types.
//!
//! Example:
@@ -91,6 +109,16 @@ using RemoveCvRef = std::remove_cv_t<std::remove_reference_t<T>>;
template <typename T>
using Decay = std::decay_t<T>;
+//! Concept satisfied when T's .get() method returns exactly type U.
+//! Used to constrain overloads that handle a specific capnp field type.
+template <typename T, typename U>
+concept FieldTypeIs = std::is_same_v<decltype(std::declval<T>().get()), U>;
+
+//! Concept satisfied when T's .get() method returns a capnp interface type
+//! (i.e., a type that exposes a nested ::Calls type in generated code).
+template <typename T>
+concept InterfaceField = requires { typename Decay<decltype(std::declval<T>().get())>::Calls; };
+
//! SFINAE helper, see using Require below.
template <typename SfinaeExpr, typename Result_>
struct _Require
@@ -142,7 +170,10 @@ struct PtrOrValue {
std::variant<T*, T> data;
template <typename... Args>
- PtrOrValue(T* ptr, Args&&... args) : data(ptr ? ptr : std::variant<T*, T>{std::in_place_type<T>, std::forward<Args>(args)...}) {}
+ PtrOrValue(T* ptr, Args&&... args) : data(std::in_place_type<T*>, ptr)
+ {
+ if (!ptr) data.template emplace<T>(std::forward<Args>(args)...);
+ }
T& operator*() { return data.index() ? std::get<T>(data) : *std::get<T*>(data); }
T* operator->() { return &**this; }
@@ -248,32 +279,49 @@ decltype(auto) TryFinally(Fn&& fn, After&& after)
}
}
+//! Set the OS-level name of the current thread
+void SetOsThreadName(const char* name);
+
//! Format current thread name as "{exe_name}-{$pid}/{thread_name}-{$tid}".
std::string ThreadName(const char* exe_name);
//! Escape binary string for use in log so it doesn't trigger unicode decode
//! errors in python unit tests.
std::string LogEscape(const kj::StringTree& string, size_t max_size);
+using Stream = kj::Own<kj::AsyncIoStream>;
+
+using ProcessId = int;
+using SocketId = int;
+inline constexpr SocketId SocketError{-1};
+
+//! Information about parent process passed to child process as a command-line
+//! argument. On unix this is the child socket fd number formatted as a string.
+using SpawnConnectInfo = std::string;
+
//! Callback type used by SpawnProcess below.
-using FdToArgsFn = std::function<std::vector<std::string>(int fd)>;
+using SpawnConnectInfoToArgsFn = std::function<std::vector<std::string>(const SpawnConnectInfo&)>;
//! Spawn a new process that communicates with the current process over a socket
-//! pair. Returns pid through an output argument, and file descriptor for the
-//! local side of the socket.
-//! The fd_to_args callback is invoked in the parent process before fork().
-//! It must not rely on child pid/state, and must return the command line
-//! arguments that should be used to execute the process. Embed the remote file
-//! descriptor number in whatever format the child process expects.
-int SpawnProcess(int& pid, FdToArgsFn&& fd_to_args);
-
-//! Call execvp with vector args.
-//! Not safe to call in a post-fork child of a multi-threaded process.
-//! Currently only used by mpgen at build time.
-void ExecProcess(const std::vector<std::string>& args);
+//! pair. Calls connect_info_to_args callback with a connection string that
+//! needs to be passed to the child process, and executes the argv command line
+//! it returns. Returns child process id and socket id.
+std::tuple<ProcessId, SocketId> SpawnProcess(SpawnConnectInfoToArgsFn&& connect_info_to_args);
+
+//! Initialize spawned child process using the SpawnConnectInfo string passed to it,
+//! returning a socket id for communicating with the parent process.
+SocketId StartSpawned(const SpawnConnectInfo& connect_info);
+
+//! Create a socket pair that can be used to communicate within a process or
+//! between parent and child processes.
+std::array<SocketId, 2> SocketPair();
+
+//! Start a process and return its process id. Caller should call WaitProcess
+//! on the returned id.
+ProcessId StartProcess(const std::vector<std::string>& args);
//! Wait for a process to exit and return its exit code.
-int WaitProcess(int pid);
+int WaitProcess(ProcessId pid);
inline char* CharCast(char* c) { return c; }
inline char* CharCast(unsigned char* c) { return (char*)c; }
### src/ipc/libmultiprocess/include/mp/version.h
@@ -24,7 +24,7 @@
//! pointing at the prior merge commit. The /doc/versions.md file should also be
//! updated, noting any significant or incompatible changes made since the
//! previous version.
-#define MP_MAJOR_VERSION 13
+#define MP_MAJOR_VERSION 14
//! Minor version number. Should be incremented in stable branches after
//! backporting changes. The /doc/versions.md file should also be updated to
### src/ipc/libmultiprocess/shell.nix
@@ -5,6 +5,7 @@
, capnprotoVersion ? null
, capnprotoSanitizers ? null # Optional sanitizers to build cap'n proto with
, cmakeVersion ? null
+, gccVersion ? null
, libcxxSanitizers ? null # Optional LLVM_USE_SANITIZER value to use for libc++, see https://llvm.org/docs/CMake.html
}:
@@ -17,10 +18,6 @@ let
};
};
capnprotoHashes = {
- "0.7.0" = "sha256-Y/7dUOQPDHjniuKNRw3j8dG1NI9f/aRWpf8V0WzV9k8=";
- "0.7.1" = "sha256-3cBpVmpvCXyqPUXDp12vCFCk32ZXWpkdOliNH37UwWE=";
- "0.8.0" = "sha256-rfiqN83begjJ9eYjtr21/tk1GJBjmeVfa3C3dZBJ93w=";
- "0.8.1" = "sha256-OZqNVYdyszro5rIe+w6YN00g6y8U/1b8dKYc214q/2o=";
"0.9.0" = "sha256-yhbDcWUe6jp5PbIXzn5EoKabXiWN8lnS08hyfxUgEQ0=";
"0.9.2" = "sha256-BspWOPZcP5nCTvmsDE62Zutox+aY5pw42d6hpH3v4cM=";
"0.10.0" = "sha256-++F4l54OMTDnJ+FO3kV/Y/VLobKVRk461dopanuU3IQ=";
@@ -53,12 +50,20 @@ let
"-g"
];
};
- })).override (lib.optionalAttrs enableLibcxx { clangStdenv = llvm.libcxxStdenv; });
+ })).override (lib.optionalAttrs enableLibcxx { clangStdenv = llvm.libcxxStdenv; }
+ # On i686, GCC and clang handle empty structs differently, causing an
+ # ABI mismatch and segfaulting in calls to functions like
+ # kj::str(kj::Exception). Prevent this by compiling capnproto with the
+ # same toolchain as libmultiprocess.
+ // lib.optionalAttrs crossPkgs.stdenv.hostPlatform.isx86_32 { clangStdenv = crossPkgs.stdenv; });
clang = if enableLibcxx then llvm.libcxxClang else llvm.clang;
clang-tools = llvm.clang-tools.override { inherit enableLibcxx; };
cmakeHashes = {
"3.12.4" = "sha256-UlVYS/0EPrcXViz/iULUcvHA5GecSUHYS6raqbKOMZQ=";
+ "4.1.1" = "sha256-sp9vGXM6oiS3djUHoQikJ+1Ixojh+vIrKcROHDBUkoI=";
+ "4.3.4" = "sha256-/e/4l7nrSddkU58rHtxut+FEDfMlZ4qXwZeEmekxrdo=";
};
+ gcc = if gccVersion == null then null else builtins.getAttr ("gcc" + gccVersion) pkgs;
cmakeBuild = if cmakeVersion == null then pkgs.cmake else (pkgs.cmake.overrideAttrs (old: {
version = cmakeVersion;
src = pkgs.fetchurl {
@@ -68,18 +73,25 @@ let
patches = [];
})).override { isMinimalBuild = true; };
in crossPkgs.mkShell {
- buildInputs = [
+ buildInputs = lib.optionals (capnprotoVersion != "none") [
capnproto
];
nativeBuildInputs = with pkgs; [
cmakeBuild
+ git
include-what-you-use
ninja
- ] ++ lib.optionals (!minimal) [
+ ] ++ lib.optional (gcc != null) gcc ++ lib.optionals (!minimal) [
clang
clang-tools
];
+ CC = if gcc == null then null else "${gcc}/bin/gcc";
+ CXX = if gcc == null then null else "${gcc}/bin/g++";
+
# Tell IWYU where its libc++ mapping lives
IWYU_MAPPING_FILE = if enableLibcxx then "${llvm.libcxx.dev}/include/c++/v1/libcxx.imp" else null;
+
+ # Avoid "SSL certificate problem: unable to get local issuer certificate" error during git clone in ci/scripts/ci.sh
+ NIX_SSL_CERT_FILE = "${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt";
}
### src/ipc/libmultiprocess/src/mp/gen.cpp
@@ -9,7 +9,6 @@
#include <capnp/schema.capnp.h> // IWYU pragma: keep
#include <capnp/schema.h>
#include <capnp/schema-parser.h>
-#include <cerrno>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
@@ -21,14 +20,13 @@
#include <kj/common.h>
#include <kj/filesystem.h>
#include <kj/memory.h>
+#include <kj/exception.h>
#include <kj/string.h>
#include <map>
#include <set>
#include <sstream>
#include <stdexcept>
#include <string>
-#include <system_error>
-#include <unistd.h>
#include <utility>
#include <vector>
@@ -207,7 +205,7 @@ struct FieldList
}
};
-std::string AccessorType(kj::StringPtr base_name, const Field& field)
+static std::string AccessorType(kj::StringPtr base_name, const Field& field)
{
const auto& f = field.param_is_set ? field.param : field.result;
const auto field_name = f.getProto().getName();
@@ -274,26 +272,19 @@ static void Generate(kj::StringPtr src_prefix,
if (p != std::string::npos) include_base.erase(p);
std::vector<std::string> args;
- args.emplace_back(capnp_PREFIX "/bin/capnp");
+ args.emplace_back(CAPNP_EXECUTABLE);
args.emplace_back("compile");
args.emplace_back("--src-prefix=");
args.back().append(src_prefix.cStr(), src_prefix.size());
for (const auto& import_path : import_paths) {
args.emplace_back("--import-path=");
args.back().append(import_path.cStr(), import_path.size());
}
- args.emplace_back("--output=" capnp_PREFIX "/bin/capnpc-c++");
+ args.emplace_back("--output=" CAPNPC_CXX_EXECUTABLE);
args.emplace_back(src_file);
- const int pid = fork();
- if (pid == -1) {
- throw std::system_error(errno, std::system_category(), "fork");
- }
- if (!pid) {
- mp::ExecProcess(args);
- }
- const int status = mp::WaitProcess(pid);
+ const int status = mp::WaitProcess(mp::StartProcess(args));
if (status) {
- throw std::runtime_error("Invoking " capnp_PREFIX "/bin/capnp failed");
+ throw std::runtime_error("Invoking " CAPNP_EXECUTABLE " failed");
}
const capnp::SchemaParser parser;
@@ -338,6 +329,7 @@ static void Generate(kj::StringPtr src_prefix,
cpp_client << "#include <kj/common.h>\n";
cpp_client << "#include <map>\n";
cpp_client << "#include <mp/proxy.h>\n";
+ cpp_client << "#include <mp/proxy-io.h>\n";
cpp_client << "#include <mp/util.h>\n";
cpp_client << "#include <string>\n";
cpp_client << "#include <vector>\n";
@@ -701,35 +693,41 @@ static void Generate(kj::StringPtr src_prefix,
int main(int argc, char** argv)
{
- if (argc < 3) {
- std::cerr << "Usage: " << PROXY_BIN << " SRC_PREFIX INCLUDE_PREFIX SRC_FILE [IMPORT_PATH...]\n";
- exit(1);
- }
- std::vector<kj::StringPtr> import_paths;
- std::vector<kj::Own<const kj::ReadableDirectory>> import_dirs;
- auto fs = kj::newDiskFilesystem();
- auto cwd = fs->getCurrentPath();
- kj::Own<const kj::ReadableDirectory> src_dir;
- KJ_IF_MAYBE(dir, fs->getRoot().tryOpenSubdir(cwd.evalNative(argv[1]))) {
- src_dir = kj::mv(*dir);
- } else {
- throw std::runtime_error(std::string("Failed to open src_prefix prefix directory: ") + argv[1]);
- }
- for (int i = 4; i < argc; ++i) {
- KJ_IF_MAYBE(dir, fs->getRoot().tryOpenSubdir(cwd.evalNative(argv[i]))) {
- import_paths.emplace_back(argv[i]);
- import_dirs.emplace_back(kj::mv(*dir));
+ int ret = 1;
+ KJ_IF_MAYBE(exception, kj::runCatchingExceptions([&]() {
+ if (argc < 3) {
+ std::cerr << "Usage: " << PROXY_BIN << " SRC_PREFIX INCLUDE_PREFIX SRC_FILE [IMPORT_PATH...]\n";
+ exit(1);
+ }
+ std::vector<kj::StringPtr> import_paths;
+ std::vector<kj::Own<const kj::ReadableDirectory>> import_dirs;
+ auto fs = kj::newDiskFilesystem();
+ auto cwd = fs->getCurrentPath();
+ kj::Own<const kj::ReadableDirectory> src_dir;
+ KJ_IF_MAYBE(dir, fs->getRoot().tryOpenSubdir(cwd.evalNative(argv[1]))) {
+ src_dir = kj::mv(*dir);
} else {
- throw std::runtime_error(std::string("Failed to open import directory: ") + argv[i]);
+ throw std::runtime_error(std::string("Failed to open src_prefix prefix directory: ") + argv[1]);
}
- }
- for (const char* path : {CMAKE_INSTALL_PREFIX "/include", capnp_PREFIX "/include"}) {
- KJ_IF_MAYBE(dir, fs->getRoot().tryOpenSubdir(cwd.evalNative(path))) {
- import_paths.emplace_back(path);
- import_dirs.emplace_back(kj::mv(*dir));
+ for (int i = 4; i < argc; ++i) {
+ KJ_IF_MAYBE(dir, fs->getRoot().tryOpenSubdir(cwd.evalNative(argv[i]))) {
+ import_paths.emplace_back(argv[i]);
+ import_dirs.emplace_back(kj::mv(*dir));
+ } else {
+ throw std::runtime_error(std::string("Failed to open import directory: ") + argv[i]);
+ }
+ }
+ for (const char* path : {CMAKE_INSTALL_PREFIX "/include", capnp_PREFIX "/include"}) {
+ KJ_IF_MAYBE(dir, fs->getRoot().tryOpenSubdir(cwd.evalNative(path))) {
+ import_paths.emplace_back(path);
+ import_dirs.emplace_back(kj::mv(*dir));
+ }
+ // No exception thrown if _PREFIX directories do not exist
}
- // No exception thrown if _PREFIX directories do not exist
+ Generate(argv[1], argv[2], argv[3], import_paths, *src_dir, import_dirs);
+ ret = 0;
+ })) {
+ std::cerr << "mpgen error: " << kj::str(*exception).cStr() << '\n';
}
- Generate(argv[1], argv[2], argv[3], import_paths, *src_dir, import_dirs);
- return 0;
+ return ret;
}
### src/ipc/libmultiprocess/src/mp/proxy.cpp
@@ -24,6 +24,7 @@
#include <kj/debug.h>
#include <kj/exception.h>
#include <kj/function.h>
+#include <kj/io.h>
#include <kj/memory.h>
#include <kj/string.h>
#include <cstdint>
@@ -32,17 +33,29 @@
#include <optional>
#include <stdexcept>
#include <string>
-#include <sys/socket.h>
#include <thread>
#include <tuple>
-#include <unistd.h>
#include <utility>
#include <vector>
namespace mp {
thread_local ThreadContext g_thread_context; // NOLINT(bitcoin-nontrivial-threadlocal)
+ThreadContext& CurrentThread()
+{
+ return g_thread_context;
+}
+
+Stream MakeStream(EventLoop&loop, SocketId socket)
+{
+ Stream stream;
+ loop.sync([&]() {
+ stream = loop.m_io_context.lowLevelProvider->wrapSocketFd(socket, kj::LowLevelAsyncIoProvider::TAKE_OWNERSHIP);
+ });
+ return stream;
+}
+
void LoggingErrorHandler::taskFailed(kj::Exception&& exception)
{
KJ_LOG(ERROR, "Uncaught exception in daemonized task.", exception);
@@ -69,10 +82,20 @@ void EventLoopRef::reset(bool relock) MP_NO_TSA
loop->m_num_refs -= 1;
if (loop->done()) {
loop->m_cv.notify_all();
- int post_fd{loop->m_post_fd};
+ // Capture loop->m_post_writer pointer before releasing the lock.
+ // The pointer can't actually change before the write() call below,
+ // but copying it with the lock held instead of accessing it
+ // directly below prevents TSAN false positives because TSAN is
+ // unaware of socketpair write() synchronization and might falsely
+ // report the pointer being used in this thread and assigned in the
+ // other thread without synchronization between.
+ kj::OutputStream* post_writer{loop->m_post_writer.get()};
loop_lock->unlock();
char buffer = 0;
- KJ_SYSCALL(write(post_fd, &buffer, 1)); // NOLINT(bugprone-suspicious-semicolon)
+ // It safe to access post_writer here because the loop can't
+ // exit until this write takes place. See "Intentionally do not
+ // break..." comment in EventLoop::loop
+ post_writer->write(&buffer, 1);
// By default, do not try to relock `loop_lock` after writing,
// because the event loop could wake up and destroy itself and the
// mutex might no longer exist.
@@ -83,7 +106,7 @@ void EventLoopRef::reset(bool relock) MP_NO_TSA
ProxyContext::ProxyContext(Connection* connection) : connection(connection), loop{*connection->m_loop} {}
-Connection::~Connection()
+Connection::~Connection() noexcept(false)
{
// Connection destructor is always called on the event loop thread. If this
// is a local disconnect, it will trigger I/O, so this needs to run on the
@@ -103,6 +126,38 @@ Connection::~Connection()
// after the calls finish.
m_rpc_system.reset();
+ // shutdownWrite is needed on Windows so pending data in the m_stream socket
+ // will be sent instead of discarded when m_stream is destroyed. On unix,
+ // this doesn't seem to be needed because data is sent more reliably.
+ //
+ // Sending pending data is important if the connection is a socketpair
+ // because when one side of the socketpair is closed, the other side doesn't
+ // seem to receive any onDisconnect event. So it is important for the other
+ // side to instead receive Cap'n Proto "release" messages (see `struct
+ // Release` in capnp/rpc.capnp) from local Client objects being destroyed so
+ // the remote side can free resources and shut down cleanly. Without this,
+ // when one side of a socket pair is closed the other side may not receive
+ // these messages, preventing the remote side from freeing ProxyServer
+ // resources and shutting down cleanly.
+ // Use kj::runCatchingExceptions instead of try/catch because on macOS with
+ // dynamic libraries, kj::Exception typeinfo differs between libcapnp and
+ // the calling binary, so catch (const kj::Exception&) silently fails to
+ // match. kj::runCatchingExceptions uses KJ's own interception mechanism
+ // which works correctly across dynamic library boundaries.
+ KJ_IF_MAYBE(exception, kj::runCatchingExceptions([&]() {
+ m_stream->shutdownWrite();
+ })) {
+ // Ignore ENOTCONN: on macOS/FreeBSD (unlike Linux), shutdown(SHUT_WR)
+ // returns ENOTCONN if the peer already closed the connection. This is
+ // expected when the destructor is triggered by a remote disconnect.
+ // Also ignore UNIMPLEMENTED: shutdownWrite() may not be supported on
+ // all stream types (e.g. streams that do not support half-close).
+ if (exception->getType() != kj::Exception::Type::DISCONNECTED &&
+ exception->getType() != kj::Exception::Type::UNIMPLEMENTED) {
+ kj::throwRecoverableException(kj::mv(*exception));
+ }
+ }
+
// ProxyClient cleanup handlers are in sync list, and ProxyServer cleanup
// handlers are in the async list.
//
@@ -206,20 +261,24 @@ EventLoop::EventLoop(const char* exe_name, LogOptions log_opts, void* context)
m_log_opts(std::move(log_opts)),
m_context(context)
{
- int fds[2];
- KJ_SYSCALL(socketpair(AF_UNIX, SOCK_STREAM, 0, fds));
- m_wait_fd = fds[0];
- m_post_fd = fds[1];
+ auto pipe = m_io_context.provider->newTwoWayPipe();
+ m_wait_stream = kj::mv(pipe.ends[0]);
+ m_post_stream = kj::mv(pipe.ends[1]);
+ KJ_IF_MAYBE(fd, m_post_stream->getFd()) {
+ m_post_writer = kj::heap<kj::FdOutputStream>(*fd);
+ } else {
+ throw std::logic_error("Could not get file descriptor for new pipe.");
+ }
}
EventLoop::~EventLoop()
{
if (m_async_thread.joinable()) m_async_thread.join();
const Lock lock(m_mutex);
- KJ_ASSERT(m_post_fn == nullptr);
+ KJ_ASSERT(m_sync_fn == nullptr);
KJ_ASSERT(!m_async_fns);
- KJ_ASSERT(m_wait_fd == -1);
- KJ_ASSERT(m_post_fd == -1);
+ KJ_ASSERT(!m_wait_stream);
+ KJ_ASSERT(!m_post_stream);
KJ_ASSERT(m_num_refs == 0);
// Spin event loop. wait for any promises triggered by RPC shutdown.
@@ -229,36 +288,34 @@ EventLoop::~EventLoop()
void EventLoop::loop()
{
- assert(!g_thread_context.loop_thread);
- g_thread_context.loop_thread = true;
- KJ_DEFER(g_thread_context.loop_thread = false);
+ assert(!CurrentThread().loop_thread);
+ CurrentThread().loop_thread = true;
+ KJ_DEFER(CurrentThread().loop_thread = false);
{
const Lock lock(m_mutex);
assert(!m_async_fns);
m_async_fns.emplace();
}
- kj::Own<kj::AsyncIoStream> wait_stream{
- m_io_context.lowLevelProvider->wrapSocketFd(m_wait_fd, kj::LowLevelAsyncIoProvider::TAKE_OWNERSHIP)};
- int post_fd{m_post_fd};
+ kj::Own<kj::AsyncIoStream>& wait_stream{m_wait_stream};
char buffer = 0;
for (;;) {
const size_t read_bytes = wait_stream->read(&buffer, 0, 1).wait(m_io_context.waitScope);
if (read_bytes != 1) throw std::logic_error("EventLoop wait_stream closed unexpectedly");
Lock lock(m_mutex);
- if (m_post_fn) {
- // m_post_fn throwing is never expected. If it does happen, the caller
- // of EventLoop::post() will return without any indication of failure,
+ if (m_sync_fn) {
+ // m_sync_fn throwing is never expected. If it does happen, the caller
+ // of EventLoop::sync() will return without any indication of failure,
// which will likely cause other bugs. Log the error and continue.
- KJ_IF_MAYBE(exception, kj::runCatchingExceptions([&]() MP_REQUIRES(m_mutex) { Unlock(lock, *m_post_fn); })) {
- MP_LOG(*this, Log::Error) << "EventLoop: m_post_fn threw: " << kj::str(*exception).cStr();
+ KJ_IF_MAYBE(exception, kj::runCatchingExceptions([&]() MP_REQUIRES(m_mutex) { Unlock(lock, *m_sync_fn); })) {
+ MP_LOG(*this, Log::Error) << "EventLoop: m_sync_fn threw: " << kj::str(*exception).cStr();
}
- m_post_fn = nullptr;
+ m_sync_fn = nullptr;
m_cv.notify_all();
} else if (done()) {
- // Intentionally do not break if m_post_fn was set, even if done()
- // would return true, to ensure that the EventLoopRef write(post_fd)
+ // Intentionally do not break if m_sync_fn was set, even if done()
+ // would return true, to ensure that the sync() m_post_writer->write()
// call always succeeds and the loop does not exit between the time
// that the done condition is set and the write call is made.
break;
@@ -268,30 +325,29 @@ void EventLoop::loop()
m_task_set.reset();
MP_LOG(*this, Log::Info) << "EventLoop::loop bye.";
wait_stream = nullptr;
- KJ_SYSCALL(::close(post_fd));
const Lock lock(m_mutex);
- m_wait_fd = -1;
- m_post_fd = -1;
+ m_post_writer = nullptr;
+ m_wait_stream = nullptr;
+ m_post_stream = nullptr;
m_async_fns.reset();
m_cv.notify_all();
}
-void EventLoop::post(kj::Function<void()> fn)
+void EventLoop::sync(kj::FunctionParam<void()> fn)
{
if (std::this_thread::get_id() == m_thread_id) {
fn();
return;
}
Lock lock(m_mutex);
EventLoopRef ref(*this, &lock);
- m_cv.wait(lock.m_lock, [this]() MP_REQUIRES(m_mutex) { return m_post_fn == nullptr; });
- m_post_fn = &fn;
- int post_fd{m_post_fd};
+ m_cv.wait(lock.m_lock, [this]() MP_REQUIRES(m_mutex) { return m_sync_fn == nullptr; });
+ m_sync_fn = &fn;
Unlock(lock, [&] {
char buffer = 0;
- KJ_SYSCALL(write(post_fd, &buffer, 1));
+ m_post_writer->write(&buffer, 1);
});
- m_cv.wait(lock.m_lock, [this, &fn]() MP_REQUIRES(m_mutex) { return m_post_fn != &fn; });
+ m_cv.wait(lock.m_lock, [this, &fn]() MP_REQUIRES(m_mutex) { return m_sync_fn != &fn; });
}
void EventLoop::startAsyncThread()
@@ -302,6 +358,7 @@ void EventLoop::startAsyncThread()
m_cv.notify_all();
} else if (!m_async_fns->empty()) {
m_async_thread = std::thread([this] {
+ SetOsThreadName("capnp-async");
Lock lock(m_mutex);
while (m_async_fns) {
if (!m_async_fns->empty()) {
@@ -427,12 +484,13 @@ kj::Promise<void> ProxyServer<ThreadMap>::makePool(MakePoolContext context)
for (uint32_t i = 0; i < count; ++i) {
const std::string thread_name = "pool/" + std::to_string(i);
std::promise<ThreadContext*> thread_context;
- std::thread thread([&loop, &thread_context, thread_name]() {
- g_thread_context.thread_name = ThreadName(loop.m_exe_name) + " (" + thread_name + ")";
- g_thread_context.waiter = std::make_unique<Waiter>();
- Lock lock(g_thread_context.waiter->m_mutex);
- thread_context.set_value(&g_thread_context);
- g_thread_context.waiter->wait(lock, [] { return !g_thread_context.waiter; });
+ std::thread thread([&loop, &thread_context, thread_name, i]() {
+ SetOsThreadName(("capnp-pool-" + std::to_string(i)).c_str());
+ CurrentThread().thread_name = ThreadName(loop.m_exe_name) + " (" + thread_name + ")";
+ CurrentThread().waiter = std::make_unique<Waiter>();
+ Lock lock(CurrentThread().waiter->m_mutex);
+ thread_context.set_value(&CurrentThread());
+ CurrentThread().waiter->wait(lock, [] { return !CurrentThread().waiter; });
});
auto thread_server = kj::heap<ProxyServer<Thread>>(m_connection, *thread_context.get_future().get(), std::move(thread));
m_connection.m_thread_pool.push_back({m_connection.m_threads.add(kj::mv(thread_server))});
@@ -447,14 +505,15 @@ kj::Promise<void> ProxyServer<ThreadMap>::makeThread(MakeThreadContext context)
const std::string from = context.getParams().getName();
std::promise<ThreadContext*> thread_context;
std::thread thread([&loop, &thread_context, from]() {
- g_thread_context.thread_name = ThreadName(loop.m_exe_name) + " (from " + from + ")";
- g_thread_context.waiter = std::make_unique<Waiter>();
- Lock lock(g_thread_context.waiter->m_mutex);
- thread_context.set_value(&g_thread_context);
+ SetOsThreadName("capnp-worker");
+ CurrentThread().thread_name = ThreadName(loop.m_exe_name) + " (from " + from + ")";
+ CurrentThread().waiter = std::make_unique<Waiter>();
+ Lock lock(CurrentThread().waiter->m_mutex);
+ thread_context.set_value(&CurrentThread());
if (loop.testing_hook_makethread_created) loop.testing_hook_makethread_created();
// Wait for shutdown signal from ProxyServer<Thread> destructor (signal
// is just waiter getting set to null.)
- g_thread_context.waiter->wait(lock, [] { return !g_thread_context.waiter; });
+ CurrentThread().waiter->wait(lock, [] { return !CurrentThread().waiter; });
});
auto thread_server = kj::heap<ProxyServer<Thread>>(m_connection, *thread_context.get_future().get(), std::move(thread));
auto thread_client = m_connection.m_threads.add(kj::mv(thread_server));
@@ -466,7 +525,7 @@ std::atomic<int> server_reqs{0};
std::string LongThreadName(const char* exe_name)
{
- return g_thread_context.thread_name.empty() ? ThreadName(exe_name) : g_thread_context.thread_name;
+ return CurrentThread().thread_name.empty() ? ThreadName(exe_name) : CurrentThread().thread_name;
}
kj::StringPtr KJ_STRINGIFY(Log v)
### src/ipc/libmultiprocess/src/mp/util.cpp
@@ -7,14 +7,17 @@
#include <cerrno>
#include <cstdio>
-#include <filesystem>
-#include <iostream>
+#include <fcntl.h>
#include <kj/common.h>
+#include <kj/debug.h>
#include <kj/string-tree.h>
+#include <optional>
#include <pthread.h>
+#include <csignal>
#include <sstream>
#include <string>
#include <sys/types.h>
+#include <spawn.h>
#include <sys/resource.h>
#include <sys/socket.h>
#include <sys/wait.h>
@@ -32,7 +35,11 @@
#include <pthread_np.h>
#endif // HAVE_PTHREAD_GETTHREADID_NP
-namespace fs = std::filesystem;
+#if __has_include(<sys/prctl.h>)
+#include <sys/prctl.h>
+#endif
+
+extern "C" char **environ; // NOLINT(readability-redundant-declaration)
namespace mp {
namespace {
@@ -59,8 +66,134 @@ size_t MaxFd()
}
}
+//! Report an error and exit from the post-fork child of a multi-threaded
+//! process, where only async-signal-safe calls (like write and _exit) are
+//! allowed. Accepting only a reference to a char array (in practice a string
+//! literal) ensures no allocation is needed at the call site.
+template <std::size_t N>
+[[noreturn]] void ChildFail(const char (&msg)[N]) noexcept
+{
+ const ssize_t written = ::write(STDERR_FILENO, msg, N - 1);
+ (void)written;
+ _exit(126);
+}
+
+enum class SpawnErrorOp
+{
+ CLOSE,
+ EXECVP,
+ READ,
+ FCNTL,
+};
+
+struct SpawnError {
+ SpawnErrorOp which;
+ int err;
+};
+
+const char* SpawnErrorName(SpawnErrorOp which)
+{
+ switch (which) {
+ case SpawnErrorOp::CLOSE: return "close";
+ case SpawnErrorOp::EXECVP: return "execvp";
+ case SpawnErrorOp::READ: return "read";
+ case SpawnErrorOp::FCNTL: return "fcntl";
+ }
+ return "unknown";
+}
+
+// Read the child's error report. Returns nullopt on success, or a SpawnError to
+// throw on failure. Success is a clean EOF: read() returns 0 with nothing
+// buffered because the child's write end was closed by a successful exec (via
+// FD_CLOEXEC). A fully-read struct is the failure the child reported. A read()
+// error, or an EOF partway through the struct (the child died mid-report), is
+// surfaced as a SpawnErrorOp::READ failure rather than being mistaken for
+// success. A single read() is not guaranteed to return all sizeof(SpawnError)
+// bytes (the socket is SOCK_STREAM, which has no message boundaries) and may be
+// interrupted by a signal, so loop until the whole struct is read.
+//
+// Note that a clean EOF is also what happens if the child is killed before it
+// reaches exec, so this function reports success in that case. There is no good
+// way to distinguish the two here, but callers will still see the failure when
+// they call WaitProcess and get the child's exit status.
+std::optional<SpawnError> ReadSpawnResult(int fd)
+{
+ SpawnError error{};
+ char* buf = reinterpret_cast<char*>(&error);
+ size_t remaining = sizeof(error);
+ while (remaining > 0) {
+ const ssize_t n = ::read(fd, buf, remaining);
+ if (n < 0) {
+ if (errno == EINTR) continue;
+ return SpawnError{.which = SpawnErrorOp::READ, .err = errno};
+ }
+ if (n == 0) {
+ if (remaining == sizeof(error)) return std::nullopt; // clean EOF: success
+ return SpawnError{.which = SpawnErrorOp::READ, .err = EPROTO}; // torn report
+ }
+ buf += n;
+ remaining -= static_cast<size_t>(n);
+ }
+ return error;
+}
+
+// Write the whole SpawnError to fd, retrying short writes and EINTR so the
+// parent never sees a torn struct. Runs in the post-fork child, so it must stay
+// async-signal-safe: it only calls write() and does not allocate or throw. This
+// is best-effort -- if the write cannot complete there is nothing useful the
+// child can do, so it stops and lets the caller _exit().
+void WriteSpawnError(int fd, const SpawnError& error)
+{
+ const char* buf = reinterpret_cast<const char*>(&error);
+ size_t remaining = sizeof(error);
+ while (remaining > 0) {
+ const ssize_t n = ::write(fd, buf, remaining);
+ if (n < 0) {
+ if (errno == EINTR) continue;
+ break;
+ }
+ buf += n;
+ remaining -= static_cast<size_t>(n);
+ }
+ if (remaining > 0) {
+ // The parent's read end is gone (e.g. the parent exited before the
+ // child could report), so the structured error can't be delivered.
+ // Leave a breadcrumb on stderr and exit. The exit code is irrelevant
+ // here since no live parent remains to wait on it.
+ ChildFail("SpawnProcess(child): failed and could not report error to parent\n");
+ }
+}
+
+// Get rid of a child process the parent is abandoning because SpawnProcess is
+// about to throw, so it is not left behind as a zombie. The child may still be
+// alive, so kill it first: waiting without that could block for as long as the
+// spawned program runs.
+void KillAndReapChild(ProcessId pid)
+{
+ (void)::kill(pid, SIGKILL);
+ while (::waitpid(pid, /*status=*/nullptr, /*options=*/0) == -1 && errno == EINTR) {}
+}
+
} // namespace
+// Copied from https://github.com/bitcoin/bitcoin/blob/d3e40af2597/src/util/threadnames.cpp#L21-L36
+void SetOsThreadName(const char* name)
+{
+#if defined(PR_SET_NAME)
+ // Only the first 15 characters are used (16 - NUL terminator)
+ ::prctl(PR_SET_NAME, name, 0, 0, 0);
+#elif defined(HAVE_PTHREAD_SETNAME_NP_3ARG)
+ pthread_setname_np(pthread_self(), "%s", const_cast<char*>(name));
+#elif defined(HAVE_PTHREAD_GETTHREADID_NP)
+ pthread_set_name_np(pthread_self(), name);
+#elif defined(__APPLE__)
+ pthread_setname_np(name);
+#else
+ // Prevent warnings for unused parameters...
+ (void)name;
+#endif
+}
+
std::string ThreadName(const char* exe_name)
{
char thread_name[16] = {0};
@@ -116,74 +249,126 @@ std::string LogEscape(const kj::StringTree& string, size_t max_size)
return result;
}
-int SpawnProcess(int& pid, FdToArgsFn&& fd_to_args)
+std::tuple<ProcessId, SocketId> SpawnProcess(SpawnConnectInfoToArgsFn&& connect_info_to_args)
{
- int fds[2];
- if (socketpair(AF_UNIX, SOCK_STREAM, 0, fds) != 0) {
- throw std::system_error(errno, std::system_category(), "socketpair");
- }
+ auto fds{SocketPair()};
+ // Only used for the child to report errors back to the parent, so a one-way
+ // pipe would be sufficient, but reuse the existing SocketPair() helper.
+ auto error_fds{SocketPair()};
// Evaluate the callback and build the argv array before forking.
//
// The parent process may be multi-threaded and holding internal library
// locks at fork time. In that case, running code that allocates memory or
// takes locks in the child between fork() and exec() can deadlock
// indefinitely. Precomputing arguments in the parent avoids this.
- const std::vector<std::string> args{fd_to_args(fds[0])};
+ const std::vector<std::string> args{connect_info_to_args(std::to_string(fds[0]))};
const std::vector<char*> argv{MakeArgv(args)};
- pid = fork();
+ ProcessId pid = fork();
if (pid == -1) {
- throw std::system_error(errno, std::system_category(), "fork");
+ const int err = errno;
+ (void)close(fds[0]);
+ (void)close(fds[1]);
+ (void)close(error_fds[0]);
+ (void)close(error_fds[1]);
+ throw std::system_error(err, std::system_category(), "fork");
}
// Parent process closes the descriptor for socket 0, child closes the
// descriptor for socket 1. On failure, the parent throws, but the child
// must _exit(126) (post-fork child must not throw).
if (close(fds[pid ? 0 : 1]) != 0) {
+ const int err = errno;
if (pid) {
(void)close(fds[1]);
- throw std::system_error(errno, std::system_category(), "close");
+ (void)close(error_fds[0]);
+ (void)close(error_fds[1]);
+ KillAndReapChild(pid);
+ throw std::system_error(err, std::system_category(), "close");
}
- static constexpr char msg[] = "SpawnProcess(child): close(fds[1]) failed\n";
- const ssize_t writeResult = ::write(STDERR_FILENO, msg, sizeof(msg) - 1);
- (void)writeResult;
+ WriteSpawnError(error_fds[0], {.which = SpawnErrorOp::CLOSE, .err = err});
_exit(126);
}
if (!pid) {
// Child process must close all potentially open descriptors, except
- // socket 0. Do not throw, allocate, or do non-fork-safe work here.
+ // socket 0 and the error-reporting socket 0. Do not throw, allocate, or
+ // do non-fork-safe work here.
const int maxFd = MaxFd();
for (int fd = 3; fd < maxFd; ++fd) {
- if (fd != fds[0]) {
+ if (fd != fds[0] && fd != error_fds[0]) {
close(fd);
}
}
+ // Clear FD_CLOEXEC on socket 0 so it survives exec. Doing this here
+ // rather than in the parent before forking avoids a window where a
+ // concurrent fork in another thread would leak the descriptor into an
+ // unrelated child process (which would not clear it).
+ // fcntl is async-signal-safe.
+ const int fds0_flags = fcntl(fds[0], F_GETFD);
+ if (fds0_flags == -1 || fcntl(fds[0], F_SETFD, fds0_flags & ~FD_CLOEXEC) == -1) {
+ const int err = errno;
+ WriteSpawnError(error_fds[0], {.which = SpawnErrorOp::FCNTL, .err = err});
+ _exit(126);
+ }
+
execvp(argv[0], argv.data());
- // NOTE: perror() is not async-signal-safe; calling it here in a
- // post-fork child may deadlock in multithreaded parents.
- // TODO: Report errors to the parent via a pipe (e.g. write errno)
- // so callers can get diagnostics without relying on perror().
- perror("execvp failed");
+
+ const int err = errno;
+ WriteSpawnError(error_fds[0], {.which = SpawnErrorOp::EXECVP, .err = err});
_exit(127);
}
- return fds[1];
+
+ // Close the parent's copy of the child's write end.
+ if (close(error_fds[0]) != 0) {
+ const int err = errno;
+ (void)close(error_fds[1]);
+ (void)close(fds[1]);
+ KillAndReapChild(pid);
+ throw std::system_error(err, std::system_category(), "close");
+ }
+
+ const std::optional<SpawnError> error{ReadSpawnResult(error_fds[1])};
+ (void)close(error_fds[1]);
+ if (error) {
+ (void)close(fds[1]);
+ KillAndReapChild(pid);
+ throw std::system_error(error->err, std::system_category(), SpawnErrorName(error->which));
+ }
+ return {pid, fds[1]};
+}
+
+SocketId StartSpawned(const SpawnConnectInfo& connect_info)
+{
+ try {
+ return std::stoi(connect_info);
+ } catch (const std::exception&) {
+ throw std::system_error(EINVAL, std::system_category(),
+ std::string("StartSpawned: invalid connect_info '") + connect_info + "'");
+ }
+}
+
+std::array<SocketId, 2> SocketPair()
+{
+ int pair[2];
+ KJ_SYSCALL(socketpair(AF_UNIX, SOCK_STREAM, 0, pair));
+ KJ_SYSCALL(fcntl(pair[0], F_SETFD, FD_CLOEXEC));
+ KJ_SYSCALL(fcntl(pair[1], F_SETFD, FD_CLOEXEC));
+ return {pair[0], pair[1]};
}
-void ExecProcess(const std::vector<std::string>& args)
+ProcessId StartProcess(const std::vector<std::string>& args)
{
const std::vector<char*> argv{MakeArgv(args)};
- if (execvp(argv[0], argv.data()) != 0) {
- perror("execvp failed");
- if (errno == ENOENT && !args.empty()) {
- std::cerr << "Missing executable: " << fs::weakly_canonical(args.front()) << '\n';
- }
- _exit(1);
+ ProcessId pid;
+ if (int err = posix_spawnp(&pid, argv[0], nullptr, nullptr, argv.data(), ::environ)) {
+ KJ_FAIL_SYSCALL("posix_spawnp", err, args.front());
}
+ return pid;
}
-int WaitProcess(int pid)
+int WaitProcess(ProcessId pid)
{
int status;
if (::waitpid(pid, &status, /*options=*/0) != pid) {
### src/ipc/libmultiprocess/test/CMakeLists.txt
@@ -28,6 +28,7 @@ if(BUILD_TESTING AND TARGET CapnProto::kj-test)
mp/test/foo.h
mp/test/listen_tests.cpp
mp/test/spawn_tests.cpp
+ mp/test/connect_tests.cpp
mp/test/test.cpp
)
include(${PROJECT_SOURCE_DIR}/cmake/TargetCapnpSources.cmake)
### src/ipc/libmultiprocess/test/mp/test/common.h
@@ -0,0 +1,27 @@
+// Copyright (c) The Bitcoin Core developers
+// Distributed under the MIT software license, see the accompanying
+// file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+#ifndef MP_TEST_COMMON_H
+#define MP_TEST_COMMON_H
+
+#include <kj/debug.h>
+#include <mp/proxy-io.h>
+
+#include <stdexcept>
+
+namespace mp {
+namespace test {
+
+//! Default event loop log handler used by tests. Logs all messages and throws
+//! on errors so calling code can assert on them.
+inline void DefaultLogHandler(LogMessage log)
+{
+ KJ_LOG(INFO, log.level, log.message);
+ if (log.level == Log::Raise) throw std::runtime_error(log.message);
+}
+
+} // namespace test
+} // namespace mp
+
+#endif // MP_TEST_COMMON_H
### src/ipc/libmultiprocess/test/mp/test/connect_tests.cpp
@@ -0,0 +1,212 @@
+// Copyright (c) The Bitcoin Core developers
+// Distributed under the MIT software license, see the accompanying
+// file COPYING or http://www.opensource.org/licenses/mit-license.php.
+#include "common.h"
+#include "unixlistener.h"
+#include <kj/async.h>
+#include <kj/common.h>
+#include <kj/debug.h>
+#include <kj/memory.h>
+#include <kj/test.h>
+#include <mp/proxy-io.h>
+#include <mp/proxy.h>
+#include <mp/test/foo.capnp.h>
+#include <mp/test/foo.capnp.proxy.h>
+#include <mp/util.h>
+#include <sys/socket.h>
+#include <unistd.h>
+
+#include <array>
+#include <chrono>
+#include <condition_variable>
+#include <cstring> // IWYU pragma: keep
+#include <future>
+#include <memory>
+#include <mutex>
+#include <optional>
+#include <stdexcept>
+#include <string>
+#include <string_view>
+#include <thread>
+
+namespace mp {
+namespace test {
+namespace {
+
+constexpr auto FAILURE_TIMEOUT = std::chrono::seconds{30};
+
+class TestSetup
+{
+public:
+ EventLoop* m_loop;
+ std::optional<EventLoopRef> m_loop_ref;
+ //! Thread variable should be after other struct members so the thread does
+ //! not start until the other members are initialized.
+ std::thread m_loop_thread;
+
+ TestSetup(LogFn log_handler = DefaultLogHandler)
+ {
+ std::promise<EventLoop*> loop_promise;
+ m_loop_thread = std::thread([&, log_handler] {
+ EventLoop loop("mptest-connect", log_handler);
+ loop_promise.set_value(&loop);
+ loop.loop();
+ });
+ m_loop = loop_promise.get_future().get();
+ m_loop_ref.emplace(*m_loop);
+ }
+
+ ~TestSetup()
+ {
+ m_loop_ref.reset();
+ m_loop_thread.join();
+ }
+};
+
+KJ_TEST("ConnectStream connects to a socket serving a valid init interface")
+{
+ TestSetup setup;
+ auto [client_fd, server_fd] = SocketPair();
+
+ std::thread server_thread([&]() {
+ EventLoop server_loop("mptest-valid-server", DefaultLogHandler);
+ std::unique_ptr<FooInit> init = std::make_unique<FooInit>();
+ ServeStream<messages::FooInit>(server_loop, MakeStream(server_loop, server_fd), *init);
+ server_loop.loop();
+ });
+
+ // FooInit has a `construct()` method, so this connects to the server and
+ // sends an IPC request that must complete successfully.
+ auto init = ConnectStream<messages::FooInit>(*setup.m_loop, MakeStream(*setup.m_loop, client_fd));
+
+ init.reset();
+ server_thread.join();
+}
+
+KJ_TEST("ConnectStream throws when the socket is already disconnected")
+{
+ TestSetup setup;
+ auto [client_fd, server_fd] = SocketPair();
+
+ KJ_SYSCALL(close(server_fd));
+
+ try {
+ auto init = ConnectStream<messages::FooInit>(*setup.m_loop, MakeStream(*setup.m_loop, client_fd));
+
+ KJ_EXPECT(false);
+ } catch (const std::runtime_error& e) {
+ std::string_view reason = e.what();
+
+ KJ_EXPECT(reason == "IPC client method call interrupted by disconnect.");
+ }
+}
+
+KJ_TEST("ConnectStream defers disconnect failure to the first IPC request for interfaces without construct()")
+{
+ TestSetup setup;
+ auto [client_fd, server_fd] = SocketPair();
+
+ KJ_SYSCALL(close(server_fd));
+
+ // Without a construct() method no IPC call is made during client
+ // creation, so ConnectStream succeeds even though the peer is gone.
+ auto foo = ConnectStream<messages::FooInterface>(*setup.m_loop, MakeStream(*setup.m_loop, client_fd));
+
+ try {
+ foo->add(1, 2);
+ KJ_EXPECT(false);
+ } catch (const std::runtime_error& e) {
+ std::string_view reason = e.what();
+
+ // There is a race between the event loop detecting the disconnect
+ // and foo->add() being called. If the onDisconnect callback fires
+ // first and nulls m_context.connection, the error is "called after
+ // disconnect"; if foo->add() submits before the callback fires,
+ // the error is "interrupted by disconnect".
+ KJ_EXPECT(reason == "IPC client method called after disconnect." ||
+ reason == "IPC client method call interrupted by disconnect.");
+ }
+}
+
+KJ_TEST("ConnectStream handles a disconnect when no client calls are made")
+{
+ std::mutex mutex;
+ std::condition_variable cv;
+ bool warned = false;
+
+ TestSetup setup([&](LogMessage log) {
+ if (log.level == Log::Warning && log.message.find("unexpected network disconnect") != std::string::npos) {
+ const std::scoped_lock lock{mutex};
+ warned = true;
+ cv.notify_all();
+ }
+ DefaultLogHandler(log);
+ });
+ auto [client_fd, server_fd] = SocketPair();
+
+ KJ_SYSCALL(close(server_fd));
+
+ auto foo = ConnectStream<messages::FooInterface>(*setup.m_loop, MakeStream(*setup.m_loop, client_fd));
+
+ // The disconnect handler registered by ProxyClientBase should run and
+ // delete the connection even when no calls are ever made.
+ std::unique_lock<std::mutex> lock(mutex);
+ KJ_EXPECT(cv.wait_for(lock, FAILURE_TIMEOUT, [&] { return warned; }));
+}
+
+KJ_TEST("ConnectStream throws when the socket disconnects after receiving data")
+{
+ TestSetup setup;
+ auto [client_fd, server_fd] = SocketPair();
+
+ std::thread server_thread([&]() {
+ char buf[128];
+
+ recv(server_fd, buf, sizeof(buf), 0);
+ KJ_SYSCALL(close(server_fd));
+ });
+
+ try {
+ auto init = ConnectStream<messages::FooInit>(*setup.m_loop, MakeStream(*setup.m_loop, client_fd));
+
+ KJ_EXPECT(false);
+ } catch (const std::runtime_error& e) {
+ std::string_view reason = e.what();
+ KJ_EXPECT(reason == "IPC client method call interrupted by disconnect.");
+ }
+ server_thread.join();
+}
+
+KJ_TEST("ConnectStream throws when a connection accepted from a listener disconnects after receiving data")
+{
+ UnixListener listener;
+ TestSetup setup;
+ int client_fd = listener.MakeConnectedSocket();
+ int server_fd = listener.release();
+
+ std::thread server_thread([&]() {
+ char buf[128];
+
+ int connection_fd = accept(server_fd, nullptr, nullptr);
+
+ if (connection_fd >= 0) {
+ recv(connection_fd, buf, sizeof(buf), 0);
+ KJ_SYSCALL(close(connection_fd));
+ }
+ KJ_SYSCALL(close(server_fd));
+ });
+
+ try {
+ auto init = ConnectStream<messages::FooInit>(*setup.m_loop, MakeStream(*setup.m_loop, client_fd));
+
+ KJ_EXPECT(false);
+ } catch (const std::runtime_error& e) {
+ std::string_view reason = e.what();
+ KJ_EXPECT(reason == "IPC client method call interrupted by disconnect.");
+ }
+ server_thread.join();
+}
+
+} // namespace
+} // namespace test
+} // namespace mp
### src/ipc/libmultiprocess/test/mp/test/foo-types.h
@@ -9,6 +9,7 @@
#include <mp/proxy-types.h>
// IWYU pragma: begin_exports
+#include <any>
#include <capnp/common.h>
#include <cstddef>
#include <mp/test/foo.capnp.h>
@@ -39,13 +40,15 @@ struct ExtendedCallback; // IWYU pragma: export
struct FooCallback; // IWYU pragma: export
struct FooFn; // IWYU pragma: export
struct FooInterface; // IWYU pragma: export
+struct BarInterface; // IWYU pragma: export
} // namespace messages
template <typename Output>
void CustomBuildField(TypeList<FooCustom>, Priority<1>, InvokeContext& invoke_context, const FooCustom& value, Output&& output)
{
BuildField(TypeList<std::string>(), invoke_context, output, value.v1);
output.setV2(value.v2);
+ BuildField(TypeList<std::vector<int>>(), invoke_context, output, value.v3);
}
template <typename Input, typename ReadDest>
@@ -55,13 +58,14 @@ decltype(auto) CustomReadField(TypeList<FooCustom>, Priority<1>, InvokeContext&
return read_dest.update([&](FooCustom& value) {
value.v1 = ReadField(TypeList<std::string>(), invoke_context, mp::Make<mp::ValueField>(custom.getV1()), ReadDestTemp<std::string>());
value.v2 = custom.getV2();
+ value.v3 = ReadField(TypeList<std::vector<int>>(), invoke_context, mp::Make<mp::ValueField>(custom.getV3()), ReadDestTemp<std::vector<int>>());
});
}
} // namespace test
template <typename Input>
-bool CustomHasField(TypeList<test::FooData>, InvokeContext& invoke_context, const Input& input)
+bool CustomHasField(TypeList<test::FooData>, InvokeContext&, const Input& input)
{
// Cap'n Proto C++ cannot distinguish null vs empty Data in List(Data), so
// interpret empty Data as null for this specific type.
@@ -72,31 +76,33 @@ inline void CustomBuildMessage(InvokeContext& invoke_context,
const test::FooMessage& src,
test::messages::FooMessage::Builder&& builder)
{
+ const auto& hook{invoke_context.connection.m_loop->testing_hook_misc};
+ if (hook) hook("build FooMessage");
builder.setMessage(src.message + " build");
}
-inline void CustomReadMessage(InvokeContext& invoke_context,
+inline void CustomReadMessage(InvokeContext&,
const test::messages::FooMessage::Reader& reader,
test::FooMessage& dest)
{
dest.message = std::string{reader.getMessage()} + " read";
}
-inline void CustomBuildMessage(InvokeContext& invoke_context,
+inline void CustomBuildMessage(InvokeContext&,
const test::FooMutable& src,
test::messages::FooMutable::Builder&& builder)
{
builder.setMessage(src.message + " build");
}
-inline void CustomReadMessage(InvokeContext& invoke_context,
+inline void CustomReadMessage(InvokeContext&,
const test::messages::FooMutable::Reader& reader,
test::FooMutable& dest)
{
dest.message = std::string{reader.getMessage()} + " read";
}
-inline void CustomPassMessage(InvokeContext& invoke_context,
+inline void CustomPassMessage(InvokeContext&,
const test::messages::FooMutable::Reader& reader,
test::messages::FooMutable::Builder builder,
std::function<void(test::FooMutable&)>&& fn)
### src/ipc/libmultiprocess/test/mp/test/foo.capnp
@@ -37,6 +37,12 @@ interface FooInterface $Proxy.wrap("mp::test::FooImplementation") {
callFnAsync @18 (context :Proxy.Context) -> ();
callIntFnAsync @21 (context :Proxy.Context, arg :Int32) -> (result :Int32);
passDataPointers @22 (arg :List(Data)) -> (result :List(Data));
+ listBars @25 (context :Proxy.Context, n :Int32) -> (result :List(BarInterface));
+ callMessageAsync @26 (context :Proxy.Context) -> (result :FooMessage);
+}
+
+interface FooInit $Proxy.wrap("mp::test::FooInit") {
+ construct @0 () -> ();
}
interface FooCallback $Proxy.wrap("mp::test::FooCallback") {
@@ -48,6 +54,11 @@ interface ExtendedCallback extends(FooCallback) $Proxy.wrap("mp::test::ExtendedC
callExtended @0 (context :Proxy.Context, arg :Int32) -> (result :Int32);
}
+interface BarInterface $Proxy.wrap("mp::test::Bar") {
+ destroy @0 (context :Proxy.Context) -> ();
+ value @1 (context :Proxy.Context) -> (result :Int32);
+}
+
interface FooFn $Proxy.wrap("ProxyCallback<std::function<int()>>") {
destroy @0 (context :Proxy.Context) -> ();
call @1 (context :Proxy.Context) -> (result :Int32);
@@ -61,11 +72,16 @@ struct FooStruct $Proxy.wrap("mp::test::FooStruct") {
hasOptionalInt @4 :Bool;
unorderedSetInt @5 :List(Int32) $Proxy.name("unordered_set_int");
mapStringInt @6 :List(StringIntPair) $Proxy.name("map_string_int");
+ setIntDesc @7 :List(Int32) $Proxy.name("set_int_desc");
+ pmrVectorInt @8 :List(Int32) $Proxy.name("pmr_vector_int");
+ unorderedSetStringHash @9 :List(Text) $Proxy.name("unordered_set_string_hash");
+ mapStringIntDesc @10 :List(StringIntPair) $Proxy.name("map_string_int_desc");
}
struct FooCustom $Proxy.wrap("mp::test::FooCustom") {
v1 @0 :Text;
v2 @1 :Int32;
+ v3 @2 :List(Int32);
}
struct FooEmpty $Proxy.wrap("mp::test::FooEmpty") {
### src/ipc/libmultiprocess/test/mp/test/foo.h
@@ -6,9 +6,11 @@
#define MP_TEST_FOO_H
#include <cassert>
+#include <cstddef>
#include <functional>
#include <map>
#include <memory>
+#include <memory_resource>
#include <optional>
#include <string>
#include <set>
@@ -18,6 +20,12 @@
namespace mp {
namespace test {
+//! Hash function used to test std::unordered_set with a custom hash.
+struct FooHash
+{
+ size_t operator()(const std::string& str) const { return str.size(); }
+};
+
struct FooStruct
{
std::string name;
@@ -26,6 +34,11 @@ struct FooStruct
std::optional<int> optional_int;
std::unordered_set<int> unordered_set_int;
std::map<std::string, int> map_string_int;
+ // Containers with non-default comparator, allocator, and hash types.
+ std::set<int, std::greater<>> set_int_desc;
+ std::pmr::vector<int> pmr_vector_int;
+ std::unordered_set<std::string, FooHash> unordered_set_string_hash;
+ std::map<std::string, int, std::greater<>> map_string_int_desc;
};
enum class FooEnum : uint8_t { ONE = 1, TWO = 2, };
@@ -34,6 +47,7 @@ struct FooCustom
{
std::string v1;
int v2;
+ std::vector<int> v3;
};
struct FooEmpty
@@ -66,6 +80,27 @@ class ExtendedCallback : public FooCallback
virtual int callExtended(int arg) = 0;
};
+class FooInit
+{
+};
+
+//! A second, arbitrary interface used to test returning a
+//! list of interface objects.
+class Bar
+{
+public:
+ virtual ~Bar() = default;
+ virtual int value() = 0;
+};
+
+//! Concrete Bar that returns a fixed value, used by listBars tests.
+class SimpleBar : public Bar
+{
+public:
+ explicit SimpleBar(int value) : m_value(value) {}
+ int value() override { return m_value; }
+ int m_value;
+};
class FooImplementation
{
public:
@@ -74,7 +109,7 @@ class FooImplementation
void addInOut(int x, int& sum) { sum += x; }
int mapSize(const std::map<std::string, std::string>& map) { return map.size(); }
FooStruct pass(FooStruct foo) { return foo; }
- void raise(FooStruct foo) { throw foo; }
+ [[noreturn]] void raise(FooStruct foo) { throw foo; }
void initThreadMap() {}
int callback(FooCallback& callback, int arg) { return callback.call(arg); }
int callbackUnique(std::unique_ptr<FooCallback> callback, int arg) { return callback->call(arg); }
@@ -91,10 +126,18 @@ class FooImplementation
double passDouble(double value) { return value; }
int passFn(std::function<int()> fn) { return fn(); }
std::vector<FooDataRef> passDataPointers(std::vector<FooDataRef> values) { return values; }
+ std::vector<std::unique_ptr<Bar>> listBars(int n)
+ {
+ std::vector<std::unique_ptr<Bar>> result;
+ result.reserve(n);
+ for (int i = 0; i < n; ++i) result.push_back(std::make_unique<SimpleBar>(i));
+ return result;
+ }
std::shared_ptr<FooCallback> m_callback;
void callFn() { assert(m_fn); m_fn(); }
void callFnAsync() { assert(m_fn); m_fn(); }
int callIntFnAsync(int arg) { assert(m_int_fn); return m_int_fn(arg); }
+ FooMessage callMessageAsync() { assert(m_fn); m_fn(); return {}; }
std::function<void()> m_fn;
std::function<int(int)> m_int_fn;
};
### src/ipc/libmultiprocess/test/mp/test/listen_tests.cpp
@@ -2,20 +2,21 @@
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
-#include <cassert>
-#include <filesystem>
+#include "common.h"
+#include "unixlistener.h"
#include <mp/test/foo.capnp.h>
#include <mp/test/foo.capnp.proxy.h>
#include <chrono>
+#include <compare>
#include <condition_variable>
-#include <cstdlib>
#include <cstring>
#include <future>
#include <functional>
#include <kj/async.h>
#include <kj/common.h>
#include <kj/debug.h>
+#include <kj/function.h>
#include <kj/memory.h>
#include <kj/test.h>
#include <memory>
@@ -24,10 +25,7 @@
#include <mp/util.h>
#include <ratio> // IWYU pragma: keep
#include <optional>
-#include <stdexcept>
#include <string>
-#include <sys/socket.h>
-#include <sys/un.h>
#include <thread>
#include <unistd.h>
@@ -37,65 +35,6 @@ namespace {
constexpr auto FAILURE_TIMEOUT = std::chrono::seconds{30};
-//! Owns a temporary Unix-domain listening socket used by ListenSetup. Tests call
-//! Connect() to create client socket FDs and release() to transfer the listening
-//! FD to ListenConnections().
-class UnixListener
-{
-public:
- UnixListener()
- {
- std::string dir_template = (std::filesystem::temp_directory_path() / "mptest-listener-XXXXXX").string();
- char* dir = mkdtemp(dir_template.data());
- KJ_REQUIRE(dir != nullptr);
- m_dir = dir;
- m_path = m_dir + "/socket";
-
- m_fd = socket(AF_UNIX, SOCK_STREAM, 0);
- KJ_REQUIRE(m_fd >= 0);
-
- sockaddr_un addr{};
- addr.sun_family = AF_UNIX;
- KJ_REQUIRE(m_path.size() < sizeof(addr.sun_path));
- std::strncpy(addr.sun_path, m_path.c_str(), sizeof(addr.sun_path) - 1);
- KJ_REQUIRE(bind(m_fd, reinterpret_cast<sockaddr*>(&addr), sizeof(addr)) == 0);
- KJ_REQUIRE(listen(m_fd, SOMAXCONN) == 0);
- }
-
- ~UnixListener()
- {
- if (m_fd >= 0) close(m_fd);
- if (!m_path.empty()) unlink(m_path.c_str());
- if (!m_dir.empty()) rmdir(m_dir.c_str());
- }
-
- int release()
- {
- assert(m_fd >= 0);
- int fd = m_fd;
- m_fd = -1;
- return fd;
- }
-
- int MakeConnectedSocket() const
- {
- int fd = socket(AF_UNIX, SOCK_STREAM, 0);
- KJ_REQUIRE(fd >= 0);
-
- sockaddr_un addr{};
- addr.sun_family = AF_UNIX;
- KJ_REQUIRE(m_path.size() < sizeof(addr.sun_path));
- std::strncpy(addr.sun_path, m_path.c_str(), sizeof(addr.sun_path) - 1);
- KJ_REQUIRE(connect(fd, reinterpret_cast<sockaddr*>(&addr), sizeof(addr)) == 0);
- return fd;
- }
-
-private:
- int m_fd{-1};
- std::string m_dir;
- std::string m_path;
-};
-
//! Runs a client EventLoop on its own thread and connects one socket FD to the
//! server. The constructed ProxyClient can be used by the test thread to make
//! calls over that connection.
@@ -104,11 +43,8 @@ class ClientSetup
public:
explicit ClientSetup(int fd)
: thread([this, fd] {
- EventLoop loop("mptest-client", [](mp::LogMessage log) {
- KJ_LOG(INFO, log.level, log.message);
- if (log.level == mp::Log::Raise) throw std::runtime_error(log.message);
- });
- client_promise.set_value(ConnectStream<messages::FooInterface>(loop, fd));
+ EventLoop loop("mptest-client", DefaultLogHandler);
+ client_promise.set_value(ConnectStream<messages::FooInterface>(loop, MakeStream(loop, fd)));
loop.loop();
})
{
@@ -135,12 +71,10 @@ class ClientSetup
class ListenSetup
{
public:
- explicit ListenSetup(std::optional<size_t> max_connections = std::nullopt)
- : thread([this, max_connections] {
- EventLoop loop("mptest-server", [](mp::LogMessage log) {
- KJ_LOG(INFO, log.level, log.message);
- if (log.level == mp::Log::Raise) throw std::runtime_error(log.message);
- });
+ explicit ListenSetup(std::optional<size_t> max_connections = std::nullopt,
+ mp::LogFn log_handler = DefaultLogHandler)
+ : thread([this, max_connections, log_handler] {
+ EventLoop loop("mptest-server", log_handler);
loop.testing_hook_disconnected = [&] {
Lock lock(counter_mutex);
++disconnected_count;
@@ -225,7 +159,7 @@ KJ_TEST("ListenConnections enforces a local connection limit")
{
// With max-connections=1, the second socket can connect to the kernel
// backlog, but ListenConnections should not accept or serve it until the
- // first accepts clients disconnects.
+ // first accepted client disconnects.
ListenSetup server(/*max_connections=*/1);
@@ -289,6 +223,43 @@ KJ_TEST("ListenConnections accepts multiple connections")
KJ_EXPECT(client3->client->add(3, 4) == 7);
}
+KJ_TEST("ListenConnections handles a client that disconnects before being accepted")
+{
+ Mutex mutex;
+ bool accept_error = false;
+ ListenSetup server(std::nullopt, [&](mp::LogMessage log) {
+ // On macOS, accept() can fail if the peer closes the connection before it is accepted.
+ // The event loop then reports this as an uncaught task exception. We catch and ignore
+ // this specific error here so that the corresponding CI job does not fail.
+ //
+ // This is a Cap'n Proto bug, fixed by https://github.com/capnproto/capnproto/pull/2748
+ if (log.level == mp::Log::Error && log.message.find("Uncaught exception in daemonized task.") != std::string::npos) {
+ Lock lock(mutex);
+ accept_error = true;
+ }
+ DefaultLogHandler(log);
+ });
+
+ // This is racy, if the close does not happen before accept(),
+ // the connection is accepted normally.
+ int fd = server.listener.MakeConnectedSocket();
+ KJ_SYSCALL(close(fd));
+
+ // Wait for the connection to either be accepted and disconnected, or fail
+ // to be accepted and log the error above.
+ const auto deadline = std::chrono::steady_clock::now() + FAILURE_TIMEOUT;
+ auto accept_error_seen = [&] { Lock lock(mutex); return accept_error; };
+ while (!accept_error_seen() && server.DisconnectedCount() < 1 && std::chrono::steady_clock::now() < deadline) {
+ std::this_thread::sleep_for(std::chrono::milliseconds(1));
+ }
+
+#ifndef __APPLE__
+ // No error is expected on platforms other than macOS.
+ KJ_REQUIRE(!accept_error_seen());
+#endif
+ KJ_EXPECT(server.DisconnectedCount() == (accept_error_seen() ? 0 : 1));
+}
+
} // namespace
} // namespace test
} // namespace mp
### src/ipc/libmultiprocess/test/mp/test/spawn_tests.cpp
@@ -4,28 +4,37 @@
#include <mp/util.h>
+#include <kj/common.h>
+#include <kj/debug.h>
#include <kj/test.h>
+#include <cerrno>
#include <chrono>
#include <compare>
#include <condition_variable>
#include <csignal>
#include <cstdlib>
#include <mutex>
#include <string>
+#include <string_view>
+#include <system_error>
#include <sys/wait.h>
#include <thread>
+#include <tuple>
#include <unistd.h>
+#include <utility>
#include <vector>
+namespace mp {
+namespace test {
namespace {
constexpr auto FAILURE_TIMEOUT = std::chrono::seconds{30};
// Poll for child process exit using waitpid(..., WNOHANG) until the child exits
// or timeout expires. Returns true if the child exited and status_out was set.
// Returns false on timeout or error.
-static bool WaitPidWithTimeout(int pid, std::chrono::milliseconds timeout, int& status_out)
+static bool WaitPidWithTimeout(ProcessId pid, std::chrono::milliseconds timeout, int& status_out)
{
const auto deadline = std::chrono::steady_clock::now() + timeout;
while (std::chrono::steady_clock::now() < deadline) {
@@ -58,7 +67,7 @@ KJ_TEST("SpawnProcess does not run callback in child")
std::thread locker([&] {
std::unique_lock<std::mutex> target_lock(target_mutex);
{
- std::lock_guard<std::mutex> g(control_mutex);
+ std::scoped_lock g{control_mutex};
locked = true;
}
control_cv.notify_one();
@@ -80,20 +89,19 @@ KJ_TEST("SpawnProcess does not run callback in child")
// test failures.
std::this_thread::sleep_for(std::chrono::milliseconds{50});
{
- std::lock_guard<std::mutex> g(control_mutex);
+ std::scoped_lock g{control_mutex};
release = true;
}
control_cv.notify_one();
});
- int pid{-1};
- const int fd{mp::SpawnProcess(pid, [&](int child_fd) -> std::vector<std::string> {
+ const auto [pid, socket]{SpawnProcess([&](SpawnConnectInfo connect_info) -> std::vector<std::string> {
// If this callback runs in the post-fork child, target_mutex appears
// locked forever (the owning thread does not exist), so this deadlocks.
- std::lock_guard<std::mutex> g(target_mutex);
- return {"true", std::to_string(child_fd)};
+ std::scoped_lock g{target_mutex};
+ return {"true", std::move(connect_info)};
})};
- ::close(fd);
+ ::close(socket);
int status{0};
// Give the child some time to exit. If it does not, terminate it and
@@ -110,3 +118,18 @@ KJ_TEST("SpawnProcess does not run callback in child")
KJ_EXPECT(exited, "Timeout waiting for child process to exit");
KJ_EXPECT(WIFEXITED(status) && WEXITSTATUS(status) == 0);
}
+
+KJ_TEST("SpawnProcess throws on execvp failure")
+{
+ try {
+ SpawnProcess([&](SpawnConnectInfo) -> std::vector<std::string> {
+ return {"/nonexistent/binary"};
+ });
+ KJ_EXPECT(false, "expected SpawnProcess to throw");
+ } catch (const std::system_error& e) {
+ KJ_EXPECT(e.code().value() == ENOENT);
+ KJ_EXPECT(std::string_view{e.what()}.find("execvp") != std::string_view::npos);
+ }
+}
+} // namespace test
+} // namespace mp
### src/ipc/libmultiprocess/test/mp/test/test.cpp
@@ -5,9 +5,9 @@
#include <mp/test/foo.capnp.h>
#include <mp/test/foo.capnp.proxy.h>
+#include <any>
#include <atomic>
#include <capnp/capability.h>
-#include <capnp/rpc.h>
#include <cassert>
#include <chrono>
#include <condition_variable>
@@ -20,11 +20,14 @@
#include <kj/common.h>
#include <kj/exception.h>
#include <kj/debug.h>
+#include <kj/function.h>
#include <kj/memory.h>
#include <kj/string.h>
#include <kj/test.h>
+#include <list> // IWYU pragma: keep
#include <map>
#include <memory>
+#include <mp/config.h>
#include <mp/proxy.h>
#include <mp/proxy.capnp.h>
#include <mp/proxy-io.h>
@@ -72,6 +75,12 @@ static_assert(std::is_integral_v<decltype(kMP_MINOR_VERSION)>, "MP_MINOR_VERSION
* this to be true to simplify shutdown and avoid needing to call
* client_disconnect manually, but false allows testing more ProxyClient
* behavior and the "IPC client method called after disconnect" code path.
+ *
+ * Also accepts an analogous server_owns_connection option controlling
+ * whether the server Connection is erased automatically when the peer
+ * disconnects. False is needed by tests that need to keep the server
+ * Connection alive across a disconnect notification, tearing it down
+ * explicitly later via server_disconnect() instead.
*/
class TestSetup
{
@@ -86,7 +95,7 @@ class TestSetup
//! not start until the other members are initialized.
std::thread thread;
- TestSetup(bool client_owns_connection = true)
+ TestSetup(bool client_owns_connection = true, bool server_owns_connection = true)
: thread{[&] {
EventLoop loop("mptest", [](mp::LogMessage log) {
// Info logs are not printed by default, but will be shown with `mptest --verbose`
@@ -95,30 +104,21 @@ class TestSetup
});
auto pipe = loop.m_io_context.provider->newTwoWayPipe();
- auto server_connection =
- std::make_unique<Connection>(loop, kj::mv(pipe.ends[0]), [&](Connection& connection) {
- auto server_proxy = kj::heap<ProxyServer<messages::FooInterface>>(
- std::make_shared<FooImplementation>(), connection);
- server = server_proxy;
- return capnp::Capability::Client(kj::mv(server_proxy));
- });
- server_disconnect = [&] { loop.sync([&] { server_connection.reset(); }); };
+ auto server_result = ServeStream<messages::FooInterface>(
+ loop, kj::mv(pipe.ends[0]), std::make_shared<FooImplementation>(), server_owns_connection);
+ server = server_result.first;
+ EventLoop::Connections::iterator server_it = server_result.second;
+ server_disconnect = [&] { loop.sync([&] { loop.m_incoming_connections.erase(server_it); }); };
server_disconnect_later = [&] {
assert(std::this_thread::get_id() == loop.m_thread_id);
- loop.m_task_set->add(kj::evalLater([&] { server_connection.reset(); }));
+ loop.m_task_set->add(kj::evalLater([&] { loop.m_incoming_connections.erase(server_it); }));
};
- // Set handler to destroy the server when the client disconnects. This
- // is ignored if server_disconnect() is called instead.
- server_connection->onDisconnect([&] { server_connection.reset(); });
-
- auto client_connection = std::make_unique<Connection>(loop, kj::mv(pipe.ends[1]));
- auto client_proxy = std::make_unique<ProxyClient<messages::FooInterface>>(
- client_connection->m_rpc_system->bootstrap(ServerVatId().vat_id).castAs<messages::FooInterface>(),
- client_connection.get(), /* destroy_connection= */ client_owns_connection);
- if (client_owns_connection) {
- (void)client_connection.release();
- } else {
- client_disconnect = [&] { loop.sync([&] { client_connection.reset(); }); };
+
+ auto client_proxy =
+ ConnectStream<messages::FooInterface>(loop, kj::mv(pipe.ends[1]), client_owns_connection);
+ if (!client_owns_connection) {
+ Connection* client_connection = client_proxy->m_context.connection;
+ client_disconnect = [&loop, client_connection] { loop.sync([&] { delete client_connection; }); };
}
client_promise.set_value(std::move(client_proxy));
@@ -164,6 +164,10 @@ KJ_TEST("Call FooInterface methods")
in.optional_int = 3;
in.map_string_int.emplace("a", 1);
in.map_string_int.emplace("b", 2);
+ in.set_int_desc = {1, 2, 3};
+ in.pmr_vector_int = {3, 1, 2};
+ in.unordered_set_string_hash = {"a", "bb"};
+ in.map_string_int_desc = {{"a", 1}, {"b", 2}};
FooStruct out = foo->pass(in);
KJ_EXPECT(in.name == out.name);
KJ_EXPECT(in.set_int.size() == out.set_int.size());
@@ -172,7 +176,7 @@ KJ_TEST("Call FooInterface methods")
}
KJ_EXPECT(in.unordered_set_int.size() == out.unordered_set_int.size());
for (const auto& elem : in.unordered_set_int) {
- KJ_EXPECT(out.unordered_set_int.count(elem) == 1);
+ KJ_EXPECT(out.unordered_set_int.contains(elem));
}
KJ_EXPECT(in.vector_bool.size() == out.vector_bool.size());
for (size_t i = 0; i < in.vector_bool.size(); ++i) {
@@ -184,6 +188,10 @@ KJ_TEST("Call FooInterface methods")
KJ_EXPECT(init->first == outit->first);
KJ_EXPECT(init->second == outit->second);
}
+ KJ_EXPECT(in.set_int_desc == out.set_int_desc);
+ KJ_EXPECT(in.pmr_vector_int == out.pmr_vector_int);
+ KJ_EXPECT(in.unordered_set_string_hash == out.unordered_set_string_hash);
+ KJ_EXPECT(in.map_string_int_desc == out.map_string_int_desc);
// Additional checks for std::optional member
KJ_EXPECT(foo->pass(in).optional_int == 3);
@@ -233,9 +241,11 @@ KJ_TEST("Call FooInterface methods")
FooCustom custom_in;
custom_in.v1 = "v1";
custom_in.v2 = 5;
+ custom_in.v3 = {10, 20, 30};
FooCustom custom_out = foo->passCustom(custom_in);
KJ_EXPECT(custom_in.v1 == custom_out.v1);
KJ_EXPECT(custom_in.v2 == custom_out.v2);
+ KJ_EXPECT(custom_in.v3 == custom_out.v3);
foo->passEmpty(FooEmpty{});
@@ -269,6 +279,16 @@ KJ_TEST("Call FooInterface methods")
KJ_REQUIRE(data_out[0] != nullptr);
KJ_EXPECT(*data_out[0] == *data_in[0]);
KJ_EXPECT(!data_out[1]);
+
+ // Test returning vector<unique_ptr<interface>> from server. This exercises
+ // BuildList with interface element types, which requires non-const iteration
+ // so unique_ptr::release() can transfer ownership to the proxy server.
+ std::vector<std::unique_ptr<Bar>> bars{foo->listBars(3)};
+ KJ_REQUIRE(bars.size() == 3u);
+ for (int i = 0; i < 3; ++i) {
+ KJ_REQUIRE(bars[i] != nullptr);
+ KJ_EXPECT(bars[i]->value() == i);
+ }
}
KJ_TEST("Call IPC method after client connection is closed")
@@ -288,7 +308,16 @@ KJ_TEST("Calling IPC method after server connection is closed")
KJ_EXPECT(foo->add(1, 2) == 3);
setup.server_disconnect();
- EXPECT_EXCEPTION(foo->add(1, 2), "IPC client method call interrupted by disconnect.");
+ try {
+ foo->add(1, 2);
+ KJ_EXPECT(false);
+ } catch (const std::runtime_error& e) {
+ std::string_view reason{e.what()};
+
+ // The disconnect handler may delete the connection before the
+ // call is processed or while the call is in flight, both errors are possible.
+ KJ_EXPECT(reason == "IPC client method called after disconnect." || reason == "IPC client method call interrupted by disconnect.");
+ }
}
KJ_TEST("Calling IPC method and disconnecting during the call")
@@ -346,6 +375,95 @@ KJ_TEST("Calling IPC method, disconnecting and blocking during the call")
signal.set_value();
}
+KJ_TEST("Calling async IPC method with a remote disconnect while results are built")
+{
+ // Regression test for bitcoin-core/libmultiprocess#348, a data race
+ // reported by ThreadSanitizer where a server worker thread called
+ // call_context.getResults() while the event loop thread was tearing down
+ // Cap'n Proto connection state (capnp::_::RpcConnectionState::disconnect()
+ // overwriting the RpcConnectionState::connection field) after an abrupt
+ // remote disconnect.
+ //
+ // The test makes an async IPC call (callMessageAsync) whose method body
+ // just signals the main thread. When the worker thread returns from the
+ // method body, it calls call_context.getResults(), reading the connection
+ // state, and starts serializing the FooMessage result, where the
+ // testing_hook_misc hook set below makes it sleep. Meanwhile the main
+ // thread destroys the client connection, and the event loop thread
+ // processes the resulting EOF, running RpcConnectionState::disconnect()
+ // and overwriting the connection state the worker thread just read, with
+ // no synchronization between the two accesses.
+ //
+ // Two details are essential for ThreadSanitizer to detect the race:
+ //
+ // - There must be no synchronization between the worker thread's
+ // getResults() call and the event loop thread's disconnect processing.
+ // The worker signals the main thread *before* getResults() and sleeps
+ // (sleeping creates no happens-before edge) across the disconnect, so
+ // the racing accesses are not ordered by any of the test's own
+ // synchronization.
+ //
+ // - The worker thread's read should come *before* the event loop thread's
+ // write, close in time. In the write-then-read order (e.g. method
+ // returning long after the disconnect), the race exists too, but
+ // ThreadSanitizer usually misses it: right after disconnect() writes the
+ // connection field, the loop thread's teardown destructors re-read it
+ // many times (~ImportClient etc. check connection.is<Connected>() to
+ // decide whether to send messages), evicting the write from the
+ // per-granule shadow history before a late reader comes along.
+ //
+ // The server Connection object is deliberately kept alive during all this
+ // by constructing TestSetup with server_owns_connection=false: destroying
+ // it would cancel the in-flight request (Connection::~Connection calls
+ // m_canceler.cancel(), setting request_canceled) and the worker would
+ // throw InterruptException instead of proceeding into getResults().
+ // Keeping it alive matches the window in the original report, where the
+ // worker races with capnp's own internal teardown, which runs before any
+ // onDisconnect notification. The connection is destroyed explicitly at the
+ // end of the test instead.
+
+ TestSetup setup{/*client_owns_connection=*/false, /*server_owns_connection=*/false};
+ ProxyClient<messages::FooInterface>* foo = setup.client.get();
+ KJ_EXPECT(foo->add(1, 2) == 3);
+ foo->initThreadMap();
+
+ // Signaled by the worker thread when the method body runs, just before it
+ // returns and the worker calls getResults() and serializes the results.
+ std::promise<void> fn_called;
+ setup.server->m_impl->m_fn = [&] { fn_called.set_value(); };
+
+ // Keep the worker thread inside the results-building step, without
+ // synchronizing, while the event loop thread processes the disconnect.
+ EventLoop& loop = *setup.server->m_context.connection->m_loop;
+ loop.testing_hook_misc = [](std::any arg) {
+ if (const char* const* tag{std::any_cast<const char*>(&arg)};
+ tag && std::string_view{*tag} == "build FooMessage") {
+ std::this_thread::sleep_for(std::chrono::milliseconds(200));
+ }
+ };
+
+ // Signaled by the worker thread when it is completely done with the
+ // request, including serializing the results.
+ std::promise<void> request_done;
+ loop.testing_hook_async_request_done = [&] { request_done.set_value(); };
+
+ // Make the IPC call from a separate thread so this thread can trigger the
+ // client disconnect while the call is executing.
+ std::thread caller{[&] {
+ EXPECT_EXCEPTION(foo->callMessageAsync(), "IPC client method call interrupted by disconnect.");
+ }};
+
+ fn_called.get_future().get();
+ setup.client_disconnect();
+ caller.join();
+
+ // Wait for the worker thread to finish the request, then tear down the
+ // server connection that was deliberately kept alive above, so the event
+ // loop is able to exit.
+ request_done.get_future().get();
+ setup.server_disconnect();
+}
+
KJ_TEST("Worker thread destroyed before it is initialized")
{
// Regression test for bitcoin/bitcoin#34711, bitcoin/bitcoin#34756 where a
@@ -462,7 +580,7 @@ KJ_TEST("Make simultaneous IPC calls on single remote thread")
// that will be used for the test.
setup.server->m_impl->m_fn = [&] {};
foo->callFnAsync();
- ThreadContext& tc{g_thread_context};
+ ThreadContext& tc{CurrentThread()};
Thread::Client *callback_thread, *request_thread;
foo->m_context.loop->sync([&] {
Lock lock(tc.waiter->m_mutex);
@@ -516,7 +634,7 @@ KJ_TEST("Call async IPC method dispatched to pool thread")
foo->initThreadMap();
setup.server->m_impl->m_int_fn = [](int n) { return n * 2; };
- ThreadContext& tc{g_thread_context};
+ ThreadContext& tc{CurrentThread()};
std::atomic<size_t> running{3};
std::promise<void> pool_ready;
foo->m_context.loop->sync([&] {
@@ -550,6 +668,68 @@ KJ_TEST("Call async IPC method dispatched to pool thread")
}
}
+#ifdef HAVE_PTHREAD_GETNAME_NP
+KJ_TEST("Worker thread has OS thread name")
+{
+ TestSetup setup;
+ ProxyClient<messages::FooInterface>* foo = setup.client.get();
+ foo->initThreadMap();
+
+ std::promise<std::string> thread_name;
+ setup.server->m_impl->m_fn = [&] { thread_name.set_value(ThreadName("")); };
+ foo->callFnAsync();
+
+ const std::string name{thread_name.get_future().get()};
+ KJ_EXPECT(name.find("/capnp-worker-") != std::string::npos, name);
+}
+
+KJ_TEST("Pool thread has OS thread name")
+{
+ TestSetup setup;
+ ProxyClient<messages::FooInterface>* foo = setup.client.get();
+ foo->initThreadMap();
+
+ std::promise<std::string> thread_name;
+ setup.server->m_impl->m_fn = [&] { thread_name.set_value(ThreadName("")); };
+
+ std::promise<void> pool_ready;
+ foo->m_context.loop->sync([&] {
+ auto pool_req = foo->m_context.connection->m_thread_map.makePoolRequest();
+ pool_req.setCount(1);
+ foo->m_context.loop->m_task_set->add(
+ pool_req.send().then([&](auto&&) { pool_ready.set_value(); }));
+ });
+ pool_ready.get_future().get();
+
+ std::promise<void> done;
+ foo->m_context.loop->sync([&] {
+ auto request{foo->m_client.callFnAsyncRequest()};
+ foo->m_context.loop->m_task_set->add(
+ request.send().then([&](auto&&) { done.set_value(); }));
+ });
+ // Wait for the reply before returning, so the connection is not torn down
+ // while the request is still in flight.
+ done.get_future().get();
+
+ const std::string name{thread_name.get_future().get()};
+ KJ_EXPECT(name.find("/capnp-pool-0-") != std::string::npos, name);
+}
+
+KJ_TEST("Async cleanup thread has OS thread name")
+{
+ std::promise<std::string> thread_name;
+ {
+ TestSetup setup;
+ // FooInterface has no destroy method, so the server ProxyServer runs
+ // its cleanup functions on the async thread when it is destroyed.
+ setup.server->m_context.cleanup_fns.emplace_front(
+ [&] { thread_name.set_value(ThreadName("")); });
+ }
+ const std::string name{thread_name.get_future().get()};
+ KJ_EXPECT(name.find("/capnp-async-") != std::string::npos, name);
+}
+#endif // HAVE_PTHREAD_GETNAME_NP
+
KJ_TEST("Call async IPC method without thread or pool errors correctly")
{
TestSetup setup;
### src/ipc/libmultiprocess/test/mp/test/unixlistener.h
@@ -0,0 +1,84 @@
+// Copyright (c) The Bitcoin Core developers
+// Distributed under the MIT software license, see the accompanying
+// file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+#ifndef MP_TEST_UNIXLISTENER_H
+#define MP_TEST_UNIXLISTENER_H
+
+#include <kj/test.h>
+#include <sys/socket.h>
+#include <sys/un.h>
+#include <unistd.h>
+
+#include <cassert>
+#include <cstdlib>
+#include <cstring>
+#include <filesystem>
+#include <string>
+
+namespace mp {
+namespace test {
+
+//! Owns a temporary Unix-domain listening socket. Tests call
+//! MakeConnectedSocket() to create client socket FDs and release() to transfer
+//! ownership of the listening FD.
+class UnixListener
+{
+public:
+ UnixListener()
+ {
+ std::string dir_template = (std::filesystem::temp_directory_path() / "mptest-listener-XXXXXX").string();
+ char* dir = mkdtemp(dir_template.data());
+ KJ_REQUIRE(dir != nullptr);
+ m_dir = dir;
+ m_path = m_dir + "/socket";
+
+ m_fd = socket(AF_UNIX, SOCK_STREAM, 0);
+ KJ_REQUIRE(m_fd >= 0);
+
+ sockaddr_un addr{};
+ addr.sun_family = AF_UNIX;
+ KJ_REQUIRE(m_path.size() < sizeof(addr.sun_path));
+ std::strncpy(addr.sun_path, m_path.c_str(), sizeof(addr.sun_path) - 1);
+ KJ_REQUIRE(bind(m_fd, reinterpret_cast<sockaddr*>(&addr), sizeof(addr)) == 0);
+ KJ_REQUIRE(listen(m_fd, SOMAXCONN) == 0);
+ }
+
+ ~UnixListener()
+ {
+ if (m_fd >= 0) close(m_fd);
+ if (!m_path.empty()) unlink(m_path.c_str());
+ if (!m_dir.empty()) rmdir(m_dir.c_str());
+ }
+
+ int release()
+ {
+ assert(m_fd >= 0);
+ int fd = m_fd;
+ m_fd = -1;
+ return fd;
+ }
+
+ int MakeConnectedSocket() const
+ {
+ int fd = socket(AF_UNIX, SOCK_STREAM, 0);
+ KJ_REQUIRE(fd >= 0);
+
+ sockaddr_un addr{};
+ addr.sun_family = AF_UNIX;
+ KJ_REQUIRE(m_path.size() < sizeof(addr.sun_path));
+ std::strncpy(addr.sun_path, m_path.c_str(), sizeof(addr.sun_path) - 1);
+ KJ_REQUIRE(connect(fd, reinterpret_cast<sockaddr*>(&addr), sizeof(addr)) == 0);
+ return fd;
+ }
+
+private:
+ int m_fd{-1};
+ std::string m_dir;
+ std::string m_path;
+};
+
+} // namespace test
+} // namespace mp
+
+#endif // MP_TEST_UNIXLISTENER_HWhy this scored 59/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.