http: prevent race condition between worker thread and I/O thread
What changed, and why it matters
This commit fixes a subtle timing bug in Bitcoin Core's built-in HTTP server. Under a specific race between a worker thread preparing a response and the I/O thread sending data, a connection could get permanently stuck waiting to write even though there was nothing left to send. Once stuck, the server would stop reading new requests from that client, effectively hanging the connection. The fix moves the 'ready to send' flag update under the same lock as the send buffer and protects reads of that flag with a separate lock to avoid deadlocks.
Treat as a reliability/DoS-hardening fix. Backport to maintained branches if HTTP RPC/REST interface is exposed. No immediate emergency response required because exploitation appears probabilistic and only affects an existing connection, but node operators serving RPC should upgrade in due course.
Security signals we found
Race condition between worker thread and I/O thread on shared state
Potential denial-of-service via hung HTTP connection
Lock ordering comment to prevent deadlock regression
Change from atomic bool to mutex-guarded bool indicating synchronization semantics changed
Evidence from the diff
The patch resolves a race condition in HTTPRequest::WriteReply() and HTTPServer::GenerateWaitSockets() involving m_send_ready and m_send_buffer. Previously, WriteReply() could append data to m_send_buffer, the I/O thread could drain it and clear m_send_ready, and then WriteReply() could set m_send_ready=true on an now-empty buffer. GenerateWaitSockets() would then poll only for write events, never read again, wedging the connection. The fix: (1) sets m_send_ready=true while still holding m_send_mutex and only when the buffer was not already empty; (2) reads m_send_ready under m_send_mutex in GenerateWaitSockets(), with a comment warning against lock-order inversion with m_sock_mutex; and (3) changes m_send_ready from std::atomic_bool to a plain bool GUARDED_BY(m_send_mutex).
Changed components
src/httpserver.cppsrc/httpserver.hHTTPRequest::WriteReplyHTTPServer::GenerateWaitSocketsHTTPRemoteClient::m_send_readyInspect captured patch +24 / −10
diff --git a/src/httpserver.cpp b/src/httpserver.cpp
index 75ccf652..a05d19db 100644
--- a/src/httpserver.cpp
+++ b/src/httpserver.cpp
@@ -595,6 +595,16 @@ void HTTPRequest::WriteReply(HTTPStatusCode status, std::span<const std::byte> r
// data. The original data will go out of scope when WriteReply() returns.
// This is analogous to the memcpy() in libevent's evbuffer_add()
m_client->m_send_buffer.insert(m_client->m_send_buffer.end(), reply_body.begin(), reply_body.end());
+
+ // If the buffer already held data, the I/O thread is (or soon will be)
+ // draining it, so flag that there is more data to send. This must happen
+ // while holding m_send_mutex and while the buffer is known non-empty:
+ // setting m_send_ready after releasing the lock would race with the I/O
+ // thread draining the buffer to empty and clearing m_send_ready in
+ // between, leaving m_send_ready set on an empty buffer. The I/O loop would
+ // then only ever poll the socket for writeability, never read the client's
+ // next request, and wedge the connection.
+ if (!send_buffer_was_empty) m_client->m_send_ready = true;
}
LogDebug(
@@ -611,10 +621,6 @@ void HTTPRequest::WriteReply(HTTPStatusCode status, std::span<const std::byte> r
// of waiting for the next iteration of the I/O loop.
if (send_buffer_was_empty) {
m_client->MaybeSendBytesFromBuffer();
- } else {
- // Inform HTTPServer I/O that data is ready to be sent to this client
- // in the next loop iteration.
- m_client->m_send_ready = true;
}
// Signal to the I/O loop that we are ready to handle the next request.
@@ -935,7 +941,12 @@ HTTPServer::IOReadiness HTTPServer::GenerateWaitSockets() const
// Check if client is ready to send data. Don't try to receive again
// until the send buffer is cleared (all data sent to client).
- Sock::Event event = (http_client->m_send_ready ? Sock::SendEvent : Sock::RecvEvent);
+ // Keep this as a separate critical section from the m_sock_mutex one above:
+ // never hold m_sock_mutex and m_send_mutex at the same time here.
+ // MaybeSendBytesFromBuffer() locks m_send_mutex then m_sock_mutex, so nesting
+ // them in the opposite order here would risk a lock-order inversion deadlock.
+ const bool send_ready{WITH_LOCK(http_client->m_send_mutex, return http_client->m_send_ready;)};
+ Sock::Event event = (send_ready ? Sock::SendEvent : Sock::RecvEvent);
io_readiness.events_per_sock.emplace(sock, Sock::Events{event});
io_readiness.httpclients_per_sock.emplace(sock, http_client);
}
diff --git a/src/httpserver.h b/src/httpserver.h
index 792b3690..9031eb61 100644
--- a/src/httpserver.h
+++ b/src/httpserver.h
@@ -483,11 +483,14 @@ public:
/// @}
/**
- * Set true by worker threads after writing a response to m_send_buffer.
- * Set false by the HTTPServer I/O thread after flushing m_send_buffer.
- * Checked in the HTTPServer I/O loop to avoid locking m_send_mutex if there's nothing to send.
- */
- std::atomic_bool m_send_ready{false};
+ * Set true by worker threads after writing a response to m_send_buffer.
+ * Set false by the HTTPServer I/O thread after flushing m_send_buffer.
+ * Checked in the HTTPServer I/O loop to decide whether to poll the socket for
+ * writeability or readability.
+ * Guarded by m_send_mutex so it stays consistent with m_send_buffer's emptiness:
+ * the two must always be updated together under the same lock.
+ */
+ bool m_send_ready GUARDED_BY(m_send_mutex){false};
/**
* Mutex that serializes the Send() and Recv() calls on `m_sock`. Reading
Why this scored 62/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.