escape control characters in json string serialiser
What the developer wrote
Authored by alhudz
escape control characters in json string serialiser
What changed, and why it matters
This commit fixes the way Monero's internal JSON serializer handles special low-value characters (control characters). Previously, characters such as vertical tab and other invisible control characters were either escaped in non-standard ways or left unescaped, which could produce JSON that violates the official JSON standard (RFC 8259). Invalid JSON can confuse other programs that read Monero's output, and in the worst case could be abused to alter how data is parsed or to inject unexpected content. The patch now correctly escapes all control characters and adds tests to prove it.
Treat this as a correctness/security hardening fix and include it in the next release. Review any consumers of epee JSON output that may have been relying on the previous non-standard escaping, and ensure downstream parsers are RFC 8259 compliant. No immediate emergency response is indicated by the diff alone, but the fix should be deployed normally.
Security signals we found
JSON control-character escaping now conforms to RFC 8259
Previously invalid escape \\v removed
Previously unescaped C0 control characters (U+0000-U+001F outside \b\f\n\r\t) now escaped
Embedded NUL bytes handled explicitly
New unit tests assert round-trip behavior and absence of raw control bytes in JSON output
Evidence from the diff
The change is in contrib/epee/src/parserse_base_utils.cpp, in transform_to_escape_sequence(). Before the patch, the function searched only a fixed list of characters (\b\f\n\r\t\v"\/) and escaped them; characters below 0x20 outside that list were emitted verbatim, and \v was emitted as \v, which is not a valid JSON escape per RFC 8259. The patch replaces the search with a predicate that triggers on any byte < 0x20, plus “, \, and /. In the replacement loop, bytes < 0x20 without a short JSON escape are now emitted as \u00xx. Unit tests verify escaping of 0x01, 0x07, 0x0b, 0x1f, embedded NUL, round-trip serialization, and absence of raw control bytes in serialized output.
Changed components
contrib/epee/src/parserse_base_utils.cppepee JSON string serializertransform_to_escape_sequence()tests/unit_tests/epee_serialization.cppInspect captured patch +60 / −9
diff --git a/contrib/epee/src/parserse_base_utils.cpp b/contrib/epee/src/parserse_base_utils.cpp
index 5a000cc..7f7f2f2 100644
--- a/contrib/epee/src/parserse_base_utils.cpp
+++ b/contrib/epee/src/parserse_base_utils.cpp
@@ -41,8 +41,14 @@ namespace misc_utils
{
std::string transform_to_escape_sequence(const std::string& src)
{
- static const char escaped[] = "\b\f\n\r\t\v\"\\/";
- std::string::const_iterator it = std::find_first_of(src.begin(), src.end(), escaped, escaped + sizeof(escaped));
+ // RFC 8259: the double quote, reverse solidus and every control character
+ // (U+0000 - U+001F) must be escaped inside a JSON string. The forward slash
+ // is escaped too, as it has been historically.
+ std::string::const_iterator it = std::find_if(src.begin(), src.end(),
+ [](char ch) {
+ const unsigned char c = static_cast<unsigned char>(ch);
+ return c < 0x20 || c == '"' || c == '\\' || c == '/';
+ });
if (it == src.end())
return src;
@@ -51,7 +57,8 @@ namespace misc_utils
res.assign(src.begin(), it);
for(; it!=src.end(); ++it)
{
- switch(*it)
+ const unsigned char c = static_cast<unsigned char>(*it);
+ switch(c)
{
case '\b': //Backspace (ascii code 08)
res+="\\b"; break;
@@ -63,18 +70,22 @@ namespace misc_utils
res+="\\r"; break;
case '\t': //Tab
res+="\\t"; break;
- case '\v': //Vertical tab
- res+="\\v"; break;
- //case '\'': //Apostrophe or single quote
- // res+="\\'"; break;
case '"': //Double quote
res+="\\\""; break;
case '\\': //Backslash character
res+="\\\\"; break;
- case '/': //Backslash character
+ case '/': //Slash character
res+="\\/"; break;
default:
- res.push_back(*it);
+ if (c < 0x20)
+ {
+ //control character without a short escape (\v is not valid JSON)
+ static const char hex[] = "0123456789abcdef";
+ const char u[] = { '\\', 'u', '0', '0', hex[c >> 4], hex[c & 0xf] };
+ res.append(u, sizeof(u));
+ }
+ else
+ res.push_back(*it);
}
}
return res;
diff --git a/tests/unit_tests/epee_serialization.cpp b/tests/unit_tests/epee_serialization.cpp
index ec39996..0f9e82d 100644
--- a/tests/unit_tests/epee_serialization.cpp
+++ b/tests/unit_tests/epee_serialization.cpp
@@ -32,6 +32,7 @@
#include <vector>
#include "serialization/keyvalue_serialization.h"
+#include "storages/parserse_base_utils.h"
#include "storages/portable_storage.h"
#include "storages/portable_storage_template_helper.h"
#include "span.h"
@@ -107,6 +108,15 @@ struct ObjWithBool
KV_SERIALIZE(b)
END_KV_SERIALIZE_MAP()
};
+
+struct ObjWithString
+{
+ std::string s;
+
+ BEGIN_KV_SERIALIZE_MAP()
+ KV_SERIALIZE(s)
+ END_KV_SERIALIZE_MAP()
+};
}
TEST(epee_json, keyword_values)
@@ -129,6 +139,36 @@ TEST(epee_json, keyword_values)
EXPECT_FALSE(epee::serialization::load_t_from_json(o, std::string("{\"b\": \xc3\x28}")));
}
+TEST(epee_json, escape_control_characters)
+{
+ using epee::misc_utils::parse::transform_to_escape_sequence;
+
+ // control characters must be escaped (RFC 8259): the short escapes are used
+ // where JSON defines them and \u00xx otherwise. \v is not a valid JSON escape.
+ EXPECT_EQ(transform_to_escape_sequence(std::string("\x01")), "\\u0001");
+ EXPECT_EQ(transform_to_escape_sequence(std::string("\x07")), "\\u0007");
+ EXPECT_EQ(transform_to_escape_sequence(std::string("\x0b")), "\\u000b");
+ EXPECT_EQ(transform_to_escape_sequence(std::string("\x1f")), "\\u001f");
+ EXPECT_EQ(transform_to_escape_sequence(std::string("a\x00" "b", 3)), "a\\u0000b");
+
+ // short escapes and printable text are unchanged
+ EXPECT_EQ(transform_to_escape_sequence(std::string("a\tb\n\r\f\b")), "a\\tb\\n\\r\\f\\b");
+ EXPECT_EQ(transform_to_escape_sequence(std::string("plain text")), "plain text");
+
+ // control characters round trip through the json serialiser and parser, and
+ // never appear verbatim in the serialised output
+ ObjWithString o{};
+ o.s = std::string("x\x01\x0b" "y", 4);
+ std::string j;
+ EXPECT_TRUE(epee::serialization::store_t_to_json(o, j));
+ EXPECT_EQ(j.find('\x01'), std::string::npos);
+ EXPECT_EQ(j.find('\x0b'), std::string::npos);
+
+ ObjWithString o2{};
+ EXPECT_TRUE(epee::serialization::load_t_from_json(o2, j));
+ EXPECT_EQ(o2.s, o.s);
+}
+
TEST(epee_binary, serialize_deserialize)
{
ParentObjWithOptChild<ObjWithOptChild> o;
Why this scored 65/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.