chore(tools): pre-process crowdin downloaded files
What changed, and why it matters
This commit adds a cleanup step to translation files downloaded from Crowdin. It replaces non-breaking spaces with regular spaces, except before French punctuation marks like ? and !. There is no security issue here; it is a routine data-quality improvement for localization files.
No security action needed. This is a normal localization tooling change.
Security signals we found
No strong security signals were identified.
Evidence from the diff
The change introduces a clean_translation() helper in core/translations/crowdin.py that uses a regex with negative lookahead to replace Unicode non-breaking space (U+00A0) characters with normal spaces, preserving them when they precede ‘?’ or ‘!’. This is applied to all translation values during the merge process. The modification is purely a normalization step for i18n string data and does not affect device firmware runtime behavior, cryptographic operations, or user-facing security properties.
Changed components
core/translations/crowdin.pyInspect captured patch +11 / −1
diff --git a/core/translations/crowdin.py b/core/translations/crowdin.py
index dc65c8062..07307e08e 100644
--- a/core/translations/crowdin.py
+++ b/core/translations/crowdin.py
@@ -3,6 +3,7 @@ from __future__ import annotations
from pathlib import Path
import collections
import json
+import re
import click
@@ -49,6 +50,13 @@ def merge() -> None:
"""Merge back translation files downloaded from Crowdin."""
tdir = TranslationsDir()
+ def clean_translation(text: str) -> str:
+ """Remove or replace non-printable characters in translation strings."""
+ # Replace non-breaking spaces with regular spaces, EXCEPT before ? and !
+ # Use negative lookahead to avoid replacing before French punctuation
+ text = re.sub(r'\u00A0(?![?!])', ' ', text)
+ return text
+
for lang in sorted(tdir.all_languages()):
merged_translations: dict[str, str | dict[str, str]] = collections.defaultdict(dict)
for layout_type in translations.ALL_LAYOUTS:
@@ -58,7 +66,9 @@ def merge() -> None:
# mapping string name to its translation (for the current layout)
layout_specific_translations: dict[str, str] = blob_json["translations"]
for key, value in layout_specific_translations.items():
- merged_translations[key][layout_type.name] = value
+ # Clean the translation value
+ cleaned_value = clean_translation(value)
+ merged_translations[key][layout_type.name] = cleaned_value
for key in merged_translations.keys():
# deduplicate entries if all translations are the same
Why this scored 15/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.