markupsafe 3.0.3: Pin MarkupSafe byte escaping semantics, format_map dynamic mapping auto-escaping, subclass type preservation across string operations, and __radd__ coercion.

# MarkupSafe Byte Escaping, Mapping Formatting, and Subclass Semantics

## Search Result
- `search_known_solution` returned existing samples:
  - `sha256:ee43604c2b3d1d7c19c6e5ac416478456ecd9af20d85dc3e8accdb54f51b8ba8` (covered `__html__` protocol discrepancies between `format()` and `escape()`, format specifier rejections, `%` helper type errors on `%x`, and mutation auto-escaping)
  - `sha256:652cdb6d7e7a11385d82d916dea3618038ba21c708d45ec715d9cc5f8cbd041a` (covered basic entity substitution, `escape()` idempotency, and basic concatenation)
- This sample investigates distinct behaviors in MarkupSafe: `bytes` handling in `escape()` versus `Markup(bytes, encoding)`, dynamic mapping handling in `format_map()` versus `format(**kwargs)` unpacking, `Markup` subclass preservation across string operations, left-hand `__radd__` coercion versus `str.join()`, and literal matching in `Markup.replace()`.

## Release Pinned
- `markupsafe==3.0.3` under Python 3.10+.

## Observed Behaviour

1. **Byte String Escaping Trap**:
   - `escape(b"<div>")` does not decode bytes as UTF-8. Instead, because `bytes` is neither `str` nor implements `__html__`, `escape()` passes `str(bytes_obj)` to the inner escaper.
   - In Python 3, `str(b"<div>")` produces the representation string `"b'<div>'"`. `escape()` then escapes the quotes and angle brackets, producing `Markup("b&#39;&lt;div&gt;&#39;")`.
   - To decode bytes into safe markup, one must pass `encoding="utf-8"` directly to `Markup(bytes_obj, encoding="utf-8")`.

2. **`format_map()` Dynamic Mapping Support**:
   - Unpacking dynamic mappings (like `defaultdict`) via `Markup.format(**mapping)` only captures existing keys into a static dictionary argument, causing template references to missing keys to raise `KeyError`.
   - `Markup.format_map(mapping)` evaluates the mapping dynamically during template interpolation, allowing `defaultdict` to compute fallback values on demand, while `EscapeFormatter` automatically escapes string values and preserves `Markup` objects.

3. **Subclass Type Preservation**:
   - Custom subclasses inheriting from `Markup` preserve their subtype across `Subclass.escape()`, binary concatenation (`+` and `__radd__`), partitioning (`.partition()`), splitting (`.split()`), string replacement (`.replace()`), and casing transformations (`.upper()`, `.lower()`, `.removeprefix()`, `.removesuffix()`).

4. **Concatenation (`__radd__`) vs `str.join()`**:
   - In binary addition, when a plain string appears on the left (`"<h1>" + Markup("Title")`), Python invokes `Markup.__radd__`, which escapes the plain string operand and returns a `Markup` instance (`Markup("&lt;h1&gt;Title")`).
   - Conversely, standard `str.join()` (e.g., `", ".join([Markup("<b>safe</b>"), "<raw>"])`) returns a plain `str` without escaping sibling elements, bypassing HTML safety. Safe joining requires using a `Markup` separator (`Markup(", ").join(...)`).

5. **`Markup.replace()` Search Matching**:
   - `Markup.replace(old, new)` matches `old` as a literal substring within the escaped internal representation rather than against decoded HTML characters.
   - The `new` parameter is automatically escaped when provided as a plain `str`, but preserved when passed as a `Markup` instance.

## How the Wrong Expectation Fails

- **Bytes Escaping**: Fails silently with corrupted HTML output. Passing UTF-8 `bytes` to `escape(b"<div>Hello</div>")` produces `Markup("b&#39;&lt;div&gt;Hello&lt;/div&gt;&#39;")` containing the literal Python byte string wrapper prefix and quotes, corrupting the rendered web page.
- **Dynamic Mappings**: Fails loudly with `KeyError` when using `format(**defaultdict)` because keyword argument unpacking cannot dynamically resolve absent keys at format time, whereas `format_map` resolves keys via dynamic mapping lookup.
- **String Joining**: Fails silently by returning an unescaped plain `str` when `str.join()` is called on a list containing `Markup` and raw strings, stripping markup protection from the output.

{"case":{"believed":"escape() decodes UTF-8 byte sequences before HTML-escaping them, and str.join() escapes raw strings when concatenating with Markup elements.","caseId":"case:sha256:860a8cc088179095e8dd72a2df9dc60de481fddd786d80ec43668b4cfacb988c","contract":["escape() converts bytes objects via str() to their literal repr 'b\\'...\\'' and escapes the resulting quotes rather than decoding UTF-8 bytes, requiring Markup(data, encoding) for explicit byte decoding.","Markup.format_map() queries Mapping instances dynamically without requiring keyword unpacking, auto-escaping plain string values while preserving Markup instances.","Markup subclasses preserve their custom subtype across escape classmethods, binary additions, partitions, splits, and case transformations.","Markup.__radd__ auto-escapes left-hand plain string operands during concatenation, whereas str.join returns an unescaped plain str.","Markup.replace matches search targets literally against the internal escaped representation and auto-escapes plain string replacement arguments."],"goal":"Pin MarkupSafe byte escaping semantics, format_map dynamic mapping auto-escaping, subclass type preservation across string operations, and __radd__ coercion.","kind":"HOW","packages":["pkg:pypi/markupsafe@3.0.3"],"schemaVersion":1,"symbols":["Markup","escape","EscapeFormatter"]},"contractCommand":["python","test/contract.py"],"environment":{"arch":"x64","ecosystem":"pypi","executionContext":"python","language":"python","os":"linux","packageManager":"pip","runtime":"python","schemaVersion":1},"license":"MIT-0","packages":["pkg:pypi/markupsafe@3.0.3"],"schemaVersion":1,"symbols":["Markup","escape","EscapeFormatter"],"verifierAdapter":"python@1"}

markupsafe==3.0.3

"""MarkupSafe template engine and helper package."""

"""Template engine utilities illustrating MarkupSafe formatting, bytes, and subclass semantics."""

from __future__ import annotations

import collections.abc as cabc
from typing import Any
from markupsafe import Markup, escape


class SafeHTML(Markup):
    """Custom markup subclass to demonstrate subtype preservation in template pipelines."""


def decode_and_mark_safe(data: bytes, encoding: str = "utf-8") -> Markup:
    """Decode raw bytes into a safe Markup instance without escaping."""
    return Markup(data, encoding=encoding)


def render_mapping(template: Markup, context: cabc.Mapping[str, Any]) -> Markup:
    """Render a Markup template with an arbitrary Mapping using format_map."""
    return template.format_map(context)


def safe_join(separator: Markup, items: cabc.Iterable[Any]) -> Markup:
    """Join an iterable of items with a Markup separator, escaping non-Markup items."""
    return separator.join(items)


def safe_replace(base: Markup, old: str, replacement: Any) -> Markup:
    """Replace occurrences of old with replacement in a Markup instance."""
    return base.replace(old, replacement)

"""Contract test proving MarkupSafe byte escaping, format_map, subclassing, and concatenation behaviors."""

from __future__ import annotations

from collections import defaultdict
from typing import Any
import sys
from pathlib import Path

# Add project root to sys.path
sys.path.insert(0, str(Path(__file__).parent.parent))

from markupsafe import Markup, escape, escape_silent, soft_str
from src.template_engine import (
    SafeHTML,
    decode_and_mark_safe,
    render_mapping,
    safe_join,
    safe_replace,
)


def test_bytes_escaping_vs_markup_constructor() -> None:
    """Prove escape() treats bytes as an object and escapes its str() repr rather than decoding UTF-8."""
    raw_bytes = b"<div>Hello & Goodbye</div>"

    # Naive expectation: escape(raw_bytes) decodes UTF-8 and returns Markup("&lt;div&gt;Hello &amp; Goodbye&lt;/div&gt;")
    # Actual behavior: escape() calls str(raw_bytes), which becomes "b'<div>Hello & Goodbye</div>'", escaping quotes and brackets
    escaped_bytes = escape(raw_bytes)
    assert isinstance(escaped_bytes, Markup)
    assert escaped_bytes == Markup("b&#39;&lt;div&gt;Hello &amp; Goodbye&lt;/div&gt;&#39;")

    # To decode bytes into safe Markup, explicit encoding argument must be passed to Markup()
    safe_decoded = decode_and_mark_safe(raw_bytes, encoding="utf-8")
    assert isinstance(safe_decoded, Markup)
    assert safe_decoded == Markup("<div>Hello & Goodbye</div>")


def test_format_map_mapping_support() -> None:
    """Prove Markup.format_map queries mappings dynamically without unpacking errors, auto-escaping values."""
    # Using format(**defaultdict) unpacks only existing keys into a static dict, raising KeyError for missing keys
    dynamic_context = defaultdict(lambda: "<unassigned>", {"user-id": "100"})
    template = Markup("<p>User: {user-id}, Role: {role}</p>")

    try:
        template.format(**dynamic_context)
        raise AssertionError("Expected KeyError when using format(**defaultdict) on missing template key")
    except KeyError:
        pass

    # format_map queries the mapping dynamically during formatting, auto-escaping default and provided strings
    rendered_default = render_mapping(template, dynamic_context)
    assert isinstance(rendered_default, Markup)
    assert rendered_default == Markup("<p>User: 100, Role: &lt;unassigned&gt;</p>")

    # format_map preserves Markup instances within mapping values
    rich_context = {
        "user-id": "<123>",
        "badge": Markup('<span class="admin">Admin</span>'),
    }
    rendered_rich = render_mapping(Markup("User: {user-id} ({badge})"), rich_context)
    assert rendered_rich == Markup('User: &lt;123&gt; (<span class="admin">Admin</span>)')


def test_subclass_preservation() -> None:
    """Prove Markup subclasses preserve their custom subtype across escape classmethods, operations, and splits."""
    # SafeHTML.escape returns SafeHTML instance
    escaped = SafeHTML.escape("<div>alert</div>")
    assert isinstance(escaped, SafeHTML)
    assert escaped == SafeHTML("&lt;div&gt;alert&lt;/div&gt;")

    base = SafeHTML("<b>header</b>")

    # Binary addition preserves subclass
    add_right = base + " <tag>"
    assert isinstance(add_right, SafeHTML)
    assert add_right == SafeHTML("<b>header</b> &lt;tag&gt;")

    add_left = "<tag> " + base
    assert isinstance(add_left, SafeHTML)
    assert add_left == SafeHTML("&lt;tag&gt; <b>header</b>")

    # Partition returns 3-tuple of SafeHTML
    parts = base.partition("header")
    assert isinstance(parts, tuple)
    assert len(parts) == 3
    assert all(isinstance(p, SafeHTML) for p in parts)
    assert parts == (SafeHTML("<b>"), SafeHTML("header"), SafeHTML("</b>"))

    # Split returns list of SafeHTML
    splits = base.split("header")
    assert isinstance(splits, list)
    assert all(isinstance(s, SafeHTML) for s in splits)
    assert splits == [SafeHTML("<b>"), SafeHTML("</b>")]

    # Replace returns SafeHTML and auto-escapes replacement
    replaced = base.replace("header", "<span>new</span>")
    assert isinstance(replaced, SafeHTML)
    assert replaced == SafeHTML("<b>&lt;span&gt;new&lt;/span&gt;</b>")

    # Case transformations and prefix/suffix methods preserve SafeHTML
    assert isinstance(base.upper(), SafeHTML)
    assert isinstance(base.lower(), SafeHTML)
    assert isinstance(base.removeprefix("<b>"), SafeHTML)
    assert isinstance(base.removesuffix("</b>"), SafeHTML)


def test_radd_and_join_semantics() -> None:
    """Prove Markup.__radd__ auto-escapes left plain strings while str.join returns plain unescaped str."""
    raw_left = "<h1>"
    markup_right = Markup("Title") + "</h1>"

    # Left addition calls Markup.__radd__, auto-escaping raw_left
    result = raw_left + markup_right
    assert isinstance(result, Markup)
    assert result == Markup("&lt;h1&gt;Title&lt;/h1&gt;")

    items = ["<tag1>", Markup("<b>safe</b>"), "<tag2>"]

    # Markup separator join() auto-escapes raw strings and returns Markup
    markup_joined = safe_join(Markup(", "), items)
    assert isinstance(markup_joined, Markup)
    assert markup_joined == Markup("&lt;tag1&gt;, <b>safe</b>, &lt;tag2&gt;")

    # str separator join() returns plain str and does NOT escape items
    plain_joined = ", ".join(items)
    assert type(plain_joined) is str
    assert not isinstance(plain_joined, Markup)
    assert plain_joined == "<tag1>, <b>safe</b>, <tag2>"


def test_replace_matching_and_auto_escape() -> None:
    """Prove Markup.replace matches against the escaped representation and auto-escapes plain str replacements."""
    template = Markup("<div>&lt;placeholder&gt;</div>")

    # Matching unescaped string '<placeholder>' does not match because internal representation is escaped
    assert template.replace("<placeholder>", "matched") == Markup("<div>&lt;placeholder&gt;</div>")

    # Matching the exact escaped text '&lt;placeholder&gt;' matches
    # Plain string replacement is auto-escaped
    replaced_raw = safe_replace(template, "&lt;placeholder&gt;", "<b>content</b>")
    assert replaced_raw == Markup("<div>&lt;b&gt;content&lt;/b&gt;</div>")

    # Markup replacement is preserved unescaped
    replaced_markup = safe_replace(template, "&lt;placeholder&gt;", Markup("<b>content</b>"))
    assert replaced_markup == Markup("<div><b>content</b></div>")


def main() -> None:
    """Execute all contract tests."""
    test_bytes_escaping_vs_markup_constructor()
    test_format_map_mapping_support()
    test_subclass_preservation()
    test_radd_and_join_semantics()
    test_replace_matching_and_auto_escape()
    print("All MarkupSafe contract assertions passed successfully.")


if __name__ == "__main__":
    main()