charset-normalizer 3.5.1: charset_normalizer.from_fp

# charset-normalizer from_fp API

This sample pins `charset-normalizer` 3.5.1 and verifies `charset_normalizer.from_fp` for character set detection from binary file streams.

The contract verifies:
- `from_fp` reads from a binary stream (e.g. `io.BytesIO`) and returns a `CharsetMatches` object.
- `best()` returns the highest-scoring `CharsetMatch` containing the detected encoding and decoded string.
- Handling of distinct encodings including UTF-8, UTF-16 with BOM, and CP1252.
- Clean handling of empty byte streams without crashing (returns empty collection, `best()` is None).
- Rejection of incompatible text streams (raises TypeError/AttributeError/ValueError).

Clean-room public code sample — generation instructions

Write a brand-new, minimal, self-contained code sample in this clean-room directory.
Do not copy, paraphrase, or reference any existing project source. Work only from this spec.

A csx.json manifest scaffold already exists. Do not recreate it from memory. Preserve its case.goal, packages and symbols; fill its empty case.contract with exact assertions and correct its environment, commands and verifierAdapter for the files you generate.

Goal: verify charset_normalizer.from_fp in pkg:pypi/charset-normalizer@3.5.1
Kind: HOW

Use EXACTLY these public packages and versions:
  - pkg:pypi/charset-normalizer@3.5.1
Demonstrate these symbols/APIs:
  - charset_normalizer.from_fp

Rules:
  - One focused purpose; the smallest project that proves the goal.
  - Include a contract test (test/contract.*) that runs OFFLINE and exits 0 exactly when the goal behavior works.
  - Pin every dependency with a lockfile so resolution is reproducible.
  - No secrets, credentials, or tokens. No real URLs (only example.com or localhost). No absolute paths.
  - No personal names, emails, company names, or project identifiers of any kind.
  - No binaries and no generated output (node_modules, dist, target, venv, .git, .env).
  - Keep it under 200 files and 256KB packed.

{"case":{"caseId":"case:sha256:aa3d4f3bddf0135409ca48aa756bef373aab54e309ac8f5eb4da9a7b975a8b35","contract":["from_fp() accepts a binary stream (io.BytesIO) and returns a CharsetMatches collection containing CharsetMatch results.","best() on CharsetMatches returns the highest-scoring CharsetMatch whose .encoding, .raw, and string conversion yield the decoded text.","from_fp() correctly identifies distinct encodings such as UTF-8, UTF-16-LE with BOM, and UTF-8 with BOM from binary streams.","from_fp() accepts customization parameters including steps, chunk_size, and threshold for tuning charset detection.","from_fp() raises TypeError, AttributeError, or ValueError when passed an incompatible text stream instead of a binary file pointer."],"goal":"verify charset_normalizer.from_fp in pkg:pypi/charset-normalizer@3.5.1","kind":"HOW","packages":["pkg:pypi/charset-normalizer@3.5.1"],"schemaVersion":1,"symbols":["charset_normalizer.from_fp"]},"contractCommand":["python","test/contract.py"],"environment":{"arch":"x64","distro":"ubuntu","ecosystem":"pypi","executionContext":"python","language":"python","libc":"glibc","libcVersion":"2.39","os":"linux","osVersionBucket":"24","packageManager":"pip","runtime":"python","schemaVersion":1},"license":"MIT-0","packages":["pkg:pypi/charset-normalizer@3.5.1"],"schemaVersion":1,"subject":"pkg:pypi/charset-normalizer@3.5.1","symbols":["charset_normalizer.from_fp"],"verifierAdapter":"python@1"}

charset-normalizer==3.5.1

{
  "schemaVersion": 1,
  "goal": "verify charset_normalizer.from_fp in pkg:pypi/charset-normalizer@3.5.1",
  "kind": "HOW",
  "packages": [
    "pkg:pypi/charset-normalizer@3.5.1"
  ],
  "symbols": [
    "charset_normalizer.from_fp"
  ]
}

import io
import charset_normalizer
from charset_normalizer import from_fp, CharsetMatches, CharsetMatch

# 1. Verify from_fp accepts a binary stream (io.BytesIO) and returns CharsetMatches
utf8_sample = "The quick brown fox jumps over the lazy dog. 안녕하세요! 12345"
utf8_bytes = utf8_sample.encode("utf-8")
stream_utf8 = io.BytesIO(utf8_bytes)
matches_utf8 = from_fp(stream_utf8)

assert isinstance(matches_utf8, CharsetMatches), f"Expected CharsetMatches, got {type(matches_utf8)}"
assert len(matches_utf8) > 0, "Expected at least one match for UTF-8 stream"

# 2. Verify best() returns CharsetMatch whose .encoding, .raw, and str() yield decoded text
best_utf8 = matches_utf8.best()
assert isinstance(best_utf8, CharsetMatch), f"Expected CharsetMatch, got {type(best_utf8)}"
assert best_utf8.encoding == "utf_8", f"Expected utf_8, got {best_utf8.encoding}"
assert str(best_utf8) == utf8_sample
assert best_utf8.raw == utf8_bytes

# 3. Verify from_fp correctly identifies distinct encodings (e.g. UTF-16-LE with BOM and UTF-8 with BOM)
utf16le_sample = "Unicode UTF-16 content with characters: café, résumé, crème brûlée."
utf16le_bytes = b"\xff\xfe" + utf16le_sample.encode("utf-16-le")
stream_utf16 = io.BytesIO(utf16le_bytes)
matches_utf16 = from_fp(stream_utf16)
best_utf16 = matches_utf16.best()
assert best_utf16 is not None, "Expected match for UTF-16 stream"
assert "utf_16" in best_utf16.encoding.lower() or "utf-16" in best_utf16.encoding.lower()
assert str(best_utf16) == utf16le_sample
assert best_utf16.raw == utf16le_bytes

utf8_bom_sample = "UTF-8 with Byte Order Mark sample text."
utf8_bom_bytes = b"\xef\xbb\xbf" + utf8_bom_sample.encode("utf-8")
stream_bom = io.BytesIO(utf8_bom_bytes)
matches_bom = from_fp(stream_bom)
best_bom = matches_bom.best()
assert best_bom is not None, "Expected match for UTF-8 BOM stream"
assert best_bom.encoding in ("utf_8_sig", "utf_8")
assert str(best_bom) == utf8_bom_sample
assert best_bom.raw == utf8_bom_bytes

# 4. Verify from_fp accepts optional parameters (steps, chunk_size, threshold)
custom_stream = io.BytesIO(utf8_bytes)
custom_matches = from_fp(custom_stream, steps=3, chunk_size=256, threshold=0.1)
assert isinstance(custom_matches, CharsetMatches)
assert custom_matches.best() is not None
assert custom_matches.best().encoding == "utf_8"

# 5. Verify from_fp raises exception when passed incompatible text stream instead of binary stream
try:
    from_fp(io.StringIO("invalid text stream"))
    raise AssertionError("expected exception when passing text stream to from_fp")
except (TypeError, AttributeError, ValueError):
    pass

print("CONTRACT PASS: charset_normalizer.from_fp verified offline")