Beispiel

PyArrow CSV ingestion retains null tokens in string columns as literal strings unless strings_can_be_null is enabled in ConvertOptions, and raises KeyError for unlisted columns unless include_missing_columns is set

sha256:96f7e54f838dd5fd2cca97dba04561e3bf2ad30732280f3e451fb70e770b6729

Fall

Ziel

PyArrow CSV ingestion retains null tokens in string columns as literal strings unless strings_can_be_null is enabled in ConvertOptions, and raises KeyError for unlisted columns unless include_missing_columns is set HOW

Pakete

pyarrow 25.0.1

Umgebung

python

Erstellt

2026-08-17T10:42:07Z

Häufige Annahme

pyarrow.csv.read_csv parses tokens defined in null_values as null values across all columns regardless of inferred column data type.

So hat der Autor des Samples festgehalten, was eine Entwicklerin oder ein Modell hier erwarten würde. Der Vertrag darunter ist das, was tatsächlich lief.

Contract

• CSV columns inferred as strings preserve tokens matching null_values as literal strings unless strings_can_be_null is explicitly set to True, whereas numeric columns convert them to nulls by default.
• assert tbl_default['num'].null_count == 2 and tbl_default['txt'].null_count == 0
• assert tbl_nulls['txt'].to_pylist() == ['hello', None, 'world', None]
• assert tbl_no_quoted['txt'].to_pylist() == ['NA', 'hello', None]
• assert reading unlisted include_columns without include_missing_columns raises KeyError
• assert tbl_missing['extra_col'].type == pa.null() and tbl_typed_missing['extra_col'].type == pa.int32()
• assert isinstance(tbl_dict['category'].type, pa.DictionaryType)
• assert csv.read_csv raises ArrowInvalid on malformed rows unless ParseOptions(invalid_row_handler=...) returns 'skip'
• assert tbl['flag'].type == pa.bool_() and tbl['status'].to_pylist() == [None, 'PENDING', None]

Dateien

• NOTES.md
• csx.json
• requirements.lock
• requirements.txt
• test/contract.py

Quellartefakt herunterladen (tar.gz)

Ursprungs-Seeder

csx-seed

Verifizierungsbelege

• python 3.12 · linux alpine/x64 · docker · CONTAINER_RUN · compile:SKIPPED · contract:PASS · load:PASS · resolve:PASS · python@1 · 2026-08-17 · ed25519:d91480838ac982c9