サンプル

PyArrow CSV ingestion retains null tokens in string columns as literal strings unless strings_can_be_null is enabled in ConvertOptions, and raises KeyError for unlisted columns unless include_missing_columns is set

sha256:96f7e54f838dd5fd2cca97dba04561e3bf2ad30732280f3e451fb70e770b6729

ケース

ゴール

PyArrow CSV ingestion retains null tokens in string columns as literal strings unless strings_can_be_null is enabled in ConvertOptions, and raises KeyError for unlisted columns unless include_missing_columns is set HOW

パッケージ

pyarrow 25.0.1

環境

python

作成日

2026-08-17T10:42:07Z

よくある思い込み

pyarrow.csv.read_csv parses tokens defined in null_values as null values across all columns regardless of inferred column data type.

このサンプルの作者が、ここで開発者やモデルが期待するであろうこととして記録したもの。下の契約が実際に実行されたものだ。

コントラクト

• CSV columns inferred as strings preserve tokens matching null_values as literal strings unless strings_can_be_null is explicitly set to True, whereas numeric columns convert them to nulls by default.
• assert tbl_default['num'].null_count == 2 and tbl_default['txt'].null_count == 0
• assert tbl_nulls['txt'].to_pylist() == ['hello', None, 'world', None]
• assert tbl_no_quoted['txt'].to_pylist() == ['NA', 'hello', None]
• assert reading unlisted include_columns without include_missing_columns raises KeyError
• assert tbl_missing['extra_col'].type == pa.null() and tbl_typed_missing['extra_col'].type == pa.int32()
• assert isinstance(tbl_dict['category'].type, pa.DictionaryType)
• assert csv.read_csv raises ArrowInvalid on malformed rows unless ParseOptions(invalid_row_handler=...) returns 'skip'
• assert tbl['flag'].type == pa.bool_() and tbl['status'].to_pylist() == [None, 'PENDING', None]

ファイル

• NOTES.md
• csx.json
• requirements.lock
• requirements.txt
• test/contract.py

ソースアーティファクトをダウンロード (tar.gz)

オリジンシーダー

csx-seed

検証レシート

• python 3.12 · linux alpine/x64 · docker · CONTAINER_RUN · compile:SKIPPED · contract:PASS · load:PASS · resolve:PASS · python@1 · 2026-08-17 · ed25519:d91480838ac982c9