json 2.9.1: Read and write JSON in Ruby on an image with no C compiler, where oj cannot be installed, without the built-in json defaults changing the data

source "https://rubygems.org"

# oj is deliberately absent, and why is the first half of the answer.
#
# Measured on ruby:3-alpine (ruby 3.4.10, x86_64-linux-musl), not assumed:
# `gem install oj -v 3.17.6 --no-document` dies in extconf with
#
#   The compiler failed to generate an executable file. (RuntimeError)
#   You have to install development tools first.
#
# because the image carries no cc, gcc, clang or make. oj is a C extension and
# there is nothing to build it with. The same holds for the json gem itself:
# 2.21.2 is the current release, and `gem install json -v 2.21.2` was measured
# failing at the same extconf step, so on this image you cannot upgrade past
# the json Ruby already has either. Adding build-base to the image is the other
# way out, if the image is yours to change.
#
# That leaves the json Ruby ships — 2.9.1 here, already compiled, because it
# was built when Ruby was. It is the C extension, not the pure-Ruby fallback;
# the contract checks that. So the question worth answering is not "oj or json"
# but which of json's defaults quietly change your data.

# minitest 5.27.0 rather than 6.0.6, for the same reason one step removed, and
# the intermediate measurement is the interesting one. `gem install minitest
# -v 6.0.6` succeeds here, which makes 6 look fine: minitest 6 needs
# prism (~> 1.5), and Ruby 3.4.10 has prism 1.5.3 built in, so nothing is
# fetched. Under bundler it fails — bundler resolves prism to the newest
# release, 1.9.0, and building that needs the compiler the image does not have:
#
#   An error occurred while installing prism (1.9.0), and Bundler cannot continue.
#
# Adding `gem "prism", "1.5.3"` does get minitest 6.0.6 installed, because
# bundler will then use the built-in copy ("prism-1.5.3 is built in to Ruby").
# That is the technique when you need a gem whose dependency has a native
# extension: pin the dependency to the version the interpreter already ships.
# It is not used here because it pins a test dependency to one image's build of
# Ruby. minitest 5.27.0 has no dependencies at all.
gem "minitest", "5.27.0"

GEM
  remote: https://rubygems.org/
  specs:
    minitest (5.27.0)

PLATFORMS
  ruby
  x86_64-linux-musl

DEPENDENCIES
  minitest (= 5.27.0)

CHECKSUMS
  bundler (4.0.18) sha256=02d9a17429de1847b4e0c9f27a9ee4b20c0a74c0a641b4e77195d6019e3618ac
  minitest (5.27.0) sha256=2d3b17f8a36fe7801c1adcffdbc38233b938eb0b4966e97a6739055a45fa77d5

BUNDLED WITH
  4.0.18

{"case":{"caseId":"case:sha256:ac3d672cf8b605d4e2324dc8fa9f6d4d7f82d5af626af1b3f21ba378058416cf","constraints":{"runtime":"ruby"},"contract":["assert JSON.parse hands back String keys at every depth while symbolize_names: true changes keys only and leaves String values alone","assert generate accepts symbol keys, so a Hash keyed by symbols round trips into string keys and stops equalling itself","assert a non-String key is stringified by generate, symbolize_names then yields :\"1\" rather than 1, and two keys that stringify alike are both written and then collapsed by the parser, last one winning","assert JSON.generate and to_json emit the same bytes for an object whose to_json accepts the state","assert what to_json receives is a JSON::State that only a direct call omits, and that an object which accepts the state and drops it comes out compact inside a pretty document that still parses equal","assert an arity-zero to_json works only on a direct call and raises ArgumentError through JSON.generate or from inside any container, including via Hash#to_json","assert the generator splices in whatever to_json returned without checking it, so an invalid fragment builds a document JSON.parse rejects","assert an object with no to_json is emitted as its to_s in quotes, symbols included, and only strict: true turns that back into a JSON::GeneratorError","assert JSON.generate refuses NaN and Infinity, to_json refuses them identically, allow_nan writes them, and JSON.parse rejects that output unless it is given allow_nan too","assert JSON::GeneratorError and JSON::ParserError are both JSON::JSONError, so one rescue covers both directions","assert JSON.dump defaults to allow_nan and writes a document JSON.parse cannot read while JSON.load can, and that JSON.load defaults to allow_blank so an empty string is nil where JSON.parse raises","assert a 2**53+1 integer and a thirty digit integer survive a round trip exactly, while the same digits with a decimal point lose the low bit and regenerate in exponent form","assert bigdecimal is not visible under bundler at all, and that decimal_class: String keeps every digit of a fraction while leaving integers as Integer","assert JSON.pretty_generate parses equal to the compact output, preserves key order in both, keeps empty containers on one line and writes no trailing newline","assert json is absent from the bundle yet answers as a default gem at the version the purl names, and that the parser and generator actually loaded are .so files under the interpreter's own archdir","assert no compiler and no make are on PATH and that requiring oj is a LoadError"],"goal":"Read and write JSON in Ruby on an image with no C compiler, where oj cannot be installed, without the built-in json defaults changing the data","kind":"HOW","packages":["pkg:gem/json@2.9.1","pkg:gem/minitest@5.27.0"],"schemaVersion":1,"symbols":["JSON.parse","JSON.generate","JSON.pretty_generate","JSON.dump","JSON.load","JSON::State","JSON::GeneratorError","JSON::ParserError","JSON::Ext::Generator","Object#to_json"]},"contractCommand":["bundle","exec","ruby","test/contract.rb"],"environment":{"arch":"x64","ecosystem":"gem","executionContext":"ruby","language":"ruby","os":"linux","packageManager":"bundler","runtime":"ruby","runtimeVersion":"3","schemaVersion":1},"license":"MIT-0","packages":["pkg:gem/json@2.9.1","pkg:gem/minitest@5.27.0"],"schemaVersion":1,"symbols":["JSON.parse","JSON.generate","JSON.pretty_generate","JSON.dump","JSON.load","JSON::State","JSON::GeneratorError","JSON::ParserError","JSON::Ext::Generator","Object#to_json"],"verifierAdapter":"gem@1"}

# What Ruby's built-in JSON really does, on a machine where you cannot install
# a faster one.
#
# The standard advice for Ruby JSON is "use oj". On an image with no C
# toolchain that advice is unavailable: oj is a C extension and it cannot be
# built, which the Gemfile records as a measurement rather than a guess. You
# are left with the json Ruby ships, so the useful question becomes which of
# its defaults change your data on the way through.
#
# Five that catch people, in the order they bite:
#
#   1. JSON.parse hands back String keys at every depth. There is no default
#      that changes it. symbolize_names: true is the whole answer, and it
#      touches keys only.
#   2. A custom to_json is honoured by JSON.generate as well as by obj.to_json,
#      so writing one puts you inside the generator — where you get the state
#      as an argument, and where nothing checks that what you returned was JSON
#      at all.
#   3. NaN and Infinity are not JSON. JSON.generate refuses to write them and
#      JSON.parse refuses to read them, both unless allow_nan, so the default
#      round trip of a float that went non-finite raises rather than corrupts.
#      JSON.dump and JSON.load do not share those defaults, which is how a file
#      one wrote becomes a file the other's counterpart cannot read.
#   4. A JSON integer of any size survives exactly, because Ruby has bignums.
#      The same digits with a decimal point become a Float and lose their tail
#      silently. decimal_class is the option that gets them back.
#   5. JSON.pretty_generate is bytes only. The parse of its output is == to the
#      parse of the compact output, every time.
#
# Everything below is what you would actually copy into an app.

require "json"

module JsonOps
  # String keys are the default at every depth; ask for symbols explicitly.
  # Keys only: a String value stays a String.
  def self.parse_symbolized(text)
    JSON.parse(text, symbolize_names: true)
  end

  # Keep the digits a Float would drop. decimal_class takes any class built
  # from the raw token, and String is the one that always works: the documented
  # choice, BigDecimal, is a bundled gem in Ruby 3.4 rather than a default one,
  # so on this image it is both missing from the bundle and unbuildable.
  # Integers are unaffected — decimal_class only sees tokens with a fraction or
  # an exponent.
  def self.parse_exact_decimals(text)
    JSON.parse(text, decimal_class: String)
  end

  # The strict path: what a server should do with a payload it did not write.
  # Non-finite floats raise on the way out and are rejected on the way back in.
  def self.round_trip(value)
    JSON.parse(JSON.generate(value))
  end

  # A value object that serializes correctly from either entry point. The
  # *args is not decoration: JSON.generate passes a JSON::State, and passing it
  # on is what makes indent and space settings reach the nested object.
  class Money
    attr_reader :cents, :currency

    def initialize(cents, currency)
      @cents = cents
      @currency = currency
    end

    def as_json
      { "cents" => cents, "currency" => currency }
    end

    def to_json(*args)
      as_json.to_json(*args)
    end
  end

  # The same class as people write it the first time. Calling it yourself
  # works, which is exactly why the bug ships: it only fails once the object is
  # inside something the generator is walking.
  class NaiveMoney < Money
    def to_json
      as_json.to_json
    end
  end

  # Accepts the state and drops it, which is the version that survives review:
  # valid JSON from both entry points, no exception ever. The cost shows up
  # only inside pretty_generate, where this object is the one line that came
  # out compact, because indent and space live in the state it threw away. No
  # parser can tell the two documents apart, so nothing downstream complains.
  class CompactMoney < Money
    def to_json(*)
      as_json.to_json
    end
  end

  # And the failure that is worse than an exception: the generator splices in
  # whatever to_json returned without looking at it, so an invalid fragment
  # becomes an invalid document that only fails at the reader.
  class LyingMoney < Money
    def to_json(*)
      "$#{format('%.2f', cents / 100.0)}"
    end
  end

  # Which pieces of a C toolchain are on PATH. Empty here, and that is the
  # whole reason oj is not an option: a gem with a native extension has nothing
  # to build with.
  def self.compilers_on_path
    ENV.fetch("PATH", "").split(File::PATH_SEPARATOR).flat_map do |dir|
      Dir.glob(File.join(dir, "{cc,gcc,clang,make}"))
    end
  end
end

require "minitest/autorun"
require_relative "../src/json_ops"

# Keeps whatever the generator handed to to_json, so the argument can be
# identified rather than inferred from the arity error.
class StateProbe
  attr_reader :received

  def initialize
    @received = []
  end

  def to_json(*args)
    @received.concat(args)
    "0"
  end
end

class JsonDefaultsContract < Minitest::Test
  MONEY = JsonOps::Money.new(1234, "USD").freeze
  MONEY_JSON = '{"cents":1234,"currency":"USD"}'.freeze

  def test_parse_returns_string_keys_and_symbolize_names_is_the_option_people_forget
    doc = '{"total":2,"items":[{"sku":"a-1","qty":1}]}'

    assert_equal %w[total items], JSON.parse(doc).keys
    assert_equal %w[sku qty], JSON.parse(doc)["items"][0].keys

    symbolized = JsonOps.parse_symbolized(doc)
    assert_equal %i[total items], symbolized.keys
    assert_equal %i[sku qty], symbolized[:items][0].keys

    # Keys only. The value stays the String it was, which is what makes
    # symbolize_names safe to use on a payload you did not write.
    assert_equal "a-1", symbolized[:items][0][:sku]
    assert_instance_of String, symbolized[:items][0][:sku]
  end

  def test_symbol_keys_do_not_survive_a_round_trip_on_their_own
    # The direction people forget: generate is happy to take symbol keys, so
    # the write side looks fine and the read side quietly hands back strings.
    assert_equal '{"cents":1234}', JSON.generate({ cents: 1234 })
    assert_equal({ "cents" => 1234 }, JsonOps.round_trip({ cents: 1234 }))
    refute_equal({ cents: 1234 }, JsonOps.round_trip({ cents: 1234 }))
  end

  def test_a_key_that_was_not_a_string_becomes_one_and_symbolize_names_cannot_undo_it
    # JSON object keys are strings by definition, so generate calls to_s on
    # whatever you used. A Hash keyed by Integer is not a round trip, and
    # asking for symbols afterwards gets you the symbol :"1", not 1.
    assert_equal '{"1":"a","b":"c"}', JSON.generate({ 1 => "a", :b => "c" })
    assert_equal({ "1" => "a" }, JsonOps.round_trip({ 1 => "a" }))
    assert_equal({ :"1" => "a" }, JsonOps.parse_symbolized(JSON.generate({ 1 => "a" })))

    # And the collision that follows is silent, in two steps: the generator
    # writes both keys, because a Hash that has two distinct keys has two
    # entries, and the parser then keeps the last one.
    assert_equal '{"1":"a","1":"b"}', JSON.generate({ 1 => "a", "1" => "b" })
    assert_equal({ "1" => "b" }, JsonOps.round_trip({ 1 => "a", "1" => "b" }))
    assert_equal({ "a" => 2 }, JSON.parse('{"a":1,"a":2}'))
  end

  def test_generate_and_to_json_agree_when_to_json_accepts_the_state
    assert_equal MONEY_JSON, MONEY.to_json
    assert_equal MONEY_JSON, JSON.generate(MONEY)
    assert_equal %({"price":#{MONEY_JSON}}), JSON.generate({ "price" => MONEY })

    # What to_json receives is a JSON::State, and only a direct call passes
    # nothing at all — which is why an arity-zero to_json passes its own unit
    # test and fails everywhere else.
    probe = StateProbe.new
    probe.to_json
    assert_empty probe.received
    JSON.generate({ "n" => probe })
    assert_kind_of JSON::State, probe.received.fetch(0)

    # Passing *args through is what carries that state down. The object that
    # accepts the state and drops it is the trap here: it is the one line of a
    # pretty document that came out compact, and since both documents parse to
    # the same data nothing downstream will ever report it.
    passed_on = JSON.pretty_generate({ "price" => MONEY })
    dropped = JSON.pretty_generate({ "price" => JsonOps::CompactMoney.new(1234, "USD") })

    assert_includes passed_on, %(\n    "cents": 1234,\n)
    assert_equal %({\n  "price": #{MONEY_JSON}\n}), dropped
    assert_equal JSON.parse(passed_on), JSON.parse(dropped)
  end

  def test_an_arity_zero_to_json_only_works_when_you_call_it_yourself
    # Measured correction to the usual framing. The split is not
    # JSON.generate versus to_json — generate honours a custom to_json too.
    # It is arity: everything except a direct call passes a JSON::State, so a
    # `def to_json` with no parameters passes its own unit test and then
    # raises ArgumentError the moment the object is nested in anything.
    naive = JsonOps::NaiveMoney.new(1234, "USD")

    assert_equal MONEY_JSON, naive.to_json

    err = assert_raises(ArgumentError) { JSON.generate(naive) }
    assert_equal "wrong number of arguments (given 1, expected 0)", err.message
    assert_raises(ArgumentError) { JSON.generate({ "price" => naive }) }
    assert_raises(ArgumentError) { { "price" => naive }.to_json }
  end

  def test_the_generator_never_checks_what_to_json_returned
    # Worse than an exception: to_json is spliced in verbatim, so a fragment
    # that is not JSON produces a document that is not JSON. Nothing fails on
    # the writing side; the failure surfaces at whoever reads it.
    lying = JsonOps::LyingMoney.new(1234, "USD")
    written = JSON.generate({ "price" => lying })

    assert_equal '{"price":$12.34}', written
    err = assert_raises(JSON::ParserError) { JSON.parse(written) }
    assert_includes err.message, "$12.34"
  end

  def test_an_object_with_no_to_json_becomes_its_to_s_unless_you_ask_for_strict
    # The default for an unknown object is not an error, it is the object's
    # to_s wrapped in quotes — an address string shipped to a client that
    # expected an object. strict: true is what turns it back into a failure.
    assert_match(/\A"#<Object/, JSON.generate(Object.new))

    err = assert_raises(JSON::GeneratorError) { JSON.generate(Object.new, strict: true) }
    assert_equal "Object not allowed in JSON", err.message

    # Symbols go the same way, which is why a Hash of symbol values looks
    # correct right up to the point someone turns strict on.
    assert_equal '"paid"', JSON.generate(:paid)
    assert_raises(JSON::GeneratorError) { JSON.generate(:paid, strict: true) }
  end

  def test_a_round_trip_of_a_non_finite_float_raises_at_the_first_step
    # NaN and Infinity are not in the JSON grammar. Both directions refuse
    # them by default, so a division that went non-finite fails loudly at
    # serialization rather than arriving somewhere as null.
    err = assert_raises(JSON::GeneratorError) { JSON.generate({ "ratio" => Float::NAN }) }
    assert_equal "NaN not allowed in JSON", err.message
    assert_raises(JSON::GeneratorError) { JSON.generate({ "ratio" => Float::INFINITY }) }
    assert_raises(JSON::GeneratorError) { JsonOps.round_trip(Float::NAN) }

    # to_json is not a way around it: same generator, same refusal.
    assert_raises(JSON::GeneratorError) { Float::NAN.to_json }

    # allow_nan is per call and needed on both sides. Writing with it and
    # reading without is the trap, because the output looks like JSON.
    written = JSON.generate({ "ratio" => Float::INFINITY }, allow_nan: true)
    assert_equal '{"ratio":Infinity}', written
    assert_raises(JSON::ParserError) { JSON.parse(written) }
    assert_equal 1, JSON.parse(written, allow_nan: true)["ratio"].infinite?
    assert JSON.parse('{"ratio":NaN}', allow_nan: true)["ratio"].nan?

    # One rescue covers both directions.
    assert_operator JSON::GeneratorError, :<, JSON::JSONError
    assert_operator JSON::ParserError, :<, JSON::JSONError
  end

  def test_dump_and_load_do_not_carry_generate_and_parse_defaults
    # The pair that quietly disagrees with the pair above: JSON.dump allows
    # non-finite floats, so it writes a document JSON.parse cannot read back.
    # Only its own counterpart, JSON.load, will take it.
    assert_equal true, JSON.dump_default_options[:allow_nan]

    written = JSON.dump({ "ratio" => Float::NAN })
    assert_equal '{"ratio":NaN}', written
    assert_raises(JSON::ParserError) { JSON.parse(written) }
    assert JSON.load(written)["ratio"].nan?

    # allow_blank is the same asymmetry on the empty input every HTTP client
    # eventually hands you.
    assert_equal true, JSON.load_default_options[:allow_blank]
    assert_nil JSON.load("")
    assert_raises(JSON::ParserError) { JSON.parse("") }
  end

  def test_an_integer_survives_exactly_where_the_same_digits_as_a_float_do_not
    # Ruby has bignums, so the JavaScript ceiling does not apply: 2**53 + 1
    # and a thirty digit integer come back byte for byte.
    assert_equal 9_007_199_254_740_993, JSON.parse("[9007199254740993]")[0]
    assert_instance_of Integer, JSON.parse("[9007199254740993]")[0]
    assert_equal "[9007199254740993]", JSON.generate(JSON.parse("[9007199254740993]"))
    assert_equal "[123456789012345678901234567890]",
                 JSON.generate(JSON.parse("[123456789012345678901234567890]"))

    # A decimal point changes the type, and Float is where the digits go. The
    # same value written with .0 loses the low bit, and even the text it
    # regenerates as no longer looks like what was sent.
    assert_equal 9_007_199_254_740_992.0, JSON.parse("[9007199254740993.0]")[0]
    assert_equal "[9.007199254740992e+15]", JSON.generate(JSON.parse("[9007199254740993.0]"))
    assert_equal 1.0, JSON.parse("[1.0000000000000000001]")[0]

    # decimal_class: BigDecimal is the documented answer and it is not one
    # here. bigdecimal is not a default gem on Ruby 3.4, so bundler hides the
    # copy that shipped with the image the way it hides anything the Gemfile
    # did not ask for — the spec is not even visible — and installing it hits
    # the same missing compiler as oj, being a C extension too.
    assert_empty Gem::Specification.find_all_by_name("bigdecimal")

    # String is the decimal_class that needs nothing, and it only touches
    # tokens that had a fraction or an exponent, so ids stay Integer while
    # money keeps every digit.
    exact = JsonOps.parse_exact_decimals('{"id":9007199254740993,"rate":1.0000000000000000001}')
    assert_equal 9_007_199_254_740_993, exact["id"]
    assert_equal "1.0000000000000000001", exact["rate"]
  end

  def test_pretty_generate_changes_the_bytes_and_nothing_else
    doc = { "b" => [1, 2], "a" => { "x" => nil }, "empty_list" => [], "empty_map" => {} }
    compact = JSON.generate(doc)
    pretty = JSON.pretty_generate(doc)

    refute_equal compact, pretty
    assert_operator pretty.bytesize, :>, compact.bytesize
    assert_equal JSON.parse(compact), JSON.parse(pretty)

    # Insertion order is preserved by both, so a diff of two pretty documents
    # is a diff of the data. Empty containers stay on one line, and there is
    # no trailing newline — append one yourself before writing a file.
    assert_equal %w[b a empty_list empty_map], JSON.parse(compact).keys
    assert_equal %w[b a empty_list empty_map], JSON.parse(pretty).keys
    assert_includes pretty, %("empty_list": [],)
    assert_includes pretty, %("empty_map": {})
    refute pretty.end_with?("\n")
  end

  def test_the_json_in_use_is_the_c_extension_ruby_ships_and_nothing_could_replace_it_here
    # The tripwire for the Gemfile's claim. Nothing in the bundle provides
    # json — Gem.loaded_specs lists only what the Gemfile resolved, and json
    # is not in it — yet JSON answers anyway, because it is a default gem
    # baked into the Ruby installation.
    refute Gem.loaded_specs.key?("json")
    spec = Gem::Specification.find_all_by_name("json").fetch(0)

    assert spec.default_gem?
    assert_equal "2.9.1", JSON::VERSION
    assert_equal JSON::VERSION, spec.version.to_s

    # And it is the compiled parser and generator, not a pure-Ruby fallback:
    # both are .so files under the interpreter's own archdir, built when Ruby
    # was built. That is the fast path oj would be replacing — and the
    # compiler that produced it is not in the image, which is why oj cannot
    # be installed and why requiring it is a LoadError rather than a
    # benchmark.
    archdir = RbConfig::CONFIG["archdir"]
    assert_includes $LOADED_FEATURES, File.join(archdir, "json/ext/parser.so")
    assert_includes $LOADED_FEATURES, File.join(archdir, "json/ext/generator.so")
    assert_equal JSON::Ext::Generator, JSON.generator
    assert_equal JSON::Ext::Parser, JSON.parser

    assert_empty JsonOps.compilers_on_path
    assert_raises(LoadError) { require "oj" }
  end
end