codesamplex

What the network found

Every line below is a measurement, not an opinion. Each one links to a published sample whose contract is executed in a pinned container with the network switched off, so you can re-run it and disagree with the result.

Sample ids are content addresses. GET /v1/samples/<id> returns the sample this page links to, and csx runs its contract in the same container the network verified it in.

The page is translated; the findings are not. Each one quotes an error string, an attribute name or a numeric result, and a translated paraphrase of a measurement is no longer the measurement.

29 findings across 8 ecosystems.

Contradicts an official source

Here the belief is not folklore: it is what the project's own documentation, or the specification it implements, says. Both halves are checkable — the quote is one link away, the measurement one container run.

  1. cargoserde 1.0.229, serde_json 1.0.151

    Believedserde's container-attribute documentation says deny_unknown_fields “is not supported in combination with flatten, neither on the outer struct nor on the flattened field”, which reads as a pair the derive will refuse.

    Measuredserde_derive implements the pair: rustc compiles a struct carrying both with an empty stderr — no error, no warning — and the ordinary case works, since a flattened struct is accepted and only genuine leftovers are rejected. What does break is narrower than the note: a flattened map beside deny_unknown_fields can never collect a key, because every key it was meant to catch is reported unknown instead; and a struct carrying deny_unknown_fields accepts, once flattened into another struct, the same key it rejects when it stands alone.

  2. npmjose 6.2.8, Node 22

    BelievedRFC 7518 §3.2 says a key “of the same size as the hash output (for instance, 256 bits for HS256) or larger MUST be used with this algorithm”, and jose implements that specification, so a five-byte HMAC secret is expected to be refused somewhere in the stack.

    Measuredjose signs and verifies an HS256 token with a five-byte key and reports nothing, and WebCrypto underneath imports the same 40-bit HMAC key just as willingly, so nothing below the library catches a weak secret either.

  3. npm@modelcontextprotocol/sdk 1.30.0, protocol 2025-11-25, Node 22

    Believedthe MCP specification splits tool failures into two channels and puts “Unknown tools” in the first one — “Protocol Errors: Standard JSON-RPC errors” — printing the example as an error object with code -32602, so a call naming a tool that does not exist should arrive as a JSON-RPC error.

    Measuredthe SDK's own McpServer answers it as a successful response instead: await client.callTool({name: "no_such_tool", arguments: {}}) resolves, its isError is true, and the -32602 arrives inside the text — “MCP error -32602: Tool no_such_tool not found” — so the code is prose rather than an error object, and a caller that detects failure by catching sees none. The contradiction is that one bullet and no more — the same page assigns input validation errors to the isError channel by design, which is where they arrive — and the same server does reject with a real McpError carrying code -32602 for an unregistered resource URI, so which channel a failure uses is decided by the handler it reached rather than by the kind of failure it is.

Widely believed, measured otherwise

Here the belief is migration advice, a habit carried over from a neighbouring library, or something everyone repeats. The measurement is the same kind of measurement.

  1. npmbcryptjs 3.0.3, Node 22

    Believeda password hash covers the whole password.

    Measuredbcrypt truncates at 72 bytes and neither hashSync nor compareSync reports it: a second, different password sharing the first one's 72-byte prefix verifies against that hash, and so does the bare prefix, while 71 bytes does not; the limit is bytes, so 36 accented characters survive and 37 lose their tail, and bcryptjs's own truncates() is a separate call you have to make yourself.

  2. npmnode:crypto scryptSync, Node 22

    Believedthe memory scrypt needs is 128 * N * r — the figure Node's own documentation quotes, hedged as “It is an error when (approximately) 128 * N * r > maxmem” — so budgeting exactly that raises the cost safely.

    Measuredbudgeting exactly 128 * N * r is rejected with ERR_CRYPTO_INVALID_SCRYPT_PARAMS; the accepted minimum is exactly 128 * r * (N + p + 2), pinned to the byte at six parameter sets that vary N, r and p independently — 3072 bytes above the quoted figure, at the defaults and at N=32768 alike, and at N=32768 the quoted figure is exactly the 32 MiB default maxmem, so the next cost step looks like it just fits and does not.

  3. pypipolars 1.43.2, Python 3.12

    Believeddf[mask] filters rows, the way it does in pandas.

    Measureda boolean mask in brackets selects COLUMNS: against a frame whose column count differs from the mask length it raises ValueError, and against a frame with as many columns as the mask has rows — the shape of most test fixtures — the length check passes and it returns the wrong columns with no error at all.

  4. composermonolog/monolog 3.9.0, PHP 8

    BelievedMonolog 3 removed the integer level constants, so an upgrade means replacing every Logger::WARNING.

    MeasuredLogger::WARNING is still defined, still 300, still equal to Level::Warning->value, and addRecord still accepts an int; what actually breaks is is_array($record), because a record is now a LogRecord object that keeps answering $record['message'] — so the migration reads as finished while the array branches quietly stop being taken.

  5. npmlightningcss 1.33.0, npm 10, Alpine

    Believednpm installs only the right native package on Alpine, because each platform package declares the libc it was built for.

    Measurednpm ci installed BOTH linux-x64 variants, glibc and musl, because the lockfile this npm 10 image wrote records os and cpu on every optional entry and never libc — every darwin, win32 and arm64 package was correctly skipped, and the unusable glibc build is a full second copy of the 10 MB addon on disk. This is npm's lockfile writer, not lightningcss, and it is fixed upstream: npm records libc from 11.11.0 on, so the repair is regenerating the lockfile rather than upgrading the npm that reads it.

  6. npmesbuild 0.25.12, Alpine

    Believedrunning esbuild on Alpine means installing its musl-specific build.

    Measuredthere is no musl build to install: none of esbuild's optional platform dependencies mentions musl, the linux-x64 package npm picks declares no libc constraint, and its binary has no ELF interpreter at all — read from the program headers on a musl image where node itself names musl's loader.

  7. npmjose 6.2.8, Node 22

    Believedcatching JWTClaimValidationFailed handles the claim checks jwtVerify performs.

    MeasuredJWTExpired is a sibling of JWTClaimValidationFailed rather than a subclass, so every expired token falls past that clause into the generic branch, while a not-yet-valid nbf — checked without being asked — is caught by it.

  8. npmzod 4.4.3, Node 22

    Believedz.coerce.number() parses a numeric string, rejecting what is not a number.

    Measuredit is Number(input) followed by the number check, not a numeric parser, so "", " ", null, false and [] are all accepted and arrive as 0 — an empty form field or a null column silently becomes zero — while "1e999" is rejected, because Number() overflows it to Infinity.

  9. npmvitest 4.1.10, Node 22

    Believeda forgotten await on expect(...).rejects makes the test pass while asserting nothing.

    Measuredwhich half is true depends on the test function: in a sync one the forgotten await still fails the test and carries the real rejection message, and in an async one the assertion settles first, so the test is reported passed and the failure becomes the run's single unhandled error — green test, red run, and any tooling reading only test states calls it a pass.

  10. npmbun:sqlite, Bun 1.3.14

    Believedthe options argument to new Database(path, options) overrides defaults.

    Measuredit replaces the open flags outright, so new Database(":memory:", {}) throws SQLiteError SQLITE_MISUSE where new Database(":memory:") works, and { create: false } lands on the same zero flags — only a true access mode or a strict/safeIntegers key puts the default back.

  11. composerguzzlehttp/guzzle 8.0.2, PHP 8

    Believedif ($e->hasResponse()) { $e->getResponse(); } is how you read the response off a Guzzle RequestException.

    Measuredon guzzle 8 neither method is declared on RequestException — getResponse moved down to ResponseException with a non-nullable return type, and hasResponse is declared on neither class — so the guzzle 7 idiom is a fatal “Call to undefined method …::getResponse()”, not a deprecation.

  12. gemjson 2.9.1, Ruby 3

    BelievedJSON.dump is JSON.generate under another name.

    MeasuredJSON.dump defaults to allow_nan, so it writes {"ratio":NaN} — a document JSON.parse refuses and only JSON.load will read back — while JSON.generate refuses the same float outright; JSON.load carries the matching asymmetry with allow_blank, returning nil for an empty string where JSON.parse raises.

  13. hexElixir's built-in JSON vs jason 1.4.4

    Believeddropping Jason for the JSON module in Elixir 1.18+ is a module rename.

    Measuredthe two encode the same payload byte for byte, but the decode error is JSON.DecodeError, so a rescue Jason.DecodeError clause compiles, still reads correctly and catches nothing; JSON.decode/1 returns a bare reason tuple rather than a struct, and JSON.decode/2 does not exist, so keys: :atoms has nowhere to go.

  14. pubcollection 1.19.1, Dart 3

    Believedtwo Lists holding the same values are equal.

    Measured== on a List, Map or Set is identity, and package:test's equals matcher deep-compares — so the assertion is green on exactly the values production calls unequal; a const collection is canonicalized and does compare equal, which is what makes the rule look inconsistent, and copying it loses the equality again.

  15. golangshopspring/decimal 1.4.0, Go 1.26

    Believeda decimal type is exact, which is the reason to reach for one.

    MeasuredDiv is DivRound reading a mutable package-level global, decimal.DivisionPrecision, which defaults to 16 — so (1/3)*3 is 0.9999999999999999 and not 1, any dependency in the process can move the precision, and nothing at the call site says so; DivRound takes the precision as an argument and QuoRem is the one that keeps the remainder.

  16. golangspf13/cobra 1.10.2, Go 1.26

    Believeda cobra command tree with no SetArgs runs with no arguments.

    Measuredwith no SetArgs at all cobra parses os.Args[1:], so a command tree driven from a test binary parses that binary's own arguments and fails on a flag nobody wrote; the guard is c.args == nil, so SetArgs(nil) reopens the same fallback and only an empty non-nil slice means “no arguments”. cobra's one escape hatch keys on the program name being exactly cobra.test, which saves its own tests and nobody else's.

  17. pypiorjson 3.11.9, Python 3.12

    Believedorjson's README says “JSONEncodeError is a subclass of TypeError”, which reads as a narrower class, so except orjson.JSONEncodeError looks tighter than except TypeError.

    Measuredorjson.JSONEncodeError is TypeError — the same object, so the README's sentence holds only in the sense that every class is a subclass of itself — and the two clauses are therefore identical: either one swallows every TypeError raised in the block, not only the encoder's. JSONDecodeError, by contrast, really is a distinct subclass of json.JSONDecodeError.

  18. pypiattrs 26.1.0 vs dataclasses, CPython 3.12

    Believed@dataclass(slots=True) is the stdlib equivalent of an attrs slotted class.

    Measuredon CPython 3.12 the stdlib leaves the __class__ closure cell pointing at the class it discarded, so zero-argument super() inside a slotted dataclass raises TypeError at call time — not at definition — while the identical attrs class works because attrs rebinds the cell; weakref.ref also raises on the slotted dataclass and not on the attrs one.

  19. pypifreezegun 1.5.5, Python 3.12

    Believedfreeze_time moves the wall clock, so monotonic timers are unaffected.

    Measuredit patches time.monotonic and time.perf_counter to the same frozen wall clock, so a duration measured across the boundary of the freeze comes out about thirty years long, and moving the frozen clock backwards makes time.monotonic() go backwards — a deadline written as monotonic() + timeout is never reached.

  20. cargoaxum 0.8.9

    Believedaxum answers a bad JSON body with 422.

    Measuredit answers with three statuses and only one of them is 422: syntactically broken JSON is 400, a body that parses but does not fit the target type — a wrong field type, or a required field left out — is 422, and a request with no Content-Type at all is 415 carrying the rejection message where the handler's 201 would have been, so a single assertion for “bad input” is wrong on two of the three. The 400's rejection body is text/plain rather than JSON; and the Content-Type check is not string equality — the header is parsed and its type has to be application, so application/json; charset=utf-8 and application/vnd.csx+json are accepted while text/json is refused exactly like a header that was never sent.

  21. cargoonce_cell 1.21.4 vs std

    Believedstd absorbed once_cell, so the dependency can go.

    Measuredalmost — and the remainder is two methods, both of them the fallible ones. Compiled with the rustc this sample was measured on, LazyLock::force_mut, DerefMut on LazyLock and OnceLock::wait all build, so the reasons usually quoted for keeping the crate are out of date, and std::cell::OnceCell and LazyCell cover its unsync half. OnceLock::get_or_try_init and try_insert do not build: both are E0658, the first behind once_cell_try with tracking issue 109737, the second behind once_cell_try_insert — so a fallible initialiser has no std spelling, and the get-then-set stand-in written in their place is not equivalent, because 16 threads racing it run the initialiser 16 times where get_or_init runs it once.

  22. golanggorm.io/driver/sqlite 1.6.0 over mattn/go-sqlite3 1.14.49, Go 1.26

    Believeda cgo package cannot be built with CGO_ENABLED=0, so the build catches it.

    Measuredgo-sqlite3 compiles a stub instead: the import builds, the binary links, and the stub still registers the database/sql driver name sqlite3, so finding that name in sql.Drivers() proves nothing about the driver working. The first thing that connects is what fails, with an error naming CGO_ENABLED=0 and saying it “requires cgo to work”, and database/sql defers even that: sql.Open only records the driver name and returns a nil error, and Ping is where “This is a stub” surfaces. So the build is green and the first connection is where it breaks.

  23. golangspf13/viper 1.21.0, Go 1.26

    BelievedAutomaticEnv feeds the environment into Unmarshal the way it feeds Get.

    MeasuredUnmarshal enumerates AllKeys and reads each key it finds, and AutomaticEnv contributes no keys to that list — it cannot, since it would have to guess names — so with CSX_LOG_LEVEL exported, GetString("log.level") returns env while AllSettings is empty and the struct field stays "", with no error anywhere to say the two disagree. The repair is making the key enumerable: either a SetDefault that never wins the lookup it just enabled, or viper.ExperimentalBindStruct, which takes the key list from the destination struct.

  24. composersymfony/console 8.1.4, PHP 8

    Believedpassing --no-interaction to CommandTester::execute turns the prompts off, the way it does on the command line.

    Measuredwhat interprets that flag is the application run, and a CommandTester is not one, so under it the option binds and does nothing: getOption('no-interaction') is true and the question is asked anyway. -v is the same dead end — bound true while the output stays at VERBOSITY_NORMAL and the verbose writeln prints nothing — and what the tester does read are execute()'s own interactive and verbosity options, or the same command driven through ApplicationTester, where both flags mean what they say.

  25. pubshelf_router 1.1.4, Dart 3

    Believeda route handler's parameter names say which capture each one receives.

    Measuredcaptures are applied positionally in the order the route pattern declares them and the closure's parameter names are never read, so a handler written (Request request, String id, String org) against /orgs/<org>/users/<id> receives the org capture in id: /orgs/acme/users/u42 answers “acme/u42” where the names promise “u42/acme”, and nothing reports it. The count is not checked at registration either — one argument too many registers cleanly and becomes a NoSuchMethodError on the first request that matches.

  26. hexecto 3.14.1

    Believedan empty string in the params either arrives as an empty string or clears the field.

    Measuredcast compares it against empty_values — [""], compared after trimming, so a whitespace-only string counts too, while a value that survives the check is stored untrimmed — and substitutes the field's declared default, which is nil only for a field that has none: sending "" for a field defaulting to "member" over a stored "admin" writes "member", so an empty form field demotes rather than clears. When the substituted default equals the data the key is absent from changes rather than present and empty, which is why the debugger shows nothing there and validate_required reports “can't be blank” for a param that did arrive.

How to check any line here

Open the sample, read its contract, run it. The contract is the sample's own test: it runs offline in a pinned container, and the signed receipt of that run is what the network stores. Nothing here rests on our reading of a library — only on what the library did.

Findings whose sample could not be confirmed live are not on this page. The list is shorter than what the sample pool contains, on purpose.