The API reference gained the Row section with the release, which is the
right home for the details and the wrong place to discover the feature
exists. Three gaps closed.
The IfxPy migration guide is the page for people in exactly the position
that prompted this: IfxPy is positional-only, so those codebases grow a
description/zip helper that yields a dict and still no attribute access.
That page now shows the helper it replaces alongside the one-liner, and
notes the .lower() in the hand-written version is already a no-op since
Informix folds unquoted identifiers.
Quickstart gains one sentence, next to the paramstyle note where someone
is already thinking about how rows come back.
The phase log records 2026.09.03, including where the testing actually
found the shadowed-names bug: not in the differential test across twenty
awkward types, which found nothing, but in six lines of name bookkeeping
that were correct when written and stopped being correct an hour later.
Verified live by content, and the #rows anchor the new cross-links point
at resolves.
The API page documents Row: the three access styles, that it is set on
the connection, that it subclasses tuple so existing code is unaffected,
the measured 9% and why that makes it opt-in, and the four behaviours
worth knowing (lower-case folding, expression column names, duplicate
names, and a column beating a method of the same name).
The types page gains the placeholder-rewriter entry. It is the only one
in that list affecting the write path rather than the read path, so
anyone who has stored HH:MM times or URLs with ports through this driver
should check what actually landed.
Test counts updated to 472/471. Verified live by content.
The rewriter that converts :N placeholders to ? was a bare regular
expression, so it rewrote the inside of string literals. Any HH:MM time,
any URL with a port, any key:value string was silently stored wrong. It
was not opt-in either: the rewrite runs whenever a statement has
parameters, whatever placeholder style the caller used. Replaced with a
scanner that substitutes only outside quotes and comments, using
Informix's own lexical rules measured against all three servers.
Row adds column-name and attribute access alongside the existing
positional access, on request from a user coming from pyodbc and
mssql-python. Opt-in at connect time, because it costs about 9% on bulk
fetch and that is not a trade to make for someone who never reads a
column by name.
472 tests on 15 and 14.10, 471 on 12.10.
I hand-listed the names a column could shadow: count and index, the two
tuple methods. Then I wrote keys(), _asdict() and _fields on the same
class and did not go back. A column called "keys" returned a bound
method instead of a value, silently, which is precisely the failure
shape this driver spent a fortnight removing from its decoders.
The fix is not a longer list. The reserved set is now computed from
dir(Row), so adding a method later cannot reopen the hole, and the test
is parametrized over that computed set so it grows with the class.
_fields and _map moved to name-mangled attributes. They are read by
repr() and _asdict(), so shadowing the public _fields with a column
would have made the machinery report the column value instead of the
column names. Mangling keeps the two apart, and there is a test that
shadows _fields and checks repr still works.
Also exercised, and all clean: value-identical output against the plain
tuple path across twenty columns covering every awkward type plus a
fully NULL row, which is the strongest available statement that Row is a
presentation layer and not a bug; zero-column and 500-column rows;
unicode, spaced, digit-leading, empty-string and dunder column names;
rows outliving eviction of their class from the bounded cache; the four
async fetch routes and the async pool; and twelve threads racing on the
class cache, which correctly share one class per shape.
472 tests on 15 and 14.10, 471 on 12.10.
Field request from a user running this alongside SQL Server, where both
pyodbc and mssql-python hand back rows that take a position, a column
name, and an attribute. The argument is readability on wide
projections: row[11] tells a reader nothing, and stays correct only
until somebody adds a column in the middle.
row_factory=Row gives all three. It is set on the connection, so it is
one line for the whole application rather than per query, which is what
"without any additional coding" has to mean in practice. A cursor can
override it.
Opt-in, not default, and the number is why. Measured on a 20,000-row
five-column fetch: 37.2 ms with tuples, 40.8 ms with Row, about 9%.
Defaulting it on would move the published 1.05-1.15x ratio against IfxPy
to roughly 1.15-1.25x. That is not a trade to make on behalf of somebody
running a bulk export that never looks at a column by name.
Row subclasses tuple, so row == (1, "x") still holds. That constraint
shaped the design more than anything else: this suite compares fetched
rows against plain tuples in hundreds of places, and so does everybody's
code. Slices degrade to plain tuples, since a slice has no column map.
Details that had to be decided rather than assumed, all measured against
the servers: Informix folds unquoted identifiers to lower case, so the
.lower() in the workaround people write by hand is a no-op. Expression
columns get names like "(count(*))" that cannot be attributes, so they
are subscript-only. Duplicate names resolve to the first occurrence,
matching pyodbc. And tuple already defines count and index, so a column
with either name gets a descriptor and wins, because on a database row
that is plainly what the caller meant.
The per-shape class is cached, so a type() call does not land on every
small query, and rows pickle by rebuilding from their field names.
We advertise paramstyle="numeric" and the wire protocol takes ?, so :N
placeholders are rewritten on the way out. That was a bare
re.sub(r":(\d+)", "?", sql), which rewrote the inside of string literals:
UPDATE jobs SET url = 'http://host:8080/x' WHERE id = ?
stored as 'http://host?/x'
Any HH:MM time, any URL with a port, any aspect ratio, any key:value
string. It wrote wrong data and said nothing about it.
None of it was opt-in. The rewrite fires whenever a statement has
parameters, whatever placeholder style the caller actually used, so
writing ? everywhere and never touching numeric style did not protect
you. It also changed the placeholder count while num_qmarks was still
computed from len(params), leaving driver and server disagreeing about
how many binds exist.
Replaced with a single pass that substitutes only outside quotes and
comments. The lexical rules are Informix's own, measured against all
three servers rather than assumed from standard SQL, and one of them
would have been got wrong by habit: a backslash escapes nothing, so
'a\'b' is an unterminated string and draws -282. A scanner written to
Postgres reflexes would desync there and corrupt everything after it.
Block comments do not nest, the first */ closes them. Braces are a
comment. :: is stepped over as a unit so a cast can never be read as the
start of a placeholder.
An unterminated quote or comment consumes the rest and substitutes
nothing further. Under-substituting leaves the server to reject SQL that
was already malformed; guessing would corrupt a literal.
This was the last place the driver inferred meaning from SQL text
instead of handling it properly. Eleven of the new tests fail against
the previous commit.
Content. The types page now warns about the two framing bugs this
release fixed, since both silently corrupt data on versions before it:
a BLOB or CLOB in any position but last shifted every column after it,
and a NULL collection did the same. The API page documents two things
that are now user-visible: only one scrollable cursor may be open per
connection (previously the second statement drew -285 and destroyed the
cursor as collateral), and transaction control written as SQL is tracked
(previously rollback() after a SQL BEGIN WORK sent nothing and reported
success). The phase log gains a section on the review itself, including
the pattern behind almost all eleven bugs. Test counts updated to
457/456.
Style. 136 em-dashes across 22 content files, plus Hero.astro and two
stylesheets, rewritten rather than substituted -- swapping the character
for a comma leaves prose that reads like it lost an argument with a
linter. Bullets that used a dash to gloss a term now use a colon;
parenthetical asides became their own sentences or moved inside
brackets. The rendered HTML is clean.
Verified against the live site by content, not status code: these pages
return 200 for every path and render the 404 body, so a stale deploy
looks perfectly healthy.
No new features. The result of going back over the driver looking for
the shape of past bugs rather than for new symptoms.
Two returned wrong data silently: smart LOBs were read 77 bytes short,
shifting every column after them, and NULL composite UDTs skipped their
length field. One did nothing at all: rollback() after a SQL BEGIN WORK
returned successfully having sent nothing, leaving the rows it was asked
to discard in place and the connection back in the pool holding an open
transaction.
The rest: five ordinary query forms failing with -260 because
classification guessed from the first word; a scrollable cursor taking
the whole connection down with it; two unguarded exits leaking failed
statements; deferred cleanup landing inside somebody else's statement,
and a GC-time finalizer that couldn't tell it was already on the thread
holding the wire lock; the async layer borrowing the process-wide thread
pool; and two readers sharing one socket stream without agreeing.
457 integration tests on 15 and 14.10, 456 on 12.10 (one skip, no CTEs
before 14.10) -- up from 414. Every fix has at least one test that fails
against 2026.09.01.
Every blocking call in informix_db.aio went through asyncio.to_thread,
which runs on the event loop's default executor. That executor belongs
to the process, not to us, and it is sized from the CPU count --
min(32, cpu_count + 4), so six threads on a two-CPU container.
Cancelled calls hold their threads. asyncio.to_thread cannot interrupt a
worker, so a cancelled await leaves the thread running the wire call
until the read timeout. Cancellation is ordinary in a web app -- a
client disconnect cancels the request task -- so a handful of them pins
every thread in the shared pool. Measured: six cancelled calls against a
six-worker default executor starve an unrelated to_thread indefinitely,
and with the executor held, four concurrent driver queries never ran at
all.
Pool concurrency was capped by the same number without saying so. A pool
with max_size=20 on a two-CPU box ran six queries at a time.
Each connection now owns one thread. That is the right size rather than
a compromise -- the sync connection serializes every wire operation on
its own lock, so a second thread could do nothing but wait for the
first. It also avoids a deadlock a shared pool-sized executor invites:
with N threads and N connections, N tasks blocked in acquire occupy
every thread while the connection they are waiting for is held by a task
that now needs a thread to finish and release it.
The executor lives on the sync connection so it survives being returned
to the pool and handed out again, rather than being rebuilt per acquire.
release() runs on the connection's own thread, which is idle by
definition and keeps release off any pool that waiters may have filled
-- release has to win that race, since it frees what they are waiting
for. connect() and pool acquire stay on the default executor: there is
no connection yet to own a thread, and neither can deadlock against
query threads any more.
close() shuts the executor down, and a weakref finalizer is the backstop
for a connection dropped without it -- ThreadPoolExecutor workers park
on the work queue rather than exiting when idle, so an executor that is
never shut down leaks its thread for the life of the process. The
thread-count test caught that gap; it was missing from the first cut.
IfxSocket owns a read-ahead buffer that BufferedSocketReader fills and
drains. Connection._drain_to_eot, _raise_sq_err and the login path
bypass that reader and call IfxSocket.read_exact directly, which recv'd
from the socket without looking at the buffer. Bytes sitting in the
buffer were skipped, and skipped bytes in a length-framed protocol do
not announce themselves -- the next read lands mid-field and every read
after it is wrong.
Nothing triggers it today. The server sends one response per request, so
recv returns exactly that response and the buffered reader consumes all
of it before control returns to a direct read. That is a property of the
traffic, not of the code, and the buffer is connection-scoped precisely
so read-ahead can cross response boundaries -- pipelined executemany
already puts several responses in flight. read_exact now drains the
buffer first, which costs one branch on a cold path and makes the two
paths agree by construction rather than by luck.
fill_recv_buf believed whatever byte count it was handed, and that count
is almost always a length field straight off the wire. A garbage
0x7FFFFFFF reads as a 2 GB request and the fill loop sits in recv until
the read timeout while the buffer grows. It now refuses above
IFX_MAX_READ_BYTES (256 MiB default) with an error naming the number,
which is the actual diagnostic: a length that absurd means framing was
already lost upstream.
BufferedSocketReader.skip advanced the cursor arithmetically with no
guard, so a negative count rewound it and re-decoded consumed bytes as
the next field. The base reader's skip delegates to read_exact and does
guard; this one diverged.
Transaction control run as SQL desynced Connection._in_transaction,
which is what commit() and rollback() are guarded by and what the pool
reads to decide whether a returned connection needs cleaning up. With
autocommit on, cursor.execute("BEGIN WORK") opened a real transaction
while the flag stayed False, so rollback() returned successfully having
sent nothing and the rows it was asked to discard survived. The
connection then went back to the pool holding an open transaction and
its locks. With autocommit off it failed instead: the driver's implicit
SQ_BEGIN fired first and the caller's BEGIN WORK got -535.
The server labels these -- 34 BEGIN, 35 COMMIT, 36 ROLLBACK, with and
without the WORK keyword, measured on all three versions. JDBC reads the
same values off the describe and calls setTxBeginState/setTxEndState.
_ensure_transaction moves to after the describe, matching JDBC's
initiateTransaction placement, which is what makes it possible to skip
for transaction control at all -- until the describe lands you cannot
know that is what the statement is.
The driver chose between "open a cursor and fetch" and "execute and
release" by checking whether the first word of the SQL was SELECT. That
misses a leading comment in any of the three Informix flavours (--,
/* */, { }), a parenthesized select, a parenthesized UNION, and a CTE.
All five are perfectly ordinary queries. All five failed with -260,
"Cursor name already in use" -- an error that describes neither the
cause nor anything the caller did. It says "cursor" because the driver
sent SQ_EXECUTE where the server was waiting to open one.
The server had been answering this question all along. statement_type is
the first field of the DESCRIBE response and parse_describe has always
put it in the metadata dict, where nothing read it.
The comment defending the heuristic said nfields couldn't distinguish
these cases because INSERT INTO t VALUES (?) also describes a column.
That was true, and it argued for the wrong conclusion: statement_type
distinguishes them exactly. Measured against all three servers -- every
SELECT form above reports 2, that INSERT reports 6, UPDATE 33, DELETE
32, CREATE 45, EXECUTE PROCEDURE 56.
The predicate is now JDBC's IfxSqli.isResultSet: type 2, or type 56 with
at least one column, since a routine may or may not return rows. That
last clause fixes EXECUTE FUNCTION as a side effect -- it used to run
down the DML path and discard the return value. It now yields it.
executemany keeps the first-word check as a cheap pre-flight reject, and
gains an authoritative one after PREPARE for the forms the first word
misses. The refusal releases the statement, so the connection survives.
Informix 12.10 has no CTEs and rejects WITH with -201 at offset 1. That
is the server declining the grammar rather than the driver routing it
wrongly, and the test skips on it.
The cursor finalizer runs at GC time on whatever thread happened to
allocate, so it must not touch the wire while a statement owns it. It
checked with _wire_lock.acquire(blocking=False), meaning to ask "is
anyone using the wire?" -- but an RLock grants a reentrant acquire to
its own owner. When GC fired on the thread that was mid-statement, the
probe returned True, the finalizer concluded it had exclusive access,
and it sent CLOSE/RELEASE into the middle of the statement it had just
interrupted. The victim got -208 on valid SQL.
Refcounting hid it. A dropped cursor is freed at the drop, before the
next statement runs. A cursor caught in a reference cycle waits for a
collection instead, and cycles are ordinary -- any traceback that holds
a cursor makes one. Reproduced by putting an abandoned scrollable cursor
in a cycle and collecting mid-statement.
_wire_lock is now a small wrapper that tracks owner and depth, so
held_by_current_thread answers the question the finalizer was actually
asking. Same-thread GC defers to the cleanup queue exactly as another
thread's would. This also retires the _is_owned() call in
_ensure_transaction, which was reaching into CPython private API for
the same information.
The finalizer's error handling had the stale-cleanup defect too: a
leftover CLOSE draws -267, an OperationalError, which is in WIRE_ERRORS,
so it force-closed a healthy connection over a no-op. Server-reported
errors are told apart from wire failures by their sqlcode, matching
_drain_pending_cleanup.
SQLI gives a session a single statement slot. SQ_CLOSE, SQ_RELEASE and
SQ_SFETCH act on whatever statement is current -- none of them names
one. Ordinary use never notices, because a non-scrollable cursor
materializes its rows and releases the statement before returning.
A scrollable cursor holds the slot open on purpose, and two things went
wrong there without saying so.
Another statement on the same connection got -285, and the scrollable
cursor was collateral damage: its next fetch came back -267, "the
transaction has been rolled back, all locks released". Two errors,
neither naming the cause, from code that reads as completely ordinary --
iterate a large result set, run a lookup partway through. It is now a
ProgrammingError that explains the constraint, and the scrollable cursor
is left alone. Re-executing the same scrollable cursor is the one caller
allowed past the check, and it now closes its own server-side cursor
first, because it collided with itself too.
The deferred-cleanup queue was flushed before every PDU. But a finalizer
enqueues precisely because it lost the wire lock, meaning another thread
is mid-statement -- so the flush landed inside that thread's own
statement and released it. The victim saw -208 when it landed before the
first fetch, -267 between fetch batches. Flushing only at a statement
boundary is both correct and sufficient: cleanup that misses one
statement is picked up by the next.
A stale queue entry was fatal on top of that. The finalizer enqueues,
the cursor is then closed properly, and the leftover CLOSE draws -267 --
an OperationalError, which is in WIRE_ERRORS, which force-closed a
healthy connection. Server-reported errors are now told apart from wire
failures by whether they carry a sqlcode.
Noted while tracing this: _build_close_pdu and _build_release_pdu write
SQ_ID followed by write_int(opcode), which frames correctly only because
the four bytes happen to land as [statement_id=0][opcode]. JDBC's
sendStatementID writes the real id there, and _read_describe_response
has been parsing it into _statement_id all along without ever using it.
Addressing CLOSE/RELEASE correctly is not enough for multiplexing on its
own -- a second cursor's SFETCH addressed to its own id returns -259 --
so that stays open rather than half-done.
A statement that fails is still allocated server-side, and skipping the
RELEASE bricks the connection: the next PREPARE collides with the leaked
one and every later call returns a nonsense error whose offset points
back at the failed SQL. That guard has now been added six times, once
per exit, each after a user found the exit. Two were still open.
The parameterized-SELECT bind drain. The build was guarded and the drain
was not, and the drain is where the interesting failures land: passing a
string where the column is an INT encodes cleanly, so the rejection
comes from the server (-1213, -415) after the guard has been passed. A
wrong-typed parameter is about as common as application mistakes get.
The scrollable-cursor open, which had no guard at all. Worst place to
lack one -- the GC-time finalizer is armed on the line after the drain,
so a failure there left the statement allocated with no fallback of any
kind. SELECT ... FOR UPDATE reaches it: prepares clean, fails at OPEN
with -526.
All six now call one helper instead of hand-rolling the cleanup, which
also fixes a defect in the two copies that had a cursor to close. CLOSE
and RELEASE shared a single contextlib.suppress block, so a CLOSE that
raised skipped the RELEASE -- losing the half that matters, since a
leaked cursor handle is a nuisance and a leaked statement breaks the
next call. Each PDU gets its own suppression now.
All four new integration tests fail against the previous commit.
Fourteen framing bugs shipped before this. Every one was a column
reading the wrong number of bytes, and not one of them raised at the
point of the mistake -- the wrong width produced a plausible value and
corrupted whatever came next, so the damage surfaced in a different
column, a different row, or a different statement entirely.
All fourteen were detectable for free. The payload is a fully extracted
bytes object of known length, so after decoding N columns the offset has
to land exactly on the end. It didn't, and nobody looked.
_row_not_consumed now raises when it doesn't, naming the column shape
and the byte delta, and it is wired into all three decode paths --
readers fast path, legacy chain, and the generated decoder.
Turning it on found two more bugs on the first run:
Smart LOBs were read as a flat 72-byte field. They use the UDT envelope:
149 bytes populated ([ind=0][len=144][144 hex chars]), 5 bytes NULL. We
consumed 72 and left 77 behind, so a BLOB or CLOB anywhere but the final
column position shifted every column after it. The 144 bytes are the
72-byte locator hex-encoded, which means BlobLocator.raw had never held
a locator, only the first half of the hex text -- unnoticed because
read_blob_column resolves through the server's lotofile and never reads
the locator.
NULL composite UDTs skipped their length field, byte for byte the
LVARCHAR NULL bug in a branch twelve lines away.
Both are the same [indicator][int32 length][data] envelope, hand-written
a third and fourth time. _read_udt_envelope is now the only copy, and it
rejects a negative length instead of rewinding the offset into bytes it
already decoded.
Tests and docs only, no behaviour change. TLS was the last untested
surface and it came back clean.
Previous coverage stopped at the handshake; everything after it ran only
over plain sockets. That gap mattered because SSLSocket.recv is not
socket.recv: it returns at most one TLS record's worth of plaintext
however much you ask for, can return fewer bytes than are available, and
plaintext buffered inside the SSL object is invisible to the OS. The
Phase 39 buffered reader asks for up to 64 KB per call and loops until
satisfied — that loop had never been pressured the same way.
tests/test_tls_traffic.py runs real SQLI through a TLS-terminating
proxy: the framing-bug types end to end, payloads at 1/4096/16383/16384/
16385/32000 bytes (straddling the ~16 KB record boundary), 500- and
5000-row bulk fetches, error recovery, concurrent TLS sessions, and
three negative cases — TLS client against a plaintext port, plaintext
client against a TLS port (must raise, not hang), and a verifying
context rejecting a self-signed cert.
All clean on all three servers. The buffered reader handles SSLSocket
semantics correctly.
Scope, stated plainly: the proxy supplies the TLS half, so this covers
the driver's TLS path — the half we own. It does NOT cover IBM's
server-side TLS listener. Setting one up on the developer-edition image
was attempted and abandoned: Informix 15 wants a PKCS#12 keystore
(onkstash takes a .p12, not the older CMS .kdb) and the engine kept
rejecting the stash with GSK_ERROR_BAD_KEYFILE_PASSWORD even with a
keystore GSKit itself could open. That half is IBM's code; everything
below ssl.wrap_socket is identical either way.
414/414 integration on 15, 14.10 and 12.10 (was 399).
Every surface is now fuzzed: type framing, fetch batching, error
recovery, cursor lifecycle, threads, pooling, transactions, async
cancellation, executemany partial failure, scrollable cursors, smart
LOBs, and TLS.
Last of the untested surface: pipelined executemany, scrollable cursors,
and smart LOBs. One bug, in the first.
executemany builds all its BIND+EXECUTE PDUs after the PREPARE and
before anything is drained — that batching is what makes the pipeline
fast. If encoding a row raises there (a value the connection's codec
cannot represent), the exception escaped without sending the RELEASE,
leaking the prepared statement. The next PREPARE collided with it and
every later call on the connection failed with an error pointing at the
previous SQL.
Same failure as 2026.08.31.2, in a sibling path.
_execute_dml_with_params has guarded this exact case for the single-row
path for a long time; the pipelined path was never given the same
treatment. That is the recurring shape of these bugs: a hazard
understood in one place and not carried to the code beside it.
What the fuzzer found sound, which is the larger part of the result:
executemany constraint failures — duplicate-key and NOT NULL at
first/mid/last of batches from 2 to 1000 rows all recover, and
COUNT(*) always agrees with a full fetch, so the drain-N-responses
invariant holds under partial failure.
Scrollable cursors — fetch_first/last/prior/relative/absolute correct
at 0, 1, 2, 5, 50, 300 rows, including off both ends (None, not a wrap
or a crash) and a full forward walk after arbitrary positioning.
Twenty abandoned scroll cursors leak nothing.
Smart LOBs — round-trip at 0, 1, 255, 256, 1023, 1024, 4095, 4096,
65535, 65536 bytes, straddling the 4096-byte SQ_FILE chunk and the
64K mark, plus recovery from failed reads.
Still not fuzzed, stated plainly: TLS is handshake-tested against a
self-signed local socket, not a real Informix TLS listener (that needs
server-side keystore + onconfig SSL setup absent from the test
containers). The SQLI layer above the socket is identical either way.
399/399 integration on 15, 14.10 and 12.10 (was 356).
pool (2026.08.31.2)
Two more found by fuzzing rather than by users, outside the type system.
Both are ordinary-path bugs a happy-path test cannot reach.
A failed statement was never released. Successful DML sent PREPARE ->
EXECUTE -> RELEASE; failing DML sent PREPARE -> EXECUTE and stopped. The
statement stayed allocated server-side, collided with the next PREPARE,
and every subsequent call on that connection returned a nonsense error
(-255 "Not in transaction" under autocommit, -285 otherwise) whose
offset pointed back at the FAILED sql rather than the new statement.
In practice: one duplicate-key violation bricked the connection. An
INSERT tripping a unique constraint is about the most routine error an
application can hit — every insert-if-not-exists pattern makes them —
and afterwards nothing on that connection worked.
The docstring on the parameterised path already described this hazard
and guarded the parameter-ENCODING failure; it never covered EXECUTE
itself failing. Fixed in all three paths (DML, parameterised DML, and
SELECT — whose fetch loop had the same gap, and whose GC-time finalizer
only covers scrollable cursors).
Cancelling a pool acquire leaked the connection permanently.
asyncio.to_thread cannot interrupt its worker, so a task cancelled while
blocked in pool.acquire() left the worker to finish and hand back a
connection nobody owned — checked out, never returned, one slot gone per
occurrence until the pool was dead.
Not theoretical for anything serving HTTP: a client disconnect cancels
the request task, and under load those cancellations land precisely
while waiting for a connection. The pool dies one slot at a time, only
under load, and PoolTimeoutError points nowhere near the cause.
acquire() now shields the inner future and returns whatever the worker
produced if the caller went away; add_done_callback fires immediately on
an already-resolved future, so the "worker finished just before the
cancel" race is the same code path rather than a second one.
The harness covered three things the type fuzzer cannot see: fetch
batching (NFETCH is a 4096-BYTE budget, so batch edges move with row
width — counts 0..1025 across three schema widths, every fetch style
required to agree), error recovery (eight error classes, repeated, each
followed by a known-good query on the same cursor and a fresh one), and
concurrency (threads sharing one connection, pooled threads that fail
before releasing, transaction isolation across checkouts, async
cancellation). Every worker reads back a value only it supplied, so a
crossed wire fails on data even when nothing raises.
Two harness false positives worth recording: Informix silently truncates
over-long strings (a literal SQL insert does the same, so the driver
matches the server), and it rejects a bare `?` in a projection — a cast
is required.
356/356 integration on 15, 14.10 and 12.10 (was 326).
Six framing bugs had reached users across three reports. Rather than
wait for a seventh, this adds a harness built to find that class of bug.
It immediately found three more, one of which hangs the connection.
BOOLEAN could not be bound as a parameter at all. _encode_bool emitted
type code 45, which the server does not accept as a bind type — it
simply stopped responding, so any execute() with a bool parameter hung
until the read timeout, or forever without one. Type 41 (how BOOLEAN is
*described* in results) hangs identically; descriptor type and bind type
are not interchangeable. Now binds 't'/'f' as CHAR and lets the server
cast, mirroring Informix's own literal syntax. A hang is worse than
wrong data, and this shipped in every release claiming BOOLEAN support.
Unscaled DECIMAL was read one byte short, shifting every following
column. Width is (precision + (scale & 1) + 3) // 2 per
IfxColumnInfo.adjustedColumnLength; we dropped the `scale & 1` term.
That only matters when precision is even and scale is odd — and an
unscaled DECIMAL(p) reports scale 255. DECIMAL(16) is 10 bytes, not 9.
Verified on the wire for ten DECIMAL/MONEY shapes. The same formula
governs DATETIME and INTERVAL, whose qualifier parity tracks their digit
count, so those agreed by coincidence; all four now share one helper
taken from the reference.
NULL CHAR/NCHAR came back as '', indistinguishable from an empty column,
so `WHERE c IS NULL` disagreed with what the driver returned. The wire
distinguishes them plainly — NULL is a leading 0x00, empty is all
spaces.
The harness (tests/test_type_matrix.py) encodes three principles, each
derived from how a real bug escaped:
1. Always put a column after the value under test. Every bug so far
corrupted the NEXT column; a trailing value can be mis-sized
invisibly. Every case ends in a sentinel, and projections rotate.
2. Vary the data, not just the type. Branch coverage is worthless if
no value takes the branch — the LVARCHAR fixture was 'lv value',
8 chars, even, never NULL, so both broken branches sat unexecuted
through 247 tests.
3. Use an oracle our codecs don't share. A Python round-trip cannot
catch a symmetric encode/decode bug: our DATETIME encoder wrote
zeros and our decoder read them back in perfect agreement. Asking
the server to render the stored value breaks that symmetry.
Exhaustive over the corpus plus seeded random multi-type rows, so
failures reproduce.
326/326 integration on 15, 14.10 and 12.10 (was 281). Fuzzer reports
clean over 80-round runs at several seeds and widths on all three.
More data corruption from the same field report as 2026.05.08.2. Anyone
with LVARCHAR columns should upgrade: 2026.08.27 is not safe for them.
Two independent errors in the LVARCHAR envelope, both shifting every
column selected after it.
1. A phantom pad byte. We appended an even-byte pad when the value
length was odd. There is no pad. Wire capture (12.10), INT8 /
LVARCHAR / INT8:
00 01 00 00 07 d1 00 00 00 00 a = INT8 2001
00 null indicator
00 00 00 0b length 11
50 61 63 6b 61 67 65 52 6f 6f 74 "PackageRoot" (odd)
00 01 00 00 00 0a 00 00 00 00 b = INT8 10, starts immediately
2. A missing length on NULL. We returned as soon as the indicator said
NULL, leaving its 4-byte length unread. The length belongs to the
envelope and is always present:
'odd' -> 00 | 00 00 00 03 | 6f 64 64 8 bytes
NULL -> 01 | 00 00 00 00 5 bytes
'' -> 00 | 00 00 00 00 5 bytes
NULL and empty string differ only in the indicator byte.
Reported symptom: INT8 10 decoding as 2560 (the same value shifted one
byte left), strings losing their first character, and IndexError or
"INT8 payload too short" once the drift ran off the payload. The
reporter isolated it by reordering columns in the projection — right
when first, wrong when later. That's the fingerprint of positional
drift and a genuinely good diagnostic.
Missed by 247 tests because the only LVARCHAR fixture was 'lv value':
8 characters, even, never NULL. Neither faulty branch ever ran. Same
gap shape as last time — code paths covered, the data reaching them
not. New tests vary parity (0,1,2,3,11,255,256), cover NULL and empty
separately, always place a column AFTER the LVARCHAR, and rotate the
projection through every position.
Separately, DATETIME lost sub-second precision on INSERT. The encoder
emitted YEAR TO SECOND unconditionally, so binding a datetime with
microseconds into a FRACTION(n) column stored zeros, silently. Reads
were always right, so it only went missing on the way in. Now widens
to FRACTION(5) when microsecond is non-zero and keeps the original
encoding otherwise, so the well-exercised path stays byte-identical.
Binding into a narrower column still truncates server-side.
Also: server_version reported 9.56 for a 12.10 server, which reads like
a client-SDK version. The login response only carries the internal
protocol version, and documenting that didn't make the name less
misleading. server_version now returns the release via one cached
DBINFO query; server_version_internal returns the raw login string.
281/281 integration on 15, 14.10 and 12.10; 123 unit tests.
Auditing the docs site against the actual module surface — prompted by
the same field report as 2026.05.08.2 — turned up a partly fictional API.
Every claim below was checked by calling it, not by reading the source.
Documented but nonexistent, now removed:
conn.transaction() AttributeError (neither sync nor async)
conn.autocommit never existed; set via connect(autocommit=)
cursor.lastrowid never existed
cursor.read_clob_column never existed
cursor.write_clob_column never existed
A copy-pasteable example in reference/types.md raised TypeError:
IntervalYM takes a single total month count, not (years=, months=).
False descriptions corrected: RowValue was described as "named-tuple-like
with .name-accessible fields" and CollectionValue as "iterable" with
"indexed access". Neither is true — both are opaque wrappers over raw
bytes plus a schema string. The docs now say so and point at SQL
projection as the way to get fields today.
Replacements are the real idioms, taken from passing tests:
CLOBs -> write_blob_column(..., clob=True); read returns bytes
SERIAL value -> SELECT DBINFO('sqlca.sqlerrd1')
transactions -> commit()/rollback() in try/except
Also added: INT8/SERIAL8/BIGSERIAL to the type table (with why INT8 is
not BIGINT), server_capabilities + server_version, the scrollable-cursor
methods, and the note that server_version reports the internal protocol
version (12.10 says 9.56, 14.10 says 9.59).
Updated the test-count claim to 400+ across three server versions,
replacing "integration tests run against 15.0.1.0.3DE".
Added a checker pass over every conn.*/cursor.* reference in the docs;
the only remaining unresolved names are deliberate mentions of IfxPy
APIs we don't implement. Site builds clean.
pyproject has carried the "Typing :: Typed" classifier without shipping
the marker file, so type checkers ignored the inline annotations
entirely and the classifier was a false claim. The annotations were
always there; only the marker was missing.
Caught during the pre-publish audit.
The lockfile still carried the pre-rename name and a stale version
(informix-db 2026.5.5.9). Same class of miss as the __version__ bug:
the rename touched pyproject.toml but not everything that records the
distribution name.
Closes the last open item from the Informix 12 field report. The driver
hardcodes several wire-framing choices that SQLI actually negotiates.
Those choices are correct on every server we've measured, but "correct
as far as we know" and "checked" are different things, and a framing
mismatch corrupts rows silently.
We were already sending SQ_PROTOCOLS with the same 8-byte client offer
IBM's JDBC driver uses, and discarding the reply. Now we decode it.
New informix_db.ServerCapabilities, reachable from any connection:
conn.server_capabilities.four_byte_offset
conn.server_capabilities.varchar_var_len
conn.server_capabilities.violated_assumptions() # [] when we agree
conn.server_version
violated_assumptions() names each place we emit or parse a fixed wire
shape that is actually capability-gated. Non-empty at connect time logs
a warning naming the bit, so an untested server produces a diagnosable
complaint instead of quiet corruption.
Nothing branches on these bits yet — this is observation and validation
only. The hardcoded framing is correct on all three supported servers,
and rewriting working parse paths to be conditional without a server
that needs it trades certainty for risk.
The measurement, and why 12/14/15 are interchangeable:
15.0.1.0.3 bdbe9ffe7fb7ffef ff
14.10.FC7W1 bdbe9ffe7fb7ffef f8
12.10.FC12W1DE bdbe9ffe7fb7ffef f0
^^^^^^^^^^^^^^^^ identical
The first 64 bits are byte-identical. Those releases don't merely behave
alike, they negotiate exactly the same capability set.
Two details worth recording. The reply is NINE bytes; JDBC's
enhancedProtocolMechanism switches on case 0..7 and drops the ninth, so
its BitSet(64) never sees it — yet that dropped byte is the only part
that differs between releases. And Cap_1 in the login response is not a
server version, it's the client's declared protocol level echoed back,
which is why JDBC tests == 316 rather than >=. The version string there
is the internal one: 12.10 reports 9.56, 14.10 reports 9.59. At the
protocol level both really are 9.x servers.
This also retires isUSVER as a red herring: it's one of six bits JDBC
pre-sets for any non-zero Cap_1, and Java's BitSet.set never clears, so
it is true on every modern server regardless of the mask.
Separately, fixed __version__. The distribution was renamed informix-db
-> informix-driver on 2026-05-08 but __init__ kept looking up the old
name. importlib.metadata needs the distribution name, not the module
name, and the miss fails silently — so every install of the renamed
package has reported "0.0.0+local". It escaped notice because a stale
informix-db distribution lingered in the dev venv and answered the
query. tests/test_package_metadata.py now pins the name against
[project].name; verified it catches the bug by reintroducing it.
247/247 integration on 15, 14.10, and 12.10. 120 unit tests.
Closes out the field report that prompted 2026.05.08.2. Full integration
suite, same commit, all green:
Informix 15.0.1.0.3DE 241 / 241
Informix 14.10.FC7W1DE 241 / 241
Informix 12.10.FC12W1DE 241 / 241
Including smart-LOB. The 28-type round-trip produces byte-identical wire
output on all three — same type codes, same encoded lengths, same values.
There is no 12-vs-14-vs-15 wire difference for anything we support.
Testing three versions by hand was tedious and undocumented, which is
part of why it never happened. Now:
make ifx-legacy-up 12.10 on 9089, 14.10 on 9090, alongside 15 on 9088
make ifx-legacy-setup blobspace1 + sbspace1 in both
make test-matrix the suite against all three
tests/setup-spaces.sh absorbs the per-image differences that made this
annoying: 12.10/14.10 use a flat INFORMIXDIR while 15 nests a versioned
subdirectory; ONCONFIG is named differently across images; `bash -lc`
wipes INFORMIXDIR and makes every utility fail with a misleading
"Unable to read $INFORMIXDIR (/usr/informix)"; and 12.10 DE ships no
ontape at all (the level-0 archive turns out to be unnecessary anyway).
Without blobspace1/sbspace1, ~21 tests fail with errors that look like
driver bugs and aren't.
The documented workflow was validated from scratch — containers removed,
recreated via compose, spaces created via the script, matrix run — rather
than written up afterward from whichever commands happened to work. The
README claim this replaces was wrong precisely because nobody did that.
Data corruption affecting every Informix version including 15. Two of the
three silently corrupt columns AFTER the offending one, so the damage
shows up far from its cause.
BOOLEAN: described as UDTFIXED(41) with encoded_length=1, but carries the
full UDT envelope on the wire — [1-byte indicator][4-byte length][data],
six bytes for a one-byte value. We read one byte and left five behind.
Captured payload for (INT, BOOLEAN 't', INT, VARCHAR 'tail'):
00 01 b2 07 | 00 00 00 00 01 74 | 00 03 64 0e | 04 74 61 69 6c
Before: (111111, b'\x00', 1, '\x00\x03d\x0e\x04tail')
After: (111111, True, 222222, 'tail')
NCHAR: fixed-width and space-padded like CHAR, but we had it grouped with
the byte-length-prefixed types. NCHAR(10) holding 'nch' read 0x6E ('n') as
a 110-byte length. Silent truncation alone, struct.error with any column
following. NVARCHAR really is length-prefixed and is unchanged; there's a
test guarding both sides now.
INT8/SERIAL8: absent from FIXED_WIDTHS, fell through to the unknown-type
path and surfaced as raw bytes. INT8 is not BIGINT — 10 bytes,
sign-magnitude, halves stored high-last:
bytes 0-1 sign word (0=NULL, 1=pos, 0xFFFF=neg)
bytes 2-5 LOW 32 bits, bytes 6-9 HIGH 32 bits
+n and -n share magnitude bytes, so a two's-complement read is wrong for
every negative while looking correct for every positive.
Not a version bug. Reported against Informix 12, but the same 28-type
round-trip against 12.10.FC12W1DE and 15.0.1.0.3DE produced byte-identical
wire output. It read as version-specific only because INT8/SERIAL8 dominate
12-era schemas while our fixtures use BIGINT.
251 tests missed all three because no fixture used these types. Added
tests/test_type_framing.py (20 integration) and tests/test_int8_unit.py
(14 unit, wire vectors from both servers). Each type is tested twice —
alone, and with trailing columns, since only the latter catches desync.
Verified: 271/271 integration on 15; 241/241 non-smart-LOB on 12.10 (the
LOB tests need an sbspace that image lacks); 20/20 framing tests on both.
Also corrected README and _fastpath docstring, which asserted 12.10
compatibility that had never been tested. The claims held up, but they
were guesses when written and cost a user debugging time.
Previously the horizontal-scroll fallback only applied at ≤640px; between
641px and 799px (tablet portrait, narrow desktop 2-col), the hero stayed
2-column so the wire-dump column could still be too narrow for the
~564px hex content, and overflow: hidden silently clipped the right side.
Lifting overflow-x: auto to all widths means: (a) any width where the
column is wider than the hex, content displays normally with no scroll;
(b) any width where the column is narrower, content becomes
horizontally-scrollable inside the dump. y stays hidden to keep the
typed-out animation's unrevealed lines clipped below the fold.
Root cause: hero's grid items inherited min-width: auto, so the wire-dump's
white-space: pre hex lines (~564px wide content) forced the hero column to
that width, propagating up to the page and causing 193px of horizontal
overflow at narrow viewports.
Fix:
- min-width: 0 on .ifx-hero, .ifx-hero__copy, .ifx-hero__visual (lets
grid items shrink below content's intrinsic min-width)
- overflow-x: auto on .ifx-wiredump (contains residual hex overflow
inside the dump, not on the page)
- Font shrunk 0.78rem → 0.62rem on mobile, ASCII column hidden, padding
tightened — readable hex without horizontal scroll inside the dump
- Eyebrow flex-wrap: wrap so the No-libcrypt suffix wraps cleanly
- Title floor 2rem → 1.75rem at 7.5vw for narrow screens
- Tightened CTA padding, install command word-break: break-all
Sweep all backticked + bold + table-cell + heading mentions of the
project's brand to match the PyPI distribution name and the docs domain.
Path references (`cd informix-db`, `git.supported.systems/.../informix-db`)
stay — those reference the actual Gitea repo directory which we did NOT
rename. Same with `import informix_db` (Python module name, separate
from distribution brand).
Also flip GitHub references to Gitea throughout the docs site:
- `github.com/rsp2k/informix-db/blob/main/X` → Gitea `/src/branch/main/X`
- `github.com/rsp2k/informix-db/tree/main/X` → Gitea same path
- `github.com/rsp2k/informix-db` (plain) → Gitea
- Hero "GitHub" CTA button → Gitea source URL
- Social icon: `github` → `seti:git` (generic git icon, not octocat)
Net result: zero stale GitHub references, brand consistency matches what
users `pip install`.
Match what users see when they `pip install informix-driver`. Also fix
two broken social links: PyPI was pointing at the now-404 informix-db
project, GitHub link was pointing at github.com/rsp2k/informix-db
(repo doesn't exist there — source lives on Gitea). Rewired to:
- Source (Gitea): git.supported.systems/warehack.ing/informix-db
- PyPI: pypi.org/project/informix-driver
- editLink: Gitea's _edit/branch/main URL pattern
Logo SVG aria-label + title also updated for accessibility consistency.
Brand-consistency with PyPI distribution name. Old domain stops serving
(no redirect — clean cutover per the rename decision).
- pyproject.toml urls: Homepage + Documentation → new domain
- README badge row + sdist exclude comment
- docs-site .env.example + astro.config.mjs site URL + DEV_DOMAIN default
- Version bumped to 2026.05.08.1 (PEP 440 post-release; PyPI 2026.5.8
still resolvable but with stale URL — yank or leave per preference)
Closes the bulk-fetch gap to within ~7-15% of IfxPy. Lever was the
buffer/I/O machinery, not the codec — Phase 37/38 had already brought
the codec close to IfxPy's C path; the remaining gap was ~450k
read_exact calls per 100k-row fetch, each doing its own recv-loop
and bytes.join.
Architecture: IfxSocket owns a connection-scoped bytearray + integer
offset cursor; BufferedSocketReader is a thin parser-view that delegates
buffer-fill to the socket. One recv() per ~64 KB instead of per field.
This is how asyncpg (buffer.pyx) and psycopg3 (pq.PGconn) structure
their read paths.
The buffer MUST be socket-scoped, not reader-scoped: the pipelined-
executemany path (Phase 33) streams N responses back-to-back across
multiple cursor reads, and a per-reader buffer would throw away
pre-fetched bytes when one reader is destroyed. (The first iteration
of this phase tried per-reader and hung on test_executemany_1000_rows.)
A/B vs Phase 38, same harness, warmed cache:
select_scaling_1000 2.90 -> 1.72 ms (-41%)
select_scaling_10000 24.32 -> 16.08 ms (-34%)
select_scaling_100000 250.36 -> 168.98 ms (-32%)
Head-to-head vs IfxPy 2.0.7:
select_scaling_1000 1.05x (basically tied)
select_scaling_10000 1.07x
select_scaling_100000 1.15x
IQR collapsed 9x at 100k (3.6 ms -> 0.4 ms) — fewer recvs means fewer
scheduler/jitter pulses showing up in the measurement.
Default ON. Set IFX_BUFFERED_READER=0 to fall back to the legacy reader
(still tested in CI as the escape hatch). Both paths green: 251/251
integration tests pass on each.
Generates a specialized row-decoder function per result-set shape via
exec(compile(src, ...)) and inlines the common fixed-width decode bodies
directly into the generated source — closing more of the C-vs-Python
codec gap on bulk fetch.
For SMALLINT/INT/SERIAL/BIGINT/BIGSERIAL/FLOAT/SMFLOAT/DATE the decode
body is inlined ("v0 = _UNPACK_INT(raw)[0]; if v0 == sentinel: v0 = None")
rather than called, eliminating one Python function call per such column
per row. BOOL deliberately left to its canonical decoder (Informix BOOL
is 't'/'T'/1, not bool(byte)).
Real A/B vs Phase 37 (median, integration container):
select_scaling[100000] 257.66 -> 227.67 ms (-12%)
wide_row_select[20] 4.27 -> 3.63 ms (-15%)
select_scaling[10000] 25.13 -> 22.58 ms (-10%)
wide_row_select[100] 15.17 -> 13.59 ms (-10%)
Win scales with row count and column count — exactly the codegen
profile expected from per-column inlining.
Generated source is printable via IFX_DEBUG_CODEGEN=1.
Three-tier composition: codegen -> reader-list -> legacy chain;
parse_tuple_payload prefers the codegen'd decoder, falls back to the
Phase 37 readers list, falls back to the legacy branch chain.
All 251 integration tests pass.
Closes some of the C-vs-Python codec gap on bulk fetch by moving
per-column dispatch decisions from row time to parse_describe time.
Same approach psycopg3 uses in its pure-Python mode (loader cache
per column).
What changed:
_resultset.py:
* New compile_column_readers(columns) builds a per-column dispatch
tuple at parse_describe time. Each tuple is (kind, *args) where
kind is a small int (FIXED/BYTE_PREFIX/CHAR/LVARCHAR/DECIMAL/
DATETIME/INTERVAL/LEGACY).
* parse_tuple_payload accepts optional readers= parameter. Fast
path uses int comparison + tuple unpack instead of the legacy
frozenset/dict-lookup chain.
* _legacy_dispatch_one_column factored out to handle rare types
(UDT/composite/UDTVAR) that fall through.
cursors.py:
* Cursor caches self._column_readers after parse_describe,
computed once via compile_column_readers. Reset on new execute.
* Fetch loop passes readers=self._column_readers.
Performance (median of 10+ rounds):
select_scaling[1000]: 2.7 ms -> 2.51 ms (-7%)
select_scaling[10000]: 25.8 ms -> 25.0 ms (-3%)
select_scaling[100000]: 271 ms -> 246 ms (-9%)
wide_row_select[5]: 2.4 ms -> 2.16 ms (-10%)
wide_row_select[20]: 5.1 ms -> 4.14 ms (-19%)
wide_row_select[50]: 10.1 ms -> 8.21 ms (-19%)
wide_row_select[100]: 19.4 ms -> 14.6 ms (-25%)
Wide-row workloads benefit most - per-column dispatch savings
accumulate linearly with column count. At 100 cols, 25% speedup.
IfxPy gap shrinks from ~2.4x to ~2.2x on bulk fetch. Real progress
but not closing-the-gap. Next lever is exec()-based codegen
(per-result-set decoder function) - possible Phase 38.
221 integration tests still pass. Benchmark suite acts as regression
test.
Architectural note: chose tuple dispatch (r[0] int compare) over
object-method dispatch (loader.load(data)) for ~20-30 ns/col speed
advantage in the inner loop. Slightly less extensible than psycopg3's
class-based loaders but materially faster in pure Python.
Adds three things to test_scaling_perf.py:
1. 100-column wide-row SELECT - codec stress test at extreme widths.
1k rows x 100 cols = 19.4 ms (~194 us/row, ~1.94 us/column-decode).
Per-column cost continues to drop with width thanks to loop
amortization (5 cols: 480 ns/col -> 100 cols: 194 ns/col).
2. 100k-row memory profile - samples RSS pre-execute, post-execute
(materialization cost), and during iteration. Real numbers:
pre-execute: 45.8 MB
post-execute: 71.2 MB (+25.4 MB = ~259 bytes/row materialization)
iteration: 0 KB extra (just walks the existing list)
Documents the in-memory cursor's actual cost: 100k rows = 25 MB,
1M rows = ~250 MB. Fair regression baseline (tripped at 500 MB).
3. 1M-row scaling gated behind IFX_BENCH_1M=1 env var. Default off
because the dev container's rootdbs runs out of space. For
production-sized servers users can opt in. The implementation
is linear-extrapolation-correct (executemany 100k -> 1M = ~15s,
SELECT 100k -> 1M = ~3s).
Note on the dev-container size limit: dev image's rootdbs is sized
for typical developer workloads, not stress testing. A 1M-row
INSERT exceeds the available pages and fails with -242 ISAM -113
(out of space). This is correct behavior - the limit is enforced
at the storage layer.
Switched RSS sampling from ru_maxrss (peak, monotonic) to
/proc/self/status VmRSS (current). Earlier runs showed flat
RSS because peak from earlier in the test session masked the
fluctuation.
Extends the IfxPy comparison bench script with scaling workloads
(1k/10k/100k rows for both executemany and SELECT). Re-runs the
full comparison with consistent measurement methodology and updates
the README with the actually-correct numbers.
Earlier comparison runs reported informix-db winning all 5
benchmarks. Re-running select_bench_table_all with consistent
measurement gives 3.04 ms, not the 891 us I cited earlier - a
3.4x discrepancy attributable to noisy warmup + small-fixture
artifacts. The "we win everything" framing was wrong.
Corrected comparison reveals two clear stories:
Bulk-insert: pure-Python wins 1.6x at scale.
executemany(10k): IfxPy 259ms -> us 161ms (1.6x faster)
executemany(100k): IfxPy 2376ms -> us 1487ms (1.6x faster)
Reason: Phase 33's pipelining eliminates per-row RTT. IfxPy's
per-call API can't pipeline.
Large-fetch: IfxPy wins 2.3-2.4x at scale.
SELECT 1k rows: IfxPy 1.2ms / us 2.7ms (IfxPy 2.3x)
SELECT 10k rows: IfxPy 11.3ms / us 25.8ms (IfxPy 2.3x)
SELECT 100k rows: IfxPy 112ms / us 271ms (IfxPy 2.4x)
Reason: C-level fetch_tuple at ~1.1us/row beats Python
parse_tuple_payload at ~2.7us/row. Real C-vs-Python codec gap
showing up at scale.
For everyday workloads (single SELECT in a request, INSERT a
handful of rows), drivers are within 5-25%. For workloads where
the gap widens, direction depends on what you're doing - bulk-
write favors us, bulk-read favors IfxPy.
README's "Compared to IfxPy" section rewritten with the corrected
numbers and an honest "when to prefer which" subsection.
tests/benchmarks/compare/README.md mirror updated.
Net narrative: a "faster at bulk-write, slower at bulk-read,
comparable elsewhere" comparison story is more honest and more
durable than a "we win everything" claim that would have collapsed
the first time a user ran their own benchmark.
Side note (lint): one ambiguous unicode `×` in cursors.py replaced
with `x`.
Phase 37 ticket: parse_tuple_payload is the bottleneck at scale.
Closing the 1.6 us/row gap to IfxPy would make us competitive on
bulk-fetch too. Possible approaches: Cython codec, deeper inlining,
per-column dispatch pre-bake.
DATA-LOSS BUG: cursor.fetchall() on result sets larger than ~200 rows
was silently truncating to the first ~200 rows. The exact cap depended
on row width and the server's per-NFETCH buffer (4096 bytes default).
The bug:
_execute_select sent NFETCH twice and stopped:
self._conn._send_pdu(self._build_curname_nfetch_pdu(cursor_name))
self._read_fetch_response()
self._conn._send_pdu(self._build_nfetch_pdu()) # comment: "DONE only"
self._read_fetch_response()
# then CLOSE+RELEASE — discarding remaining queued rows
The "second fetch returns DONE only" comment was wrong. For any
result set larger than the server's per-NFETCH batch, the second
fetch returns more tuples AND there are still tuples queued
server-side. The cursor closed and dropped them.
Latent for 30 phases because every existing test used either a small
result set (FIRST 10) or relied on row counts that fit naturally in
1-2 batches. Discovered by Phase 34's scaling benchmark when
SELECT FIRST 100000 from a 100k-row table returned 200 rows.
The fix: loop NFETCH until a response yields zero new tuples.
self._conn._send_pdu(self._build_curname_nfetch_pdu(cursor_name))
rows_before = len(self._rows)
self._read_fetch_response()
rows_received = len(self._rows) - rows_before
while rows_received > 0:
self._conn._send_pdu(self._build_nfetch_pdu())
rows_before = len(self._rows)
self._read_fetch_response()
rows_received = len(self._rows) - rows_before
249 integration tests pass. The scaling benchmark suite (Phase 34,
shipping next) is the regression test going forward.
Workaround for users on older versions: use scrollable cursors
(cursor(scrollable=True)) which use the SQ_SFETCH protocol path
and don't have this bug.
If you've been using this driver for queries returning large result
sets, your queries may have been truncating silently. Re-run them
against 2026.05.05.7+ to verify your data.
The serial-loop executemany paid one wire round-trip per row (~30us/
row on loopback). It was the one benchmark where IfxPy beat us in
the comparison work - 10% slower at executemany(1000) in txn.
Phase 33 pipelines the BIND+EXECUTE PDUs: build all N PDUs, send
them back-to-back, then drain all N responses. Eliminates per-row
RTT entirely.
Performance impact:
* executemany(1000) in txn: 31.3 ms -> 11.0 ms (2.85x faster)
* executemany(100) autocommit: 173 ms -> 154 ms (11% faster)
* executemany(1000) autocommit: 1740 ms -> 1590 ms (9% faster)
(Autocommit gets smaller wins because server-side log flushes
dominate - Phase 21.1's "autocommit cliff".)
IfxPy comparison flipped: us 10% slower -> us 2.05x faster on bulk
inserts. We now win all 5 head-to-head benchmarks against the C-bound
driver.
Margaret Hamilton review surfaced one CRITICAL concern (C1) - the
pipeline assumes Informix sends N responses for N pipelined PDUs
even when one fails. If the server cut the stream short, the drain
loop would deadlock on the next read.
Verified by 3 new integration tests in tests/test_executemany_pipeline.py:
* test_pipelined_executemany_mid_batch_constraint_violation (row 500/1000)
* test_pipelined_executemany_first_row_fails (row 0/100)
* test_pipelined_executemany_last_row_fails (row 99/100)
All confirm Informix sends N responses; wire stays aligned; connection
is usable after.
Plus 4 lower-priority fixes Hamilton recommended:
* H1: documented _raise_sq_err self-drains-SQ_EOT invariant + tripwire
* H2: docstring warning about O(N) lock duration; chunk for huge batches
* M1: prepend row-index to exception message rather than reformat
* M2: documented sendall-no-timeout caveat on hostile networks
77 unit + 239 integration + 33 benchmark = 349 tests; ruff clean.
Note: Phase 32 (Tier 1+2 benchmarks) was tagged without bumping
pyproject.toml's version string. .5 was git-tag-only; .6 is the next
published version increment.
Tier 1 — make existing benchmarks reliable:
* Bumped slow-bench rounds: cold_connect_disconnect 5->15, executemany
series 3->10. Single-round outliers no longer dominate.
* Switched bench reporting to median + IQR. Mean was being moved by
individual GC pauses / scheduler hiccups (IfxPy executemany IQR
was 8.2 ms on a 28 ms median - 29% spread - mean was unreliable).
* Updated ifxpy_bench.py to also report median + IQR alongside mean
for cross-comparable numbers.
* Makefile bench targets now show median, iqr, mean, stddev, ops, rounds.
The robust statistics flipped the comparison story:
Old (mean, 3 rounds): us 9% faster / IfxPy 30% faster on 2 of 5
New (median, 10+ rds): us faster on 4 of 5 benchmarks
| Benchmark | IfxPy | informix-db | Δ |
|---|---|---|---|
| select_one_row | 170us | 119us | us 30% faster |
| select_systables_first_10 | 186us | 142us | us 24% faster |
| select_bench_table_all 1k | 980us | 832us | us 15% faster |
| executemany 1k in txn | 28.3ms | 31.3ms | us 10% slower |
| cold_connect_disconnect | 12.0ms | 10.7ms | us 11% faster |
Tier 2 — add benchmarks for claims we make but don't verify:
tests/benchmarks/test_observability_perf.py:
* test_streaming_fetch_memory_profile — RSS sampling during a
cursor iteration. Documents memory growth shape; regression
wall at 100 MB / 1k rows. Currently flat (in-memory cursor
doesn't grow detectably for 278 rows).
* test_select_1_latency_percentiles — 1000-query distribution
with p50/p90/p95/p99/max. Result: p99/p50 = 1.42x (tight tail).
p50=108us, p99=153us.
* test_concurrent_pool_throughput[2,4,8] — N worker threads
through pool, measures aggregate QPS + per-thread fairness.
Plateaus at ~6K QPS (server-bound); per-thread latency scales
~linearly with N (server serialization expected).
README.md (project root): updated Compared-to-IfxPy table with
the median-based numbers + IQR awareness note.
tests/benchmarks/compare/README.md: added "Statistical robustness"
section explaining why median over mean for fair comparison.
236 integration tests pass; ruff clean.
