Compare commits

..

7 Commits

Author SHA1 Message Date
c85aaf2ecb Docs: put named row access where someone leaving IfxPy will find it
The API reference gained the Row section with the release, which is the
right home for the details and the wrong place to discover the feature
exists. Three gaps closed.

The IfxPy migration guide is the page for people in exactly the position
that prompted this: IfxPy is positional-only, so those codebases grow a
description/zip helper that yields a dict and still no attribute access.
That page now shows the helper it replaces alongside the one-liner, and
notes the .lower() in the hand-written version is already a no-op since
Informix folds unquoted identifiers.

Quickstart gains one sentence, next to the paramstyle note where someone
is already thinking about how rows come back.

The phase log records 2026.09.03, including where the testing actually
found the shadowed-names bug: not in the differential test across twenty
awkward types, which found nothing, but in six lines of name bookkeeping
that were correct when written and stopped being correct an hour later.

Verified live by content, and the #rows anchor the new cross-links point
at resolves.
2026-09-03 16:06:34 -06:00
acc21b8b81 Docs: row_factory reference, and the write-path corruption warning
The API page documents Row: the three access styles, that it is set on
the connection, that it subclasses tuple so existing code is unaffected,
the measured 9% and why that makes it opt-in, and the four behaviours
worth knowing (lower-case folding, expression column names, duplicate
names, and a column beating a method of the same name).

The types page gains the placeholder-rewriter entry. It is the only one
in that list affecting the write path rather than the read path, so
anyone who has stored HH:MM times or URLs with ports through this driver
should check what actually landed.

Test counts updated to 472/471. Verified live by content.
2026-09-03 16:00:53 -06:00
9fbb97199f Release 2026.09.03 — placeholder rewriter, and named row access
The rewriter that converts :N placeholders to ? was a bare regular
expression, so it rewrote the inside of string literals. Any HH:MM time,
any URL with a port, any key:value string was silently stored wrong. It
was not opt-in either: the rewrite runs whenever a statement has
parameters, whatever placeholder style the caller used. Replaced with a
scanner that substitutes only outside quotes and comments, using
Informix's own lexical rules measured against all three servers.

Row adds column-name and attribute access alongside the existing
positional access, on request from a user coming from pyodbc and
mssql-python. Opt-in at connect time, because it costs about 9% on bulk
fetch and that is not a trade to make for someone who never reads a
column by name.

472 tests on 15 and 14.10, 471 on 12.10.
2026-09-03 15:59:26 -06:00
ce5a582048 Testing Row found the same mistake I keep making
I hand-listed the names a column could shadow: count and index, the two
tuple methods. Then I wrote keys(), _asdict() and _fields on the same
class and did not go back. A column called "keys" returned a bound
method instead of a value, silently, which is precisely the failure
shape this driver spent a fortnight removing from its decoders.

The fix is not a longer list. The reserved set is now computed from
dir(Row), so adding a method later cannot reopen the hole, and the test
is parametrized over that computed set so it grows with the class.

_fields and _map moved to name-mangled attributes. They are read by
repr() and _asdict(), so shadowing the public _fields with a column
would have made the machinery report the column value instead of the
column names. Mangling keeps the two apart, and there is a test that
shadows _fields and checks repr still works.

Also exercised, and all clean: value-identical output against the plain
tuple path across twenty columns covering every awkward type plus a
fully NULL row, which is the strongest available statement that Row is a
presentation layer and not a bug; zero-column and 500-column rows;
unicode, spaced, digit-leading, empty-string and dunder column names;
rows outliving eviction of their class from the bounded cache; the four
async fetch routes and the async pool; and twelve threads racing on the
class cache, which correctly share one class per shape.

472 tests on 15 and 14.10, 471 on 12.10.
2026-09-03 15:51:46 -06:00
805fa58eb2 Rows can answer to a column name, if you ask for it
Field request from a user running this alongside SQL Server, where both
pyodbc and mssql-python hand back rows that take a position, a column
name, and an attribute. The argument is readability on wide
projections: row[11] tells a reader nothing, and stays correct only
until somebody adds a column in the middle.

row_factory=Row gives all three. It is set on the connection, so it is
one line for the whole application rather than per query, which is what
"without any additional coding" has to mean in practice. A cursor can
override it.

Opt-in, not default, and the number is why. Measured on a 20,000-row
five-column fetch: 37.2 ms with tuples, 40.8 ms with Row, about 9%.
Defaulting it on would move the published 1.05-1.15x ratio against IfxPy
to roughly 1.15-1.25x. That is not a trade to make on behalf of somebody
running a bulk export that never looks at a column by name.

Row subclasses tuple, so row == (1, "x") still holds. That constraint
shaped the design more than anything else: this suite compares fetched
rows against plain tuples in hundreds of places, and so does everybody's
code. Slices degrade to plain tuples, since a slice has no column map.

Details that had to be decided rather than assumed, all measured against
the servers: Informix folds unquoted identifiers to lower case, so the
.lower() in the workaround people write by hand is a no-op. Expression
columns get names like "(count(*))" that cannot be attributes, so they
are subscript-only. Duplicate names resolve to the first occurrence,
matching pyodbc. And tuple already defines count and index, so a column
with either name gets a descriptor and wins, because on a database row
that is plainly what the caller meant.

The per-shape class is cached, so a type() call does not land on every
small query, and rows pickle by rebuilding from their field names.
2026-09-03 15:42:56 -06:00
088171325d The placeholder rewriter could not see a string literal
We advertise paramstyle="numeric" and the wire protocol takes ?, so :N
placeholders are rewritten on the way out. That was a bare
re.sub(r":(\d+)", "?", sql), which rewrote the inside of string literals:

    UPDATE jobs SET url = 'http://host:8080/x' WHERE id = ?
                      stored as 'http://host?/x'

Any HH:MM time, any URL with a port, any aspect ratio, any key:value
string. It wrote wrong data and said nothing about it.

None of it was opt-in. The rewrite fires whenever a statement has
parameters, whatever placeholder style the caller actually used, so
writing ? everywhere and never touching numeric style did not protect
you. It also changed the placeholder count while num_qmarks was still
computed from len(params), leaving driver and server disagreeing about
how many binds exist.

Replaced with a single pass that substitutes only outside quotes and
comments. The lexical rules are Informix's own, measured against all
three servers rather than assumed from standard SQL, and one of them
would have been got wrong by habit: a backslash escapes nothing, so
'a\'b' is an unterminated string and draws -282. A scanner written to
Postgres reflexes would desync there and corrupt everything after it.
Block comments do not nest, the first */ closes them. Braces are a
comment. :: is stepped over as a unit so a cast can never be read as the
start of a placeholder.

An unterminated quote or comment consumes the rest and substitutes
nothing further. Under-substituting leaves the server to reject SQL that
was already malformed; guessing would corrupt a literal.

This was the last place the driver inferred meaning from SQL text
instead of handling it properly. Eleven of the new tests fail against
the previous commit.
2026-09-03 12:45:58 -06:00
7ffc148112 Docs: cover the 2026.09.02 review, and drop every em-dash
Content. The types page now warns about the two framing bugs this
release fixed, since both silently corrupt data on versions before it:
a BLOB or CLOB in any position but last shifted every column after it,
and a NULL collection did the same. The API page documents two things
that are now user-visible: only one scrollable cursor may be open per
connection (previously the second statement drew -285 and destroyed the
cursor as collateral), and transaction control written as SQL is tracked
(previously rollback() after a SQL BEGIN WORK sent nothing and reported
success). The phase log gains a section on the review itself, including
the pattern behind almost all eleven bugs. Test counts updated to
457/456.

Style. 136 em-dashes across 22 content files, plus Hero.astro and two
stylesheets, rewritten rather than substituted -- swapping the character
for a comma leaves prose that reads like it lost an argument with a
linter. Bullets that used a dash to gloss a term now use a colon;
parenthetical asides became their own sentences or moved inside
brackets. The rendered HTML is clean.

Verified against the live site by content, not status code: these pages
return 200 for every path and render the 404 body, so a stale deploy
looks perfectly healthy.
2026-09-02 10:36:44 -06:00
35 changed files with 1386 additions and 148 deletions

View File

@ -2,6 +2,50 @@
All notable changes to `informix-db`. Versioning is [CalVer](https://calver.org/) — `YYYY.MM.DD` for date-based releases, `YYYY.MM.DD.N` for same-day post-releases per PEP 440.
## 2026.09.03 — A placeholder rewriter that could not see a string literal, and named row access
### Upgrade if you write string literals containing a colon
The driver rewrites `:1` placeholders to the `?` the wire protocol takes. That rewrite was a bare regular expression, and a regular expression cannot see a string literal, so it rewrote the inside of one:
```sql
UPDATE jobs SET url = 'http://host:8080/x' WHERE id = ?
-- stored as 'http://host?/x'
```
Any `HH:MM` time, any URL with a port, any aspect ratio, any `key:value` string. It wrote wrong data and said nothing about it.
Two things made it wider than it looks. It was never opt-in: the rewrite runs whenever a statement has parameters, whatever placeholder style you actually used, so writing `?` everywhere and never touching numeric style gave no protection. And it changed the placeholder *count* while the driver still told the server there were `len(params)` of them, leaving the two disagreeing about how many binds exist.
It is now a single pass that substitutes only outside quotes and comments. The lexical rules are Informix's own, measured against all three servers rather than assumed from standard SQL, and one of them would have been got wrong from habit: a backslash escapes nothing, so `'a\'b'` is an unterminated string and draws `-282`. A scanner written to Postgres reflexes would have desynced there and corrupted everything after it. Block comments do not nest, the first `*/` closes them. Braces are a comment. `::` is stepped over as a unit so a cast can never be read as the start of a placeholder.
An unterminated quote or comment consumes the rest and substitutes nothing further. Under-substituting hands malformed SQL to the server to reject; guessing would corrupt a literal.
This was the last place the driver inferred meaning from SQL text instead of handling it properly.
### Rows that answer to a column name
Requested by a user running this alongside SQL Server, where both `pyodbc` and `mssql-python` hand back rows addressable three ways at once. The argument is readability on a wide projection: `row[11]` tells a reader nothing, and stays correct only until somebody adds a column in the middle.
```python
conn = informix_db.connect(..., row_factory=informix_db.Row)
cur.execute("SELECT tabid, tabname FROM systables")
row = cur.fetchone()
row[0], row["tabname"], row.tabname
```
Set on the connection, so it is one line for an application rather than per query. A cursor can override it. Pools and the async API forward it unchanged.
`Row` subclasses `tuple`, so `row == (1, "x")` is still true and code that treats rows as sequences keeps working. Slices degrade to plain tuples, since a slice has no column map.
**It is opt-in, and here is the number.** On a 20,000-row five-column fetch: 37.2 ms with tuples, 40.8 ms with `Row`, about 9%. Supporting `row["name"]` means `__getitem__` is a Python method rather than C-level tuple indexing, which costs roughly 39 ns on every subscript. Defaulting it on would move the published 1.05-1.15x ratio against IfxPy to roughly 1.15-1.25x. That is a reasonable trade for readable application code and a bad one for a bulk export that never looks at a column by name, so it is a choice rather than a decision made on your behalf.
Details that were measured rather than assumed. Informix folds unquoted identifiers to lower case, so `SELECT Config_Key` is reachable as `row.config_key`, and the `.lower()` in the workaround people write by hand is already a no-op. Expression columns get server-generated names like `(count(*))` that cannot be attributes, so they are subscript-only. Duplicate names resolve to the first occurrence, matching `pyodbc`. And a column always beats a method of the same name: `tuple` defines `count` and `index`, `Row` adds `keys`, `_asdict` and `_fields`, and any column with one of those names wins, because otherwise it would hand back a bound method in silence. That reserved set is computed from the class rather than hand-listed, which is what caught it: the hand-listed version covered the two `tuple` methods and missed all three of `Row`'s own.
### Verified
**472** integration tests on 15.0.1.0.3DE and 14.10.FC7W1DE, **471** on 12.10.FC12W1DE (one skip, no common table expressions before 14.10), up from 457. Both changes have tests that fail against `2026.09.02`.
## 2026.09.02 — A systematic review, and eleven bugs it found
No new features. This is the result of going back over the driver looking for the *shape* of past bugs rather than for new symptoms, and it turned up more than expected — including two that silently returned wrong data and one where `rollback()` did nothing at all.

View File

@ -13,7 +13,7 @@
</h1>
<p class="ifx-hero__lede">
Every other Informix driver wraps IBM's C SDK or the JDBC JAR. We weren't into that.
So we read the protocol and wrote it ourselves — PEP 249, sync + async, pooled, TLS.
So we read the protocol and wrote it ourselves: PEP 249, sync and async, pooled, TLS.
Within 10% of IBM's own C driver on bulk fetches, <strong>1.6× faster</strong> on
bulk inserts. No compile step. No <code>LD_LIBRARY_PATH</code> ritual. No
<code>libcrypt.so.1</code> from 2018.

View File

@ -1,6 +1,6 @@
---
title: Architecture overview
description: How the layers stack — socket, framing, codec, resultset, cursor, connection, pool.
description: How the layers stack, from socket through framing, codec, resultset, cursor, connection, and pool.
sidebar:
order: 2
---
@ -28,13 +28,13 @@ The driver is six layers, each with a single responsibility, each testable in is
The lowest layer. Wraps `socket.socket` with a connection-scoped read buffer (Phase 39). One `recv(64K)` per ~64 KB of incoming data; parsers read into the buffer via `struct.unpack_from(buf, offset)` rather than slicing copies.
Everything above this layer is `bytes` and `bytearray` arithmetic no syscalls except through `IfxSocket.read_exact(n)` and `IfxSocket.write_all(buf)`.
Everything above this layer is `bytes` and `bytearray` arithmetic, with no syscalls except through `IfxSocket.read_exact(n)` and `IfxSocket.write_all(buf)`.
See [The buffered reader →](/explain/buffered-reader/) for why the buffer lives here and not on the parser.
## Protocol / PDU framing
`_protocol.py` reads and writes SQLI PDUs. Each PDU is parsed into a typed Python representation: `SqInfo`, `SqVersion`, `SqTuple`, `SqId`, etc. The framing layer doesn't know what the PDUs *mean* only how to read and write the byte shapes.
`_protocol.py` reads and writes SQLI PDUs. Each PDU is parsed into a typed Python representation: `SqInfo`, `SqVersion`, `SqTuple`, `SqId`, etc. The framing layer doesn't know what the PDUs *mean*, only how to read and write the byte shapes.
The PDU types and their fields were reverse-engineered from three sources:
@ -44,9 +44,9 @@ The PDU types and their fields were reverse-engineered from three sources:
## Codec / Per-column readers
`converters.py` and `_resultset.py` together. The codec layer maps Informix SQL types to Python types — see [SQL ↔ Python types](/reference/types/) for the full table.
`converters.py` and `_resultset.py` together. The codec layer maps Informix SQL types to Python types. See [SQL ↔ Python types](/reference/types/) for the full table.
Phase 37 introduced **per-column reader strategy**: at PREPARE time, the driver builds a list of decoder functions (one per column) keyed by SQL type. At fetch time, decoding a row is `[reader(payload) for reader in column_readers]` no per-column dispatch overhead.
Phase 37 introduced **per-column reader strategy**: at PREPARE time, the driver builds a list of decoder functions (one per column) keyed by SQL type. At fetch time, decoding a row is `[reader(payload) for reader in column_readers]`, with no per-column dispatch overhead.
Phase 38 went further with `exec()`-based codegen: for the hottest tables, the driver generates a flat decoder function with all readers inlined and dispatch decisions baked in. The generated function is the equivalent of unrolling the per-column dispatch into straight-line code.

View File

@ -1,6 +1,6 @@
---
title: Async strategy
description: Why informix-driver wraps a sync core in a thread pool instead of going fully async and what that costs.
description: Why informix-driver wraps a sync core in a thread pool instead of going fully async, and what that costs.
sidebar:
order: 4
---
@ -19,13 +19,13 @@ Three options for adding async support to a sync database driver:
2. **Thread-pool wrapping.** Keep the sync core. Wrap each public method with `loop.run_in_executor()`. ~250 lines of code, sync tests still apply, no protocol-layer changes.
3. **Dual implementations.** Maintain two parallel code paths — one sync, one async. Most code duplicated. Worst of both worlds.
3. **Dual implementations.** Maintain two parallel code paths, one sync and one async. Most code duplicated. Worst of both worlds.
We picked option 2.
## Why option 2 was the right call
For typical database workloads — request-scoped connections, mostly waiting on I/O — the practical difference between option 1 and option 2 is small:
For typical database workloads, meaning request-scoped connections that mostly wait on I/O, the practical difference between option 1 and option 2 is small:
- **Latency**: option 1 has a slight edge (no thread context switch), but the difference is dwarfed by the actual database round-trip (~80 µs LAN, ~ms WAN). For a single query, option 2 adds ~510 µs of executor overhead.
- **Throughput under concurrency**: option 1 wins when you have N coroutines on M physical cores with M < N. The thread pool needs to context-switch between threads; the async loop just runs the next coroutine. For 10100 concurrent FastAPI requests on a 4-core box, this difference is small.
@ -39,7 +39,7 @@ The honest costs:
- **One worker thread per concurrent in-flight query.** With 100 concurrent queries, you have 100 threads. This is fine for I/O-bound work (Python releases the GIL during socket reads) but doesn't scale beyond a few hundred concurrent queries on a single process.
- **Thread-pool sizing matters.** The default executor size (5 × CPU count) is fine for most workloads. For high-concurrency workloads, you may want a larger executor.
- **Cancellation requires thought.** A cancelled `await cur.execute()` cancels the coroutine, but the worker thread continues running until the syscall returns. The connection is marked dirty until then. Phase 27 made this safe — cancelled workers cannot leak onto recycled pool connections — but the underlying syscall does still complete.
- **Cancellation requires thought.** A cancelled `await cur.execute()` cancels the coroutine, but the worker thread continues running until the syscall returns. The connection is marked dirty until then. Phase 27 made this safe, in that cancelled workers cannot leak onto recycled pool connections, but the underlying syscall does still complete.
## What it doesn't cost

View File

@ -7,7 +7,7 @@ sidebar:
import { Aside } from '@astrojs/starlight/components';
The bulk-fetch gap against IfxPy stayed stubbornly at ~2× from Phase 36 through Phase 38. Two phases of codec optimization shrank it by a few percent each. Phase 39 — a connection-scoped buffered reader — closed it from 2.4× to ~1.051.15× in about thirty minutes of code plus ten minutes of architectural debugging.
The bulk-fetch gap against IfxPy stayed stubbornly at ~2× from Phase 36 through Phase 38. Two phases of codec optimization shrank it by a few percent each. Phase 39, a connection-scoped buffered reader, closed it from 2.4× to ~1.051.15× in about thirty minutes of code plus ten minutes of architectural debugging.
This page is about both the technical change and the failure mode that hid the win for two phases.
@ -26,7 +26,7 @@ The headline "I/O dominated" was true. The interesting half is the breakdown of
- Actual `recv()` syscalls: ~153 ms
- Python wrapper overhead: ~400 ms
That ~400 ms was our own buffer abstraction a `read_exact` loop that called `recv()` per fragment, reassembled fragments via `bytes.join`, and traversed two layers of cursor wrappers per call. For 100,000 rows that's **451,402 calls to `read_exact`**, each one paying Python wrapper cost the kernel didn't cause.
That ~400 ms was our own buffer abstraction: a `read_exact` loop that called `recv()` per fragment, reassembled fragments via `bytes.join`, and traversed two layers of cursor wrappers per call. For 100,000 rows that's **451,402 calls to `read_exact`**, each one paying Python wrapper cost the kernel didn't cause.
The kernel was doing maybe 2530 ms of work. The other 130 ms of the gap-vs-IfxPy was friction we had introduced ourselves.
@ -63,21 +63,21 @@ Result: **one `recv()` per ~64 KB of incoming data**, not per field.
The natural thing to call this is "BufferedSocketReader". The natural thing to do is put the bytearray on the reader. That's what I did first.
Then `test_executemany_1000_rows` hung. The kernel stack via `cat /proc/PID/wchan` said `wait_woken` — process blocked in `recv()` waiting for bytes that weren't coming.
Then `test_executemany_1000_rows` hung. The kernel stack via `cat /proc/PID/wchan` said `wait_woken`, meaning the process was blocked in `recv()` waiting for bytes that weren't coming.
The bug was foreseeable, and it was architectural rather than implementational. Phase 33's pipelined `executemany` sends N BIND+EXECUTE PDUs back-to-back and drains responses afterward. Each cursor read constructs a *new* reader instance. When my reader did `recv(64K)` and pulled in 600 bytes — 200 bytes for response 1, 400 bytes for response 2 — it consumed bytes for response 2 *and then was destroyed*. The next reader called `recv()`, the kernel buffer was empty, and we waited forever for bytes the kernel had already given to a dead reader.
The bug was foreseeable, and it was architectural rather than implementational. Phase 33's pipelined `executemany` sends N BIND+EXECUTE PDUs back-to-back and drains responses afterward. Each cursor read constructs a *new* reader instance. When my reader did `recv(64K)` and pulled in 600 bytes, 200 for response 1 and 400 for response 2, it consumed bytes belonging to response 2 *and then was destroyed*. The next reader called `recv()`, the kernel buffer was empty, and we waited forever for bytes the kernel had already given to a dead reader.
The fix moved the buffer one level down. The bytearray and offset cursor live on `IfxSocket` (the connection-scoped wrapper) — readers are short-lived parser-views, the buffer outlives them.
The fix moved the buffer one level down. The bytearray and offset cursor live on `IfxSocket`, the connection-scoped wrapper. Readers are short-lived parser-views, and the buffer outlives them.
```python
# WRONG (first pass) buffer scoped to reader
# WRONG (first pass): buffer scoped to reader
class BufferedSocketReader:
def __init__(self, sock):
self.sock = sock
self.buf = bytearray() # ← dies with the reader
self.offset = 0
# RIGHT (Phase 39) buffer scoped to connection
# RIGHT (Phase 39): buffer scoped to connection
class IfxSocket:
def __init__(self, sock):
self.sock = sock
@ -122,7 +122,7 @@ The buffered reader ships **enabled by default** in version 2026.05.05.12. To op
IFX_BUFFERED_READER=0 python my_app.py
```
The flag is read once at connection construction. Existing connections in a pool aren't affected by changing the env at runtime close and reopen the pool to flip behavior.
The flag is read once at connection construction. Existing connections in a pool aren't affected by changing the env at runtime, so close and reopen the pool to flip behavior.
<Aside type="note">
The flag exists to make A/B measurement easy. There's no expected reason to disable it in production. If you hit a workload where the buffered reader is slower, that's a bug and we'd like to know.
@ -132,9 +132,9 @@ The flag is read once at connection construction. Existing connections in a pool
The general pattern: **what's visible gets optimization attention; what's invisible gets written off as irreducible**.
The codec is visible there's a loop, a `_decode_varchar` function, a `struct.unpack` call. You can read the inner loop and reason about it. Phases 37 and 38 attacked it, both got modest wins.
The codec is visible: there's a loop, a `_decode_varchar` function, a `struct.unpack` call. You can read the inner loop and reason about it. Phases 37 and 38 attacked it, both got modest wins.
The I/O machinery looked invisible. `_socket.read_exact` is eight lines. The cursor's `_SocketReader` wrapper is twelve. The framing reads — `read_short`, `read_int`, `read_exact(payload_size)` — are one-liners. What could be slow about that?
The I/O machinery looked invisible. `_socket.read_exact` is eight lines. The cursor's `_SocketReader` wrapper is twelve. The framing reads (`read_short`, `read_int`, `read_exact(payload_size)`) are one-liners. What could be slow about that?
It was carrying ~30% of total wall time. Two phases of changelogs implicitly blamed "the protocol" for the remaining gap. The actual culprit was a few lines of `bytes.join` in a wrapper from Phase 1 that nobody had revisited.
@ -142,6 +142,6 @@ The lesson is small and easy to state: a profile turns vibes into an attack surf
## Read more
- **[Architecture overview →](/explain/architecture/)** where the buffered reader sits in the layer stack.
- **[Phase log →](/explain/phase-log/)** the full progression from Phase 1 through Phase 39+.
- **["The 156 Milliseconds I'd Been Hand-Waving About"](https://ryanmalloy.com/collaborations/the-156-milliseconds-i-d-been-hand-waving-about/)** Claude's reflection on the session in which Phase 39 shipped, including the two pushbacks that triggered the work.
- **[Architecture overview →](/explain/architecture/)**: where the buffered reader sits in the layer stack.
- **[Phase log →](/explain/phase-log/)**: the full progression from Phase 1 through Phase 39+.
- **["The 156 Milliseconds I'd Been Hand-Waving About"](https://ryanmalloy.com/collaborations/the-156-milliseconds-i-d-been-hand-waving-about/)**: Claude's reflection on the session in which Phase 39 shipped, including the two pushbacks that triggered the work.

View File

@ -43,11 +43,11 @@ The driver was built across 39+ phases, each with a focused scope and a decision
| 23 | Health checks | Pool validates idle connections before return |
| 24 | Statement caching | Per-connection prepared-statement cache |
| 25 | Fast-path call (`SQ_FPROUTINE`) | Direct UDF/SPL invocation, bypassing PREPARE |
| 26 | **CRITICAL** | Pool returned connections with open transactions — fixed |
| 26 | **CRITICAL** | Pool returned connections with open transactions (fixed) |
| 27 | **CRITICAL** | Per-connection wire lock + async cancellation safety |
| 28 | **HIGH** | `_raise_sq_err` bare-except masking wire desync — fixed |
| 29 | Cursor finalizers | Server-side resource leak on mid-fetch raise — fixed |
| 30 | Hardening pass | 5 medium-severity audit findings all closed |
| 28 | **HIGH** | `_raise_sq_err` bare-except masking wire desync (fixed) |
| 29 | Cursor finalizers | Server-side resource leak on mid-fetch raise (fixed) |
| 30 | Hardening pass | 5 medium-severity audit findings, all closed |
After Phase 30: **0 critical, 0 high, 0 medium audit findings remain.** Driver is production-ready.
@ -67,21 +67,84 @@ After Phase 30: **0 critical, 0 high, 0 medium audit findings remain.** Driver i
The Phase 3739 trajectory is documented in detail at [The buffered reader →](/explain/buffered-reader/), including the architectural mistake the first pass got wrong.
## Field reports and systematic review (2026.082026.09)
Phase 30's audit found nothing left. Then real schemas arrived, and the audit's clean bill of health turned out to be a statement about the questions it had asked rather than about the driver.
A field report from a user running Informix 12 surfaced three type-framing bugs in a single afternoon: `INT8` / `SERIAL8` never decoded at all, `NCHAR` losing its first character, and `BOOLEAN` corrupting every column after it. Fuzzing the type matrix found more, and each one had the same shape as the last.
`2026.09.02` went back over the driver looking for that shape rather than for new symptoms, and found eleven bugs.
| Area | What was wrong |
|---|---|
| Row framing | Nothing ever checked that a row consumed its own payload. Fourteen framing bugs had reached users, every one detectable for free |
| Smart LOBs | Read as a flat 72-byte field when they occupy 149, shifting every column after a non-final `BLOB` |
| Composites | A NULL collection skipped its length field, the same bug as NULL `LVARCHAR` twelve lines away |
| Transactions | `rollback()` after a SQL `BEGIN WORK` sent nothing and reported success |
| Classification | Comments, CTEs and parenthesized selects were run as DML and failed with `-260` |
| Scrollable cursors | A second statement on the connection got `-285` and destroyed the cursor too |
| Statement release | Two of eight exits leaked a failed statement, bricking the connection |
| Cursor finalizers | Cleanup could land inside another statement, and the lock probe was blind to its own thread |
| Async | The whole layer shared the process-wide thread pool, so cancellations starved it |
| Socket | Two readers on one stream agreed only by accident of how the server replies |
The pattern behind almost all of them: a hazard that was understood and guarded at the site where it was first observed, rather than at the abstraction that owned it. The guard then never travelled to its siblings. Four separate hand-written copies of the same UDT envelope, three of them wrong. Six hand-rolled copies of the same statement-release cleanup, two missing entirely.
Two of the fixes came from asking the server instead of guessing. `statement_type` and `statement_id` had both been parsed out of every DESCRIBE response into a metadata dict that nothing read, while the code that needed them inferred the answer from the first word of the SQL and got it wrong five ways.
The cheapest fix was also the most valuable. Asserting that a row decoder lands exactly on the end of its payload costs one integer comparison, and it would have caught all fourteen framing bugs at the byte where each happened rather than three releases later in somebody's result set.
Test count went from 241 to 457 across three server versions over this stretch.
### 2026.09.03
Two more, one of each kind.
The `:N` placeholder rewriter was a bare regular expression, so it
rewrote the inside of string literals: `'http://host:8080/x'` was stored
as `'http://host?/x'`. Same family as the statement-classification bug
above, and the last place the driver inferred meaning from SQL text
instead of handling it. Replaced with a scanner using Informix's own
lexical rules, measured rather than assumed. One of those rules would
have been got wrong from habit: a backslash escapes nothing, so `'a\'b'`
is an unterminated string and draws `-282`. A scanner written to
Postgres reflexes desyncs there and corrupts everything after it.
`row_factory=Row` adds column-name and attribute access, on request from
a user coming from `pyodbc`. Opt-in, because it costs about 9% on bulk
fetch and that is not a trade to make for someone who never reads a
column by name.
Testing the second one found the pattern again, this time in code
written an hour earlier. The names a column could shadow were
hand-listed as `count` and `index`, the two `tuple` methods; then `keys`,
`_asdict` and `_fields` were added to the same class and the list was
never revisited. A column called `keys` returned a bound method, in
silence. The fix was to compute the set from the class rather than
lengthen the list, and to parametrize the test over that computed set so
it grows with the class.
The interesting part is where testing found it. A differential test
across twenty columns covering every awkward type, populated and fully
NULL, found nothing. The bug was in six lines of name bookkeeping that
looked obviously correct, and had been correct when written. Anything
that must stay in sync with a class by hand eventually will not.
## Notable architectural pivots
The decision log calls out four moments where the obvious choice would have been wrong:
1. **Phase 10/11** — abandoning `SQ_FPROUTINE` + `SQ_LODATA` for `SQ_FILE` intercept. Smaller, simpler, same correctness.
2. **Phase 16** — thread-pool async instead of full async refactor. ~88% less code, same FastAPI surface.
3. **Phase 27** — adding a per-connection wire lock instead of relying on PEP 249's "don't share connections" advice. Made accidental sharing safe rather than catastrophic.
4. **Phase 39** — buffer on the connection, not on the reader. Got it wrong on the first pass; the bug surfaced as a hang on pipelined `executemany`. Fixed in ten minutes once the architectural mistake was named.
1. **Phase 10/11**: abandoning `SQ_FPROUTINE` + `SQ_LODATA` for `SQ_FILE` intercept. Smaller, simpler, same correctness.
2. **Phase 16**: thread-pool async instead of a full async refactor. ~88% less code, same FastAPI surface.
3. **Phase 27**: adding a per-connection wire lock instead of relying on PEP 249's "don't share connections" advice. Made accidental sharing safe rather than catastrophic.
4. **Phase 39**: buffer on the connection, not on the reader. Got it wrong on the first pass; the bug surfaced as a hang on pipelined `executemany`. Fixed in ten minutes once the architectural mistake was named.
## What's next
The roadmap (loose, not committed):
- **Phase 40+ (codec)**: Numpy-backed bulk decode for homogeneous columns. ~5× speedup target on analytical workloads.
- **Phase 4x (protocol)**: Optional Cython acceleration for the codec hot loop. Would compromise "pure Python" — gated behind a build flag.
- **Phase 4x (protocol)**: Optional Cython acceleration for the codec hot loop. Would compromise "pure Python", so it would sit behind a build flag.
- **Phase 5x (API)**: Native `callproc` with named parameters, IBM-specific scrollable cursor extensions for full IfxPy parity.
The phase log is updated as work lands. The repo's [`CHANGELOG.md`](https://git.supported.systems/warehack.ing/informix-db/src/branch/main/CHANGELOG.md) is the source of truth for shipped changes.

View File

@ -26,14 +26,14 @@ The order-of-magnitude intuition: pure-Python is ~2× slower than C-bound for **
The benefits are mostly deployment, not performance:
- **50 KB wheel** installable in a slim Docker image without a build toolchain.
- **No `libcrypt.so.1`** works on Arch, Fedora 35+, RHEL 9, and any modern Linux.
- **Python 3.103.14** no minor-version-specific C extension breakage. We've shipped on the day each new Python released.
- **Type annotations everywhere** `py.typed` flag, full coverage in mypy / pyright.
- **Auditable codepaths** every byte that enters or leaves a socket goes through Python code you can read. No "the C extension does it" excuses.
- **Async without `run_in_executor`** native `async def` API, FastAPI-compatible.
- **50 KB wheel**: installable in a slim Docker image without a build toolchain.
- **No `libcrypt.so.1`**: works on Arch, Fedora 35+, RHEL 9, and any modern Linux.
- **Python 3.103.14**: no minor-version-specific C extension breakage. We've shipped on the day each new Python released.
- **Type annotations everywhere**: `py.typed` flag, full coverage in mypy / pyright.
- **Auditable codepaths**: every byte that enters or leaves a socket goes through Python code you can read. No "the C extension does it" excuses.
- **Async without `run_in_executor`**: native `async def` API, FastAPI-compatible.
For most real workloads, deployment friction matters more than 2 µs/row. The 92 MB OneDB tarball, the four `LD_LIBRARY_PATH` entries, the absent `libcrypt.so.1` those costs are paid every time you deploy. The 2 µs/row codec gap is paid once per row, and only if your workload is read-heavy enough for it to dominate.
For most real workloads, deployment friction matters more than 2 µs/row. The 92 MB OneDB tarball, the four `LD_LIBRARY_PATH` entries, the absent `libcrypt.so.1`: those costs are paid every time you deploy. The 2 µs/row codec gap is paid once per row, and only if your workload is read-heavy enough for it to dominate.
## Where the ceiling sits
@ -48,7 +48,7 @@ Five fields × ~250 ns/field + ~250 ns overhead = ~1.5 µs. We're at ~2.0 µs wh
Strategies for closing further:
- **Cython / mypyc compilation.** Could shave 30-50% off the codec hot loop. Would compromise the "pure Python" claim there'd be a build step.
- **Cython / mypyc compilation.** Could shave 30-50% off the codec hot loop. Would compromise the "pure Python" claim, because there'd be a build step.
- **Bytecode optimization via `exec()`-codegen** (the Phase 38 approach). Marginal further wins; we've already extracted most of what's available.
- **Numpy-backed bulk decode** for homogeneous columns. Promising for analytical workloads. ~5× speedup possible for `SELECT col FROM huge_table` over the current per-row approach. Probably Phase 41+.
@ -58,6 +58,6 @@ For I/O-bound workloads we're already at the ceiling. The buffered reader closed
Pure-Python costs us ~515% on bulk-fetch workloads and zero (or favorable) on everything else. The deployment, async, and modern-Python wins are large and don't depend on workload.
If the codec gap matters for your case — analytical reporting against a wide table, pulling millions of rows in a single SELECT — IfxPy is probably the right tool today. If you're doing transactional or bulk-load work, FastAPI services, or any deployment where IBM's C SDK is friction, `informix-driver` is the right tool.
If the codec gap matters for your case, meaning analytical reporting against a wide table or pulling millions of rows in a single SELECT, IfxPy is probably the right tool today. If you're doing transactional or bulk-load work, FastAPI services, or any deployment where IBM's C SDK is friction, `informix-driver` is the right tool.
The driver chose the goal*first pure-socket Informix driver in any language* over the local optimum. Phase 37 onward is a sustained effort to make that choice cost as little as possible.
The driver chose the goal, *first pure-socket Informix driver in any language*, over the local optimum. Phase 37 onward is a sustained effort to make that choice cost as little as possible.

View File

@ -1,13 +1,13 @@
---
title: The SQLI wire protocol
description: A short tour of Informix's SQLI protocol — PDU framing, the handshake, statement execution, fetch.
description: A short tour of Informix's SQLI protocol, covering PDU framing, the handshake, statement execution, and fetch.
sidebar:
order: 1
---
import { Aside } from '@astrojs/starlight/components';
SQLI is Informix's wire protocol — the same protocol IBM's CSDK and JDBC driver speak. It's a binary, length-prefixed PDU stream over a single TCP connection.
SQLI is Informix's wire protocol, the same one IBM's CSDK and JDBC driver speak. It's a binary, length-prefixed PDU stream over a single TCP connection.
This page is a short tour. The byte-level reference (with hex annotations) lives in [`docs/PROTOCOL_NOTES.md`](https://git.supported.systems/warehack.ing/informix-db/src/branch/main/docs/PROTOCOL_NOTES.md) in the repo.
@ -71,14 +71,14 @@ The full lifecycle for `SELECT id FROM users WHERE id = ?`:
← SQ_ID
```
For pipelined `executemany`, the driver sends `SQ_OPEN`+`SQ_FETCH` (or `SQ_BIND`+`SQ_EXEC`) for all N rows back-to-back without waiting for responses, then drains all responses at the end. This is what gives the 1.6× win over IfxPy on bulk inserts — see [Bulk inserts](/how-to/executemany/).
For pipelined `executemany`, the driver sends `SQ_OPEN`+`SQ_FETCH` (or `SQ_BIND`+`SQ_EXEC`) for all N rows back-to-back without waiting for responses, then drains all responses at the end. This is what gives the 1.6× win over IfxPy on bulk inserts. See [Bulk inserts](/how-to/executemany/).
## Smart-LOB transfer
`SQ_FILE` (0x62) is a self-contained PDU type that carries chunks of BLOB/CLOB data. It's used by Informix's `lotofile` and `filetoblob` server functions. The driver intercepts these PDUs at the wire level and reassembles them client-side no `SQ_FPROUTINE` / `SQ_LODATA` machinery needed.
`SQ_FILE` (0x62) is a self-contained PDU type that carries chunks of BLOB/CLOB data. It's used by Informix's `lotofile` and `filetoblob` server functions. The driver intercepts these PDUs at the wire level and reassembles them client-side, with no `SQ_FPROUTINE` / `SQ_LODATA` machinery needed.
This was the architectural pivot in [Phase 10/11](/explain/phase-log/) that made smart-LOBs work end-to-end in pure Python. Reading and writing GB-sized BLOBs goes through the same socket as any other query.
<Aside type="note">
The protocol has many more PDU types than this page covers mostly variants for specific server features (PUT, GET-DESCRIPTOR, ROWDESC, DBINFO, COLLINFO, etc.). The complete list is in [`docs/PROTOCOL_NOTES.md`](https://git.supported.systems/warehack.ing/informix-db/src/branch/main/docs/PROTOCOL_NOTES.md), with hex captures for each.
The protocol has many more PDU types than this page covers, mostly variants for specific server features (PUT, GET-DESCRIPTOR, ROWDESC, DBINFO, COLLINFO, etc.). The complete list is in [`docs/PROTOCOL_NOTES.md`](https://git.supported.systems/warehack.ing/informix-db/src/branch/main/docs/PROTOCOL_NOTES.md), with hex captures for each.
</Aside>

View File

@ -56,7 +56,7 @@ async def get_user(user_id: int, conn = Depends(get_conn)):
## Cancellation
If a client disconnects mid-request, FastAPI cancels the task. `informix-driver` is cancellation-safe the cancellation propagates cleanly, the in-flight worker is reaped, and the connection returns to the pool clean (transactions rolled back). You don't need to wrap anything in `try/finally`.
If a client disconnects mid-request, FastAPI cancels the task. `informix-driver` is cancellation-safe: the cancellation propagates cleanly, the in-flight worker is reaped, and the connection returns to the pool clean, with transactions rolled back. You don't need to wrap anything in `try/finally`.
<Aside type="note">
This is a Phase 27 invariant: async cancellation cannot leak running workers onto recycled connections. The earlier behavior was a `High` audit finding; the fix is a CI tripwire test that's been green every commit since.
@ -89,5 +89,5 @@ async def create_order(order: OrderIn, conn = Depends(get_conn)):
`HTTPException` is an `Exception`, so raising one inside the block rolls back before FastAPI turns it into a response.
<Aside type="note">
There is no `conn.transaction()` context manager on either the sync or async connection an earlier version of this page showed one. Wrapping this in your own `@asynccontextmanager` helper is a few lines if you want the shorthand.
There is no `conn.transaction()` context manager on either the sync or async connection, though an earlier version of this page showed one. Wrapping this in your own `@asynccontextmanager` helper is a few lines if you want the shorthand.
</Aside>

View File

@ -1,11 +1,11 @@
---
title: Optimize bulk SELECT
description: How the buffered reader works in practice — when it's on, when it isn't, how to A/B-measure your workload.
description: How the buffered reader works in practice, when it's on, when it isn't, and how to A/B-measure your workload.
sidebar:
order: 5
---
The connection-scoped buffered reader (Phase 39) is **enabled by default** as of `2026.05.05.12`. For most workloads you don't need to touch anything the bulk-fetch gap against IfxPy is now ~515% rather than ~140%.
The connection-scoped buffered reader (Phase 39) is **enabled by default** as of `2026.05.05.12`. For most workloads you don't need to touch anything, since the bulk-fetch gap against IfxPy is now ~515% rather than ~140%.
For the architectural rationale, see [The buffered reader →](/explain/buffered-reader/).
@ -33,7 +33,7 @@ For typical bulk-SELECT workloads expect a 3040% wall-time reduction. For wor
## When the speedup is largest
Workloads where every column read makes ~45 small `recv()` calls — i.e. tabular data, narrow rows, large row counts. The buffered reader replaces N small `recv()` calls with one `recv(64K)` per ~64 KB of incoming data.
Workloads where every column read makes ~45 small `recv()` calls, meaning tabular data, narrow rows, and large row counts. The buffered reader replaces N small `recv()` calls with one `recv(64K)` per ~64 KB of incoming data.
| Workload shape | Speedup |
|---|---:|

View File

@ -1,6 +1,6 @@
---
title: Run the dev container
description: IBM Informix Developer Edition in Docker — first-time setup, sbspace for smart-LOBs, common troubleshooting.
description: IBM Informix Developer Edition in Docker, covering first-time setup, sbspace for smart-LOBs, and common troubleshooting.
sidebar:
order: 8
---
@ -20,9 +20,9 @@ docker run -d --name informix-dev \
icr.io/informix/informix-developer-database:15.0.1.0.3DE
```
- **`9088`** clear-text SQLI listener
- **`9089`** TLS-enabled SQLI listener (server-side cert is self-signed; use `tls=True` in dev)
- **`--privileged`** required for the dev image's shared-memory tuning
- **`9088`**: clear-text SQLI listener
- **`9089`**: TLS-enabled SQLI listener (server-side cert is self-signed; use `tls=True` in dev)
- **`--privileged`**: required for the dev image's shared-memory tuning
The image takes ~90 seconds to initialize. Watch for `oninit running`:
@ -84,10 +84,10 @@ pytest -m integration
**"Connection refused" on 9088**: the image is still initializing. Wait for `oninit running` in the logs.
**Login succeeds, queries fail with `-329` (database does not exist)**: you're connecting to a database that hasn't been created yet. Use `database="sysmaster"` for ad-hoc testing it always exists.
**Login succeeds, queries fail with `-329` (database does not exist)**: you're connecting to a database that hasn't been created yet. Use `database="sysmaster"` for ad-hoc testing, since it always exists.
**`-908` (system error / shared memory)**: the container needs `--privileged`. Restart with that flag.
<Aside type="tip">
For a long-running dev environment, set `restart: unless-stopped` in a compose file. The image is well-behaved on restart the database survives container shutdown.
For a long-running dev environment, set `restart: unless-stopped` in a compose file. The image is well-behaved on restart, and the database survives container shutdown.
</Aside>

View File

@ -1,6 +1,6 @@
---
title: Bulk inserts (executemany)
description: How to bulk-load with executemany — and the 53× transaction-vs-autocommit gotcha you'll hit otherwise.
description: How to bulk-load with executemany, including the 53× transaction-vs-autocommit gotcha you'll hit otherwise.
sidebar:
order: 4
---
@ -49,7 +49,7 @@ The default is `autocommit=False`, so this only catches you if you've explicitly
## Why it's faster than IfxPy
IfxPy's `executemany` calls `IfxPy.execute(stmt, tuple)` internally per row. That's one round-trip per row — for 10,000 rows on a 80 µs RTT, that's 800 ms of just waiting for ACKs.
IfxPy's `executemany` calls `IfxPy.execute(stmt, tuple)` internally per row. That's one round-trip per row, so for 10,000 rows on an 80 µs RTT it adds up to 800 ms of just waiting for ACKs.
Phase 33 changed our `executemany` to **pipeline** the BIND+EXECUTE PDUs:
@ -78,7 +78,7 @@ with conn:
cur = conn.cursor()
for batch in chunks(huge_iterator, 10_000):
cur.executemany("INSERT INTO logs ...", batch)
# one transaction, many batched executemany calls one commit at the end
# one transaction, many batched executemany calls, one commit at the end
```
10,000 rows per chunk is a reasonable default; the per-chunk Python memory cost is `~ N × bytes_per_row`. For 10k tuples of 5 small fields that's a few MB.
@ -95,7 +95,7 @@ new_id = cur.fetchone()[0]
`sqlca.sqlerrd1` is where Informix records the serial value from the most recent INSERT on this connection, so read it before running any other statement.
There is no `cursor.lastrowid` an earlier version of this page said there was.
There is no `cursor.lastrowid`, though an earlier version of this page said there was.
For batch inserts that need the IDs, the idiomatic pattern is:

View File

@ -27,11 +27,11 @@ conn = informix_db.connect(
)
```
The `informix-driver` keyword-argument form is closer to `psycopg`/`asyncpg` shapes. Connection strings aren't supported (deliberately — they're a security and parsing footgun).
The `informix-driver` keyword-argument form is closer to `psycopg`/`asyncpg` shapes. Connection strings aren't supported, deliberately: they're a security and parsing footgun.
## The DB-API surface is the same
`cursor()`, `execute()`, `fetchone()`, `fetchmany()`, `fetchall()`, `executemany()`, `description`, `rowcount`, `close()` all behave per PEP 249.
`cursor()`, `execute()`, `fetchone()`, `fetchmany()`, `fetchall()`, `executemany()`, `description`, `rowcount`, and `close()` all behave per PEP 249.
The exception hierarchy is identical: `Error`, `Warning`, `InterfaceError`, `DatabaseError`, `DataError`, `OperationalError`, `IntegrityError`, `InternalError`, `ProgrammingError`, `NotSupportedError`.
@ -45,9 +45,39 @@ The exception hierarchy is identical: `Error`, `Warning`, `InterfaceError`, `Dat
- **Async API** (`from informix_db import aio`)
- **Connection pool** (`informix_db.create_pool` / `aio.create_pool`)
- **Type-safe annotations** `informix-driver` ships with `py.typed`
- **Type-safe annotations**: `informix-driver` ships with `py.typed`
- **Python 3.12+ support**
- **Pipelined `executemany`** — 1.6× faster than IfxPy's per-row implementation
- **Pipelined `executemany`**: 1.6× faster than IfxPy's per-row implementation
- **Rows addressable by name**: `row["col"]` and `row.col`, not just `row[0]`
## Reading rows by name
IfxPy gives you positional access and nothing else, which is why most
IfxPy codebases grow a helper like this:
```python
cols = [c[0].lower() for c in cur.description]
row_dict = dict(zip(cols, cur.fetchone()))
```
That gets you a dict and still no attribute access. Ask for `Row`
instead and all three work at once:
```python
conn = informix_db.connect(..., row_factory=informix_db.Row)
cur.execute("SELECT config_key, config_value FROM settings")
row = cur.fetchone()
row[0], row["config_key"], row.config_key
```
Set on the connection, so it applies to every cursor from it. The
`.lower()` in the hand-written version is already a no-op, incidentally:
Informix folds unquoted identifiers to lower case.
It is opt-in because it costs about 9% on bulk fetch. See
[the API reference](/reference/api/#rows) for the numbers and the
edge cases.
## Migrating incrementally

View File

@ -1,6 +1,6 @@
---
title: Use the connection pool
description: Sync and async connection pools — sizing, timeouts, lifecycle, threading.
description: Sync and async connection pools, covering sizing, timeouts, lifecycle, and threading.
sidebar:
order: 2
---
@ -32,7 +32,7 @@ with pool.connection() as conn:
pool.close()
```
The context manager guarantees the connection returns to the pool on normal exit *and* on exception. Connections returned to the pool get rolled back automatically you never see a dirty connection from `pool.connection()`.
The context manager guarantees the connection returns to the pool on normal exit *and* on exception. Connections returned to the pool get rolled back automatically, so you never see a dirty connection from `pool.connection()`.
## Async pool
@ -56,16 +56,16 @@ async def main():
asyncio.run(main())
```
Same semantics, `async`-aware. Cancellation is cancellation-safe — a cancelled task does not leak an in-flight worker onto a recycled connection.
Same semantics, `async`-aware. Cancellation is safe here too: a cancelled task does not leak an in-flight worker onto a recycled connection.
## Sizing
A reasonable starting point: `min_size = 2`, `max_size = (CPU cores) × 2`. Most Informix workloads are I/O-bound, so the right size is "enough to saturate the network plus some headroom for spikes" usually 816 for typical web/API services.
A reasonable starting point: `min_size = 2`, `max_size = (CPU cores) × 2`. Most Informix workloads are I/O-bound, so the right size is "enough to saturate the network plus some headroom for spikes", usually 816 for typical web/API services.
`max_size` should be **smaller than the server's `MAX_CONCURRENT_CONNECTIONS`** — the server fails new logins past its limit, and the pool will surface that as `OperationalError` after waiting `acquire_timeout`.
`max_size` should be **smaller than the server's `MAX_CONCURRENT_CONNECTIONS`**. The server fails new logins past its limit, and the pool will surface that as `OperationalError` after waiting `acquire_timeout`.
## Threading
PEP 249 says: connections should not be shared between threads. The pool gives each thread its own connection naturally `pool.connection()` returns a different connection each time and each one stays held until the context manager exits.
PEP 249 says: connections should not be shared between threads. The pool gives each thread its own connection naturally, since `pool.connection()` returns a different connection each time and each one stays held until the context manager exits.
Phase 27 added a per-connection wire lock that makes accidental sharing safe (interleaved PDUs serialize correctly), but you should still give each thread its own connection. The lock is a backstop, not a license.

View File

@ -34,7 +34,7 @@ The `BLOB_PLACEHOLDER` token in the SQL marks where the BLOB data goes. Other pa
## Reading a CLOB
CLOBs use the same methods as BLOBs there is no separate `read_clob_column`. `read_blob_column` returns `bytes` for both, so decode it with the encoding the column was written in:
CLOBs use the same methods as BLOBs, and there is no separate `read_clob_column`. `read_blob_column` returns `bytes` for both, so decode it with the encoding the column was written in:
```python
raw: bytes = cur.read_blob_column(

View File

@ -30,7 +30,7 @@ conn = informix_db.connect(
)
```
Bring-your-own context is the recommended production pattern — you get full control of certificate verification, hostname checking, ciphers, and TLS version pinning.
Bring-your-own context is the recommended production pattern, giving you full control of certificate verification, hostname checking, ciphers, and TLS version pinning.
## Dev / self-signed: tls=True
@ -38,7 +38,7 @@ Bring-your-own context is the recommended production pattern — you get full co
informix_db.connect(host="127.0.0.1", port=9089, ..., tls=True)
```
`tls=True` is a convenience for development — it builds a default context with `check_hostname=False` and `verify_mode=CERT_NONE`. **Do not use this in production.**
`tls=True` is a convenience for development. It builds a default context with `check_hostname=False` and `verify_mode=CERT_NONE`. **Do not use this in production.**
## Server-side configuration
@ -48,7 +48,7 @@ The Informix server needs a TLS listener entry in `sqlhosts`:
informix_tls onsoctcp myhost 9089
```
Plus a server-side keystore. The IBM Developer Edition Docker image ships with a TLS listener already enabled on `9089` no configuration needed.
Plus a server-side keystore. The IBM Developer Edition Docker image ships with a TLS listener already enabled on `9089`, with no configuration needed.
<Aside type="tip">
If your connection hangs at the handshake, you've probably pointed at the non-TLS port (`9088` instead of `9089`). The non-TLS listener will accept the TCP connection but won't speak TLS, so the handshake stalls.

View File

@ -1,6 +1,6 @@
---
title: informix-driver
description: Pure-Python driver for IBM Informix IDS. Speaks the SQLI wire protocol over a raw socket — no CSDK, no JVM, no native libraries.
description: Pure-Python driver for IBM Informix IDS. Speaks the SQLI wire protocol over a raw socket, with no CSDK, no JVM, and no native libraries.
template: splash
hero:
tagline: ''
@ -21,12 +21,12 @@ import { Card, CardGrid, Icon } from '@astrojs/starlight/components';
<div class="ifx-feature">
<Icon name="seti:python" class="ifx-feature__icon" />
<h3>~10% behind IfxPy on bulk fetches</h3>
<p>Phase 39's buffered reader closed the gap from 2.4× to within measurement noise of the C driver. The remaining ~10% is honest physics for now.</p>
<p>Phase 39's buffered reader closed the gap from 2.4× to within measurement noise of the C driver. The remaining ~10% is honest physics, for now.</p>
</div>
<div class="ifx-feature">
<Icon name="puzzle" class="ifx-feature__icon" />
<h3>50 KB wheel. Zero native deps.</h3>
<p>No 92 MB OneDB tarball. No <code>libcrypt.so.1</code> from 2018. No <code>LD_LIBRARY_PATH</code> ritual. Works on Python 3.103.14 including the versions IfxPy doesn't.</p>
<p>No 92 MB OneDB tarball. No <code>libcrypt.so.1</code> from 2018. No <code>LD_LIBRARY_PATH</code> ritual. Works on Python 3.103.14, including the versions IfxPy doesn't.</p>
</div>
<div class="ifx-feature">
<Icon name="sun" class="ifx-feature__icon" />
@ -36,7 +36,7 @@ import { Card, CardGrid, Icon } from '@astrojs/starlight/components';
<div class="ifx-feature">
<Icon name="approve-check" class="ifx-feature__icon" />
<h3>PEP 249, no surprises</h3>
<p><code>connect()</code>, <code>Connection</code>, <code>Cursor</code>, <code>description</code>, <code>rowcount</code>, the full DB-API exception hierarchy — and threadsafe sharing through a per-connection wire lock.</p>
<p><code>connect()</code>, <code>Connection</code>, <code>Cursor</code>, <code>description</code>, <code>rowcount</code>, the full DB-API exception hierarchy, plus threadsafe sharing through a per-connection wire lock.</p>
</div>
<div class="ifx-feature">
<Icon name="document" class="ifx-feature__icon" />
@ -66,7 +66,7 @@ That's it. No `IBM_DB_HOME`. No DSN file. No `libcrypt.so.1`.
The existing tools were not my style.
Every other Informix driver in any language wraps either IBM's C Client SDK or the JDBC JAR. `IfxPy`, the legacy `informixdb`, ODBC bridges, JPype/JDBC, Perl `DBD::Informix` — all of them. To our knowledge **`informix-driver` is the first pure-socket Informix driver in any language**.
Every other Informix driver in any language wraps either IBM's C Client SDK or the JDBC JAR. That means all of them: `IfxPy`, the legacy `informixdb`, ODBC bridges, JPype/JDBC, and Perl's `DBD::Informix`. To our knowledge **`informix-driver` is the first pure-socket Informix driver in any language**.
The OneDB CSDK is a 92 MB tarball. It needs `libcrypt.so.1` (deprecated 2018, missing on Arch, Fedora 35+, RHEL 9). It needs four `LD_LIBRARY_PATH` entries. It needs `setuptools < 58`. And IfxPy itself is broken on Python 3.12+. For containerized deployments, ETL pipelines, FastAPI services, or anywhere a build toolchain on the runtime is friction, this driver is the alternative that didn't previously exist. Now it does.
@ -82,7 +82,7 @@ The OneDB CSDK is a 92 MB tarball. It needs `libcrypt.so.1` (deprecated 2018, mi
[Read →](/start/vs-ifxpy/)
</Card>
<Card title="The buffered reader" icon="information">
How Phase 39 closed the bulk-fetch gap from 2.4× to ~1.1× and the architectural mistake the first pass got wrong.
How Phase 39 closed the bulk-fetch gap from 2.4× to ~1.1×, and the architectural mistake the first pass got wrong.
[Read →](/explain/buffered-reader/)
</Card>
<Card title="Architecture" icon="puzzle">

View File

@ -51,7 +51,7 @@ informix_db.connect(
| Method / property | Description |
|---|---|
| `cursor(scrollable=False)` | Returns a new `Cursor`. |
| `cursor(scrollable=False)` | Returns a new `Cursor`. Only one scrollable cursor may be open per connection; see below. |
| `commit()` | Commits the current transaction. |
| `rollback()` | Rolls back the current transaction. |
| `close()` | Closes the connection. Idempotent. |
@ -64,7 +64,7 @@ informix_db.connect(
`Connection` is also a context manager (`with informix_db.connect(...) as conn:`), which closes on exit.
There is no `conn.transaction()` helper and no `conn.autocommit` attribute — earlier versions of this page listed both, and neither has ever existed. Set autocommit at connect time with `connect(autocommit=True)`, and manage transactions with `commit()` / `rollback()`:
There is no `conn.transaction()` helper and no `conn.autocommit` attribute. Earlier versions of this page listed both, and neither has ever existed. Set autocommit at connect time with `connect(autocommit=True)`, and manage transactions with `commit()` / `rollback()`:
```python
conn = informix_db.connect(..., autocommit=False)
@ -78,12 +78,67 @@ except Exception:
raise
```
:::caution[One statement per session]
Informix gives a session a single statement slot. A non-scrollable cursor never trips over this, because it materializes its rows and releases the statement before `execute()` returns. A **scrollable** cursor holds the slot open on purpose, so while one is open, any other statement on that same connection raises `ProgrammingError`:
```python
scroll = conn.cursor(scrollable=True)
scroll.execute("SELECT id, body FROM big_table ORDER BY id")
scroll.fetch_first()
other = conn.cursor()
other.execute("SELECT COUNT(*) FROM audit") # ProgrammingError
```
Close the scrollable cursor first, or use a second connection for the other query. Re-running `execute()` on the scrollable cursor itself is fine; it closes its own server-side cursor first.
Before `2026.09.02` this was not refused. The server answered the second statement with `-285` and destroyed the scrollable cursor as well, whose next fetch then returned `-267`, "the transaction has been rolled back". Two errors, neither naming the cause.
:::
:::note[Transaction control written as SQL]
`commit()` and `rollback()` are the supported way to end a transaction, but `cursor.execute("BEGIN WORK")` and its `COMMIT` / `ROLLBACK` counterparts work too, and the connection tracks them from `2026.09.02` onward.
That matters because it used not to. Under `autocommit=True`, a SQL `BEGIN WORK` opened a real transaction the connection never learned about, and since `rollback()` is guarded by that state, **it returned successfully having sent nothing**. The rows it was asked to discard survived, and a pooled connection went back into circulation still holding the transaction and its locks.
:::
:::note[Two version numbers, and why]
The login response carries Informix's *internal* protocol version, not the release you installed: 12.10 announces itself as `9.56`, 14.10 as `9.59`, and 15 as `15.0.1.0.3`. At the protocol level the two older releases really are 9.x servers, which is why all three speak an identical SQLI dialect.
That string is available for free as `server_version_internal`. Because it reads as a wrong answer, `server_version` asks the server for its release with `DBINFO('version','full')` — one query on first access, cached for the life of the connection, and it falls back to the internal string rather than raising if no database is open.
That string is available for free as `server_version_internal`. Because it reads as a wrong answer, `server_version` asks the server for its release with `DBINFO('version','full')`. That is one query on first access, cached for the life of the connection, and it falls back to the internal string rather than raising if no database is open.
:::
## Rows
Fetch methods return plain tuples by default. Pass `row_factory` to get rows that also answer to a column name and an attribute, the shape `pyodbc` and `mssql-python` provide:
```python
conn = informix_db.connect(..., row_factory=informix_db.Row)
cur = conn.cursor()
cur.execute("SELECT tabid, tabname FROM systables")
row = cur.fetchone()
row[0] # 1
row["tabname"] # 'systables'
row.tabname # 'systables'
```
Set on the connection, so it applies to every cursor from it. A cursor can override it with `cur.row_factory = informix_db.Row`. Pools and the async API forward it unchanged.
`Row` subclasses `tuple`, so `row == (1, "systables")` is still true and anything treating rows as sequences keeps working. Slices return plain tuples, since a slice has no column map. `row.keys()` gives the column names and `row._asdict()` gives a dict.
:::note[Why it is opt-in]
On a 20,000-row five-column fetch: 37.2 ms with tuples, 40.8 ms with `Row`, about 9%. Supporting `row["name"]` means `__getitem__` becomes a Python method rather than C-level tuple indexing, which costs roughly 39 ns on every subscript.
Defaulting it on would move the [1.05-1.15x ratio against IfxPy](/start/vs-ifxpy/) to roughly 1.15-1.25x. That is a good trade for readable application code and a bad one for a bulk export that never looks at a column by name, so it is yours to make.
:::
Four behaviours worth knowing:
- Informix folds unquoted identifiers to lower case, so `SELECT Config_Key` is reachable as `row.config_key`. The `.lower()` people add by hand when building this themselves is already a no-op.
- Expression columns get server-generated names like `(count(*))`, which cannot be Python attributes. Those are reachable by subscript only.
- Duplicate column names resolve to the first occurrence, matching `pyodbc`. Positional access still reaches both.
- **A column beats a method of the same name.** `tuple` defines `count` and `index`; `Row` adds `keys`, `_asdict` and `_fields`. A column with any of those names wins, because otherwise it would hand back a bound method instead of a value. The methods stay reachable as `tuple.count(row, x)` and `Row.keys(row)`.
## Cursor
| Method / property | Description |
@ -105,7 +160,7 @@ That string is available for free as `server_version_internal`. Because it reads
| `rownumber` | Current 0-indexed position, or `None` before the first row. |
| `arraysize` | Default `fetchmany()` size. |
There are no `read_clob_column` / `write_clob_column` methods and no `lastrowid` attribute — earlier versions of this page listed all three and none have existed.
There are no `read_clob_column` / `write_clob_column` methods and no `lastrowid` attribute. Earlier versions of this page listed all three, and none have existed.
CLOBs go through the BLOB methods. `read_blob_column` returns `bytes` either way, so decode it yourself with the column's encoding:
@ -120,7 +175,7 @@ raw = cur.read_blob_column("SELECT txt FROM docs WHERE id = ?", (1,))
text = raw.decode("iso-8859-1")
```
For a server-assigned `SERIAL` after an INSERT, ask the server Informix exposes it through `DBINFO`:
For a server-assigned `SERIAL` after an INSERT, ask the server. Informix exposes it through `DBINFO`:
```python
cur.execute("INSERT INTO people (name) VALUES (?)", ("ada",))

View File

@ -1,6 +1,6 @@
---
title: Performance baselines
description: Single-connection benchmark results — codec, framing, end-to-end queries, vs IfxPy.
description: Single-connection benchmark results for codec, framing, end-to-end queries, and IfxPy.
sidebar:
order: 5
---

View File

@ -13,7 +13,7 @@ sidebar:
| `IFX_DEBUG_WIRE` | unset | When set to a truthy value, log wire-level PDU framing to stderr. Verbose; for debugging only. |
| `IFX_PROTOCOL_TRACE` | unset | When set to a path, write annotated wire captures to the file. |
| `IFX_DISABLE_PIPELINE` | unset | Disables pipelined `executemany` (Phase 33). Use only to A/B-measure. |
| `INFORMIXSERVER` | | Read by `connect()` if `server=` is not provided. |
| `INFORMIXSERVER` | *(none)* | Read by `connect()` if `server=` is not provided. |
Environment variables are read at connection construction. Changing them at runtime doesn't affect existing connections.
@ -64,4 +64,4 @@ informix_db.connect(
)
```
`CLIENT_LOCALE` is set automatically from `client_locale=` don't put it in `env=`.
`CLIENT_LOCALE` is set automatically from `client_locale=`, so don't put it in `env=`.

View File

@ -9,7 +9,7 @@ sidebar:
|---|---|---|
| `SMALLINT` / `INT` / `SERIAL` | `int` | Arbitrary precision on the Python side. |
| `BIGINT` / `BIGSERIAL` | `int` | 8-byte two's complement on the wire. |
| `INT8` / `SERIAL8` | `int` | The *legacy* 64-bit integer — a different wire format from `BIGINT`, not an alias. See below. |
| `INT8` / `SERIAL8` | `int` | The *legacy* 64-bit integer, a different wire format from `BIGINT` rather than an alias. See below. |
| `FLOAT` / `SMALLFLOAT` | `float` | IEEE 754 double / single. |
| `DECIMAL(p,s)` / `MONEY` | `decimal.Decimal` | Exact precision preserved. |
| `CHAR` / `NCHAR` | `str` | Fixed width, space-padded; trailing spaces stripped on decode. |
@ -18,11 +18,11 @@ sidebar:
| `DATE` | `datetime.date` | |
| `DATETIME YEAR TO …` | `datetime.datetime` / `datetime.time` / `datetime.date` | The Python type depends on the field range. |
| `INTERVAL DAY TO FRACTION` | `datetime.timedelta` | |
| `INTERVAL YEAR TO MONTH` | `informix_db.IntervalYM` | Custom type `datetime.timedelta` can't represent year-month intervals. |
| `INTERVAL YEAR TO MONTH` | `informix_db.IntervalYM` | Custom type, because `datetime.timedelta` can't represent year-month intervals. |
| `BYTE` / `TEXT` (legacy in-row blobs) | `bytes` / `str` | |
| `BLOB` / `CLOB` (smart-LOBs) | `informix_db.BlobLocator` / `informix_db.ClobLocator` | Opaque server-side locators. Read via `cursor.read_blob_column`, write via `cursor.write_blob_column`. |
| `ROW(…)` | `informix_db.RowValue` | Raw payload plus schema string **not** decomposed into fields. See below. |
| `SET(…)` / `MULTISET(…)` / `LIST(…)` | `informix_db.CollectionValue` | Raw payload plus element schema **not** iterable. See below. |
| `BLOB` / `CLOB` (smart-LOBs) | `informix_db.BlobLocator` / `informix_db.ClobLocator` | Opaque server-side locators, 72 bytes. Read via `cursor.read_blob_column`, write via `cursor.write_blob_column`. |
| `ROW(…)` | `informix_db.RowValue` | Raw payload plus schema string, **not** decomposed into fields. See below. |
| `SET(…)` / `MULTISET(…)` / `LIST(…)` | `informix_db.CollectionValue` | Raw payload plus element schema, **not** iterable. See below. |
| `NULL` | `None` | |
## INT8 and SERIAL8 are not BIGINT
@ -34,16 +34,20 @@ Informix has two unrelated 64-bit integer types and they share nothing on the wi
| `BIGINT` / `BIGSERIAL` (type 52 / 53) | 8 bytes, two's complement, big-endian |
| `INT8` / `SERIAL8` (type 17 / 18) | 10 bytes, sign-magnitude, halves stored high-last |
Both decode to plain Python `int`, so this only matters if you're reading the wire yourself. It's worth knowing that `INT8` stores `+n` and `n` with *identical* magnitude bytes and the sign in a leading word — decoding it as a signed 64-bit integer looks correct for every positive value and is wrong for every negative one.
Both decode to plain Python `int`, so this only matters if you're reading the wire yourself. It's worth knowing that `INT8` stores `+n` and `n` with *identical* magnitude bytes and the sign in a leading word. Decoding it as a signed 64-bit integer looks correct for every positive value and is wrong for every negative one.
`INT8`/`SERIAL8` are common in schemas predating Informix 11.50, which is when `BIGINT` arrived.
:::caution[Upgrade if you use these types]
`2026.08.27` fixed three decoding bugs: `INT8`/`SERIAL8` weren't decoded at all and returned raw `bytes`; `NCHAR` lost its first character; and `BOOLEAN` corrupted every column after it.
`2026.08.31` fixed two more in `LVARCHAR` framing a phantom pad byte on odd-length values, and a missing length field on NULLs. Either shifted **every column selected after the LVARCHAR**, producing wrong integers, strings missing their first character, or an outright `IndexError` on wide rows. The same release stopped `DATETIME` binds silently discarding sub-second precision.
`2026.08.31` fixed two more in `LVARCHAR` framing: a phantom pad byte on odd-length values, and a missing length field on NULLs. Either shifted **every column selected after the LVARCHAR**, producing wrong integers, strings missing their first character, or an outright `IndexError` on wide rows. The same release stopped `DATETIME` binds silently discarding sub-second precision.
If your schema has `LVARCHAR` columns, `2026.08.27` is not safe — upgrade to `2026.08.31`.
`2026.09.02` fixed two more, both found by a new check that verifies each row consumed exactly its own payload. Smart LOBs were read as a flat 72-byte field when they actually occupy 149 bytes populated and 5 when NULL, so a `BLOB` or `CLOB` anywhere but the **last** column shifted every column after it. A NULL `SET` / `MULTISET` / `LIST` / `ROW` skipped its length field and did the same. That release also made a mismatch raise where it happens, naming the column and the byte delta, instead of handing back plausible-looking wrong values.
`2026.09.03` fixed one on the **write** path rather than the read path, which makes it the only entry here that affects data already stored. The `:N` placeholder rewriter was a regular expression and could not see a string literal, so it rewrote the inside of one: `'http://host:8080/x'` was stored as `'http://host?/x'`. Any `HH:MM` time, URL with a port, or `key:value` string was affected, whichever placeholder style you used. If you have written such values through this driver, they are worth checking.
If your schema has `LVARCHAR` columns, `2026.08.27` is not safe. If you select a `BLOB` or `CLOB` in any position other than last, or a nullable collection column, nothing before `2026.09.02` is safe. Upgrade.
:::
## LVARCHAR wire framing
@ -54,7 +58,7 @@ Worth knowing if you're reading captures. Every Informix server we've tested des
[1-byte null indicator][4-byte length][content]
```
No padding, and the length field is present even when the indicator says NULL. NULL and empty string differ *only* in that indicator byte both carry a length of zero.
No padding, and the length field is present even when the indicator says NULL. NULL and empty string differ *only* in that indicator byte, since both carry a length of zero.
## DATETIME field ranges
@ -68,17 +72,17 @@ Informix's `DATETIME YEAR TO X` is field-range typed. The Python type returned d
## Decimal precision
`DECIMAL` columns preserve their declared precision and scale through the codec. A `DECIMAL(10,2)` column with value `123.45` decodes to `Decimal("123.45")` exactly no float intermediate.
`DECIMAL` columns preserve their declared precision and scale through the codec. A `DECIMAL(10,2)` column with value `123.45` decodes to `Decimal("123.45")` exactly, with no float intermediate.
For binding `Decimal` values into INSERT/UPDATE, the driver uses the column's declared scale. Pass `Decimal` for exact values; `float` works but may cause rounding at the column scale.
## NULL
`NULL` is `None` in both directions. Use `IS NULL` / `IS NOT NULL` in SQL `WHERE x = ?` with `None` returns no rows even where `x IS NULL`.
`NULL` is `None` in both directions. Use `IS NULL` / `IS NOT NULL` in SQL, because `WHERE x = ?` with `None` returns no rows even where `x IS NULL`.
## Type extensions
`informix_db.IntervalYM` represents `INTERVAL YEAR TO MONTH`. It takes a **single total month count** — mirroring the server's own representation — with `years` and `remainder_months` available as derived properties:
`informix_db.IntervalYM` represents `INTERVAL YEAR TO MONTH`. It takes a **single total month count**, mirroring the server's own representation, with `years` and `remainder_months` available as derived properties:
```python
from informix_db import IntervalYM
@ -100,8 +104,8 @@ Negative intervals are supported; the sign lives on `months` and propagates to b
`informix_db.RowValue` and `informix_db.CollectionValue` are returned for `ROW`, `SET`, `MULTISET`, and `LIST` columns. Both are **opaque wrappers, not decomposed values**:
```python
row_val.raw # bytes the server's textual representation, e.g. b"ROW('Alice',30)"
row_val.schema # str the column's declared schema
row_val.raw # bytes: the server's textual representation, e.g. b"ROW('Alice',30)"
row_val.schema # str: the column's declared schema
coll_val.raw # bytes, e.g. b'LIST{10,20,30}'
coll_val.kind # 'set' | 'multiset' | 'list' | 'collection'
@ -110,7 +114,7 @@ coll_val.element_schema # str
They are not iterable and do not expose fields by name. Fully parsing a composite type means reimplementing JDBC's `IfxComplexInput`, which we haven't done.
If you need individual fields today, project them in SQL — the server does the decomposition for you and you get ordinary typed columns back:
If you need individual fields today, project them in SQL. The server does the decomposition for you and you get ordinary typed columns back:
```python
cur.execute("SELECT person.name, person.age FROM staff") # str, int

View File

@ -55,7 +55,7 @@ docker logs -f informix-dev
```
<Aside type="tip">
The `--privileged` flag is required by the dev image — it tries to manage shared memory limits. For production servers this isn't a thing.
The `--privileged` flag is required by the dev image, which tries to manage shared memory limits. For production servers this isn't a thing.
</Aside>
## 3. Run your first query
@ -88,7 +88,7 @@ Then:
python hello.py
```
You should see five rows from Informix's system catalog. If you do, congratulations you've spoken SQLI to an IBM database from pure Python with zero native code in the call stack.
You should see five rows from Informix's system catalog. If you do, congratulations: you've spoken SQLI to an IBM database from pure Python with zero native code in the call stack.
## What just happened
@ -119,7 +119,12 @@ with informix_db.connect(host="127.0.0.1", port=9088, user="informix",
print(cur.fetchone())
```
`?` and `:1` both work — Informix's native paramstyle is `numeric`, but `?` is supported as a synonym.
`?` and `:1` both work. Informix's native paramstyle is `numeric`, but `?` is supported as a synonym.
Rows come back as plain tuples. If you would rather read them by column
name, pass `row_factory=informix_db.Row` to `connect()` and you get
`row["name"]` and `row.name` alongside `row[0]`. See
[the API reference](/reference/api/#rows).
## 5. Use the connection pool
@ -144,7 +149,7 @@ with pool.connection() as conn:
pool.close()
```
The pool is thread-safe and has a per-connection wire lock accidental sharing across threads doesn't corrupt the wire stream, though PEP 249 advice still holds (one connection per thread).
The pool is thread-safe and has a per-connection wire lock, so accidental sharing across threads doesn't corrupt the wire stream, though PEP 249 advice still holds (one connection per thread).
## What's next
@ -154,7 +159,7 @@ The pool is thread-safe and has a per-connection wire lock — accidental sharin
2. **Connecting to production?** [Connect with TLS →](/how-to/tls/) covers TLS-listener configuration and bring-your-own-context patterns.
3. **Bulk-loading?** [Bulk inserts (executemany) →](/how-to/executemany/) — and the 53× transaction-vs-autocommit gotcha you'll hit otherwise.
3. **Bulk-loading?** [Bulk inserts (executemany) →](/how-to/executemany/), including the 53× transaction-vs-autocommit gotcha you'll hit otherwise.
4. **Migrating from IfxPy?** [Migrate from IfxPy →](/how-to/migrate-from-ifxpy/) covers the API differences and the things IfxPy does that we don't (yet).

View File

@ -7,7 +7,7 @@ sidebar:
import { Aside } from '@astrojs/starlight/components';
[IfxPy](https://pypi.org/project/IfxPy/) is IBM's official Python driver a C extension that wraps the OneDB Client SDK (CSDK), which itself wraps the same SQLI wire protocol `informix-driver` speaks directly. It's the reasonable comparison: same protocol, same server, same workload, different transport.
[IfxPy](https://pypi.org/project/IfxPy/) is IBM's official Python driver, a C extension that wraps the OneDB Client SDK (CSDK), which itself wraps the same SQLI wire protocol `informix-driver` speaks directly. It's the reasonable comparison: same protocol, same server, same workload, different transport.
Numbers below are **median + IQR over 10+ rounds**, all against the same IBM Informix Developer Edition Docker container on the same host. Methodology and reproduction steps live in [`tests/benchmarks/compare/`](https://git.supported.systems/warehack.ing/informix-db/src/branch/main/tests/benchmarks/compare) in the repo.
@ -33,9 +33,9 @@ Numbers below are **median + IQR over 10+ rounds**, all against the same IBM Inf
### Bulk inserts at scale
The clearest win is bulk insert throughput. `executemany(10_000_rows)` runs in **161 ms** vs IfxPy's **259 ms** `informix-driver` is 1.6× faster.
The clearest win is bulk insert throughput. `executemany(10_000_rows)` runs in **161 ms** vs IfxPy's **259 ms**, so `informix-driver` is 1.6× faster.
The mechanism is pipelining. Phase 33 changed `executemany` to send all N BIND+EXECUTE PDUs back-to-back **before** draining any response. IfxPy's C-level `IfxPy.execute(stmt, tuple)` makes one round-trip per row N RTTs at ~80 µs each adds up to the 100 ms gap.
The mechanism is pipelining. Phase 33 changed `executemany` to send all N BIND+EXECUTE PDUs back-to-back **before** draining any response. IfxPy's C-level `IfxPy.execute(stmt, tuple)` makes one round-trip per row, and N RTTs at ~80 µs each adds up to the 100 ms gap.
```python
# Both drivers
@ -44,8 +44,8 @@ cur.executemany(
rows, # list of 10_000 tuples
)
# informix-driver: 161 ms — 10k PDUs sent, then 10k responses drained
# IfxPy: 259 ms — 10k round-trips, each blocking on response
# informix-driver: 161 ms (10k PDUs sent, then 10k responses drained)
# IfxPy: 259 ms (10k round-trips, each blocking on response)
```
### Containerized deployment
@ -62,7 +62,7 @@ IfxPy's deployment surface is dramatically larger:
- 92 MB IBM OneDB Client tarball
- `setuptools < 58` build pin
- `LD_LIBRARY_PATH` configuration for four directories
- `libcrypt.so.1` (deprecated 2018 missing on Arch, Fedora 35+, RHEL 9)
- `libcrypt.so.1` (deprecated 2018, missing on Arch, Fedora 35+, RHEL 9)
- C compiler in the build image
For slim images, multi-stage builds, FaaS deployments, or anywhere build-toolchain-on-the-runtime is friction, `informix-driver` is the only reasonable option.
@ -88,13 +88,13 @@ async def main():
rows = await cur.fetchall()
```
IfxPy has no async support every call blocks the event loop. Using IfxPy from FastAPI requires `loop.run_in_executor()` boilerplate, and the thread pool isn't connection-aware so you give up the natural fairness of an async pool.
IfxPy has no async support, so every call blocks the event loop. Using IfxPy from FastAPI requires `loop.run_in_executor()` boilerplate, and the thread pool isn't connection-aware so you give up the natural fairness of an async pool.
## When IfxPy wins
### Large analytical fetches
For queries pulling 10k+ rows where per-row decode cost dominates, IfxPy is currently 515% faster. The C-level `fetch_tuple` decoder is ~1.1 µs/row; our Python `parse_tuple_payload` is ~2.0 µs/row after Phase 39 (down from ~2.7 before). At 100k rows the gap is ~80 ms wall-clock meaningful but not disqualifying.
For queries pulling 10k+ rows where per-row decode cost dominates, IfxPy is currently 515% faster. The C-level `fetch_tuple` decoder is ~1.1 µs/row; our Python `parse_tuple_payload` is ~2.0 µs/row after Phase 39 (down from ~2.7 before). At 100k rows the gap is ~80 ms wall-clock, which is meaningful but not disqualifying.
The gap is closing phase by phase:
@ -109,7 +109,7 @@ If you're running analytical reports that pull millions of rows in a single SELE
### Workloads built around CSDK extensions
If your existing code uses IBM-specific cursor extensions (`cursor.callproc` with named parameters, IBM's specific scrollable cursor semantics around `last`/`prior`/`relative`, `cursor.set_chunk_size` for fetch tuning), the migration to `informix-driver` is straightforward but not zero-cost. We support the core PEP 249 surface plus our own scrollable cursor API — see [the migration guide](/how-to/migrate-from-ifxpy/).
If your existing code uses IBM-specific cursor extensions (`cursor.callproc` with named parameters, IBM's specific scrollable cursor semantics around `last`/`prior`/`relative`, `cursor.set_chunk_size` for fetch tuning), the migration to `informix-driver` is straightforward but not zero-cost. We support the core PEP 249 surface plus our own scrollable cursor API. See [the migration guide](/how-to/migrate-from-ifxpy/).
## Methodology
@ -117,7 +117,7 @@ Benchmarks are pytest-benchmark fixtures in `tests/benchmarks/compare/` against
Reported numbers are **median over 10+ rounds**, with IQR included. Why median over mean: the first round of any run includes JIT warmup, page-cache miss, and a TCP slow-start round-trip. The mean is contaminated by these one-shot costs in a way that misrepresents steady-state behavior. Median + IQR is what we report.
IfxPy's IQR on the 100k-row SELECT is ~21% (Docker→host loopback noise, plus the C extension's allocation patterns). Our IQR is ~0.2%. The headline 1.15× ratio at 100k rows is partly that noise — a fair reading is "515% slower than IfxPy on large fetches", and the lower bound may already be within measurement noise.
IfxPy's IQR on the 100k-row SELECT is ~21% (Docker→host loopback noise, plus the C extension's allocation patterns). Our IQR is ~0.2%. The headline 1.15× ratio at 100k rows is partly that noise. A fair reading is "515% slower than IfxPy on large fetches", and the lower bound may already be within measurement noise.
To reproduce:
@ -146,4 +146,4 @@ Use IfxPy when:
- You're running large analytical SELECTs and the 515% decode-side gap matters
- You're constrained to Python ≤ 3.11 anyway
For everything else the cost-benefit favors `pip install informix-driver`.
For everything else, the cost-benefit favors `pip install informix-driver`.

View File

@ -8,7 +8,7 @@ sidebar:
The existing tools were not my style.
Every Informix driver in any language `IfxPy`, the legacy `informixdb`, ODBC bridges, JPype/JDBC, Perl `DBD::Informix` — wraps either IBM's C Client SDK or the JDBC JAR. To our knowledge `informix-driver` is the **first pure-socket Informix driver in any language**.
Every Informix driver in any language wraps either IBM's C Client SDK or the JDBC JAR. That covers `IfxPy`, the legacy `informixdb`, ODBC bridges, JPype/JDBC, and Perl's `DBD::Informix`. To our knowledge `informix-driver` is the **first pure-socket Informix driver in any language**.
## The problem with IBM's C SDK
@ -19,19 +19,19 @@ The IBM Informix Client SDK (CSDK), now packaged as part of OneDB Client, is a 9
- Permissive `CFLAGS` for the C extension build
- Manual download of the 92 MB ODBC tarball
- Four `LD_LIBRARY_PATH` directories
- `libcrypt.so.1` — deprecated in 2018, missing on Arch, Fedora 35+, RHEL 9
- `libcrypt.so.1`, deprecated in 2018 and missing on Arch, Fedora 35+, RHEL 9
For containerized deployments, ETL pipelines, FastAPI services, or anywhere Python lives and IBM's C SDK is friction, the friction compounds. `informix-driver`'s install is `pip install informix-driver` (`import informix_db` the distribution name dodges PyPI's 2008-vintage `informixdb` package, the import name is what you'd expect). The wheel is ~50 KB. There are zero runtime dependencies.
For containerized deployments, ETL pipelines, FastAPI services, or anywhere Python lives and IBM's C SDK is friction, the friction compounds. `informix-driver`'s install is `pip install informix-driver` (`import informix_db`; the distribution name dodges PyPI's 2008-vintage `informixdb` package, while the import name is what you'd expect). The wheel is ~50 KB. There are zero runtime dependencies.
## What it does
`informix-driver` opens a TCP socket to an Informix server's SQLI listener and speaks the wire protocol directly — the same protocol IBM's JDBC driver uses, the same protocol the CSDK speaks under the hood. No native code is in the thread of execution.
`informix-driver` opens a TCP socket to an Informix server's SQLI listener and speaks the wire protocol directly. It is the same protocol IBM's JDBC driver uses, and the same one the CSDK speaks under the hood. No native code is in the thread of execution.
The wire protocol was reverse-engineered through three sources:
1. **Decompiled IBM JDBC driver** (`com.informix.jdbc.IfxConnection` and friends), used as a clean-room reference for PDU shapes and protocol semantics.
2. **Annotated `socat` captures** of real client/server traffic against the IBM Informix Developer Edition Docker image.
3. **Differential testing** against `IfxPy` — every codec path is tested against the C driver's behavior on the same data.
3. **Differential testing** against `IfxPy`, so every codec path is checked against the C driver's behavior on the same data.
The result is a PEP 249 compliant driver with a sync API, an async API (FastAPI / asyncio compatible), a connection pool, TLS support, smart-LOB read/write, scrollable cursors, fast-path stored procedure invocation, and bulk-insert / bulk-fetch performance within ~1060% of the C driver depending on workload.
@ -76,12 +76,12 @@ Every finding from a system-wide failure-mode audit (data correctness, wire safe
**0 critical, 0 high, 0 medium audit findings remain.** Every architectural change went through a Margaret Hamilton-style review focused on silent-failure modes, recovery paths, and documented invariants. Each documented invariant is paired with either a runtime guard or a CI tripwire test.
400+ tests across unit / integration / benchmark suites. The integration suite runs against the official IBM Informix Developer Edition Docker images and passes 414/414 on **all three** of 12.10.FC12W1DE, 14.10.FC7W1DE, and 15.0.1.0.3DE — `make test-matrix` runs the lot.
470+ tests across unit / integration / benchmark suites. The integration suite runs against the official IBM Informix Developer Edition Docker images and passes 472/472 on 15.0.1.0.3DE and 14.10.FC7W1DE, and 471/472 on 12.10.FC12W1DE (the single skip is a common table expression, which 12.10 predates). `make test-matrix` runs the lot.
That matrix exists because it turned out to be needed. A user reported corrupted result sets on Informix 12; the cause was three framing bugs that affected every version including the one we tested against, and they'd survived because no fixture used the affected types. Testing one server and inferring the rest is how that happens.
## Read next
- **[Install & first query →](/start/quickstart/)** five minutes from `pip install` to a real SELECT against a Docker-hosted Informix.
- **[Compared to IfxPy →](/start/vs-ifxpy/)** full head-to-head benchmarks, methodology, and reproduction.
- **[Architecture →](/explain/architecture/)** — how the layers stack: socket, framing, codec, resultset, cursor.
- **[Install & first query →](/start/quickstart/)**: five minutes from `pip install` to a real SELECT against a Docker-hosted Informix.
- **[Compared to IfxPy →](/start/vs-ifxpy/)**: full head-to-head benchmarks, methodology, and reproduction.
- **[Architecture →](/explain/architecture/)**: how the layers stack, from socket through framing, codec, resultset, and cursor.

View File

@ -243,7 +243,7 @@
color: var(--sl-color-gray-2);
}
/* Supported Systems "joint" badge appears below every page's footer */
/* Supported Systems "joint" badge, appears below every page's footer */
.ifx-ss-badge {
margin-top: 3.5rem;
padding: 0;
@ -337,7 +337,7 @@
* ============================================================ */
@media (max-width: 640px) {
/* Defensive guard against any descendant forcing horizontal scroll
/* Defensive guard against any descendant forcing horizontal scroll:
the wire-dump's white-space: pre content was overflowing the
hero column and propagating up to the page. overflow-x: hidden
on .ifx-hero contains it without affecting page-level scroll. */

View File

@ -1,7 +1,7 @@
/*
* informix-driver docs theme
* - Charcoal base (no purple gradients, ever)
* - Amber accent CRT-monitor nod, distinct from sibling sites' cyan
* - Amber accent, a CRT-monitor nod distinct from sibling sites' cyan
* - Inter for body, IBM Plex Mono for technical bytes
*/
@ -82,7 +82,7 @@
--sl-color-hairline-shade: rgba(120, 80, 12, 0.28);
}
/* Tighten heading rhythm Starlight defaults are a touch loose for technical docs */
/* Tighten heading rhythm; Starlight defaults are a touch loose for technical docs */
.sl-markdown-content h2 {
margin-top: 2.5rem;
border-top: 1px solid var(--sl-color-hairline-light);
@ -104,7 +104,7 @@
border-radius: 4px;
}
/* Tables: dense, technical, with amber column rules — for type-mapping & benchmark tables */
/* Tables: dense, technical, amber column rules, for type-mapping & benchmark tables */
.sl-markdown-content table {
border-collapse: collapse;
font-variant-numeric: tabular-nums;
@ -124,7 +124,7 @@
padding: 0.5rem 0.75rem;
}
/* Anchor links underline, no rainbow */
/* Anchor links: underline, no rainbow */
.sl-markdown-content a:not(.sl-anchor-link) {
text-decoration: underline;
text-decoration-color: var(--sl-color-accent);

View File

@ -1,6 +1,6 @@
[project]
name = "informix-driver"
version = "2026.09.02"
version = "2026.09.03"
description = "Pure-Python driver for IBM Informix IDS — speaks the SQLI wire protocol over raw sockets. No CSDK, no JVM, no native libraries."
readme = "README.md"
license = { text = "MIT" }

View File

@ -50,6 +50,7 @@ from .pool import (
PoolTimeoutError,
create_pool,
)
from .rows import Row
# PEP 249 module-level globals
apilevel = "2.0"
@ -85,6 +86,7 @@ __all__ = [
"PoolClosedError",
"PoolTimeoutError",
"ProgrammingError",
"Row",
"RowValue",
"ServerCapabilities",
"Warning",
@ -111,6 +113,7 @@ def connect(
client_locale: str = "en_US.8859-1",
env: dict[str, str] | None = None,
autocommit: bool = False,
row_factory: object | None = None,
tls: bool | ssl.SSLContext = False,
tls_server_hostname: str | None = None,
) -> Connection:
@ -153,4 +156,5 @@ def connect(
client_locale=client_locale,
env=env,
autocommit=autocommit,
row_factory=row_factory,
)

View File

@ -283,6 +283,7 @@ class Connection:
client_locale: str = "en_US.8859-1",
env: dict[str, str] | None = None,
autocommit: bool = False, # honored from Phase 3 onward
row_factory: object | None = None,
tls: bool | ssl.SSLContext = False,
tls_server_hostname: str | None = None,
):
@ -337,6 +338,10 @@ class Connection:
# before the next DML in non-autocommit mode. We default to "no
# open txn" — the first DML will trigger SQ_BEGIN.
self._in_transaction = False
# Default row type for cursors from this connection. None means
# plain tuples, which is the zero-cost default; see
# informix_db.rows for the opt-in named-access type.
self.row_factory = row_factory
# Tri-state: True after first successful SQ_BEGIN, False after
# an unlogged-DB rejection (-201). None until we've tried.
# Used to avoid repeatedly probing on unlogged DBs.

View File

@ -50,6 +50,7 @@ from .exceptions import (
NotSupportedError,
ProgrammingError,
)
from .rows import Row, make_row_class
if TYPE_CHECKING:
from .connections import Connection
@ -60,7 +61,6 @@ if TYPE_CHECKING:
_cursor_counter = itertools.count(1)
_NUMERIC_PLACEHOLDER_RE = __import__("re").compile(r":(\d+)")
# Phase 28: pre-built CLOSE and RELEASE PDU bytes for cursor finalizers.
@ -247,17 +247,103 @@ def _finalize_cursor(
conn._wire_lock.release()
_ASCII_DIGITS = frozenset("0123456789")
def _rewrite_numeric_to_qmark(sql: str) -> str:
"""Convert ``:1`` / ``:2`` placeholders (paramstyle="numeric") to ``?``.
Informix's wire protocol uses ``?`` natively. Since we expose
``paramstyle="numeric"`` in the public API (matches Informix
ESQL/C convention), we rewrite before sending. Trivial cases only
strings and comments are NOT escaped, so SQL containing literal
``:1`` inside string literals will be wrongly substituted. Phase 5
can add a proper SQL tokenizer.
Informix's wire protocol uses ``?`` natively, and we advertise
``paramstyle="numeric"`` to match the ESQL/C convention, so the
placeholders are rewritten on the way out.
This used to be ``re.sub(r":(\\d+)", "?", sql)``, which cannot see a
string literal and so rewrote the contents of one. Any ``HH:MM``
time, any URL with a port, any aspect ratio::
UPDATE jobs SET url = 'http://host:8080/x' WHERE id = ?
stored as 'http://host?/x'
Nothing about that is opt-in. The rewrite runs whenever a statement
has parameters, whatever placeholder style the caller actually used,
so writing ``?`` everywhere and never touching numeric style did not
protect you. It also changed the placeholder *count* while
``num_qmarks`` was still computed from ``len(params)``, leaving the
driver and the server disagreeing about how many binds exist.
So: a single pass that substitutes only outside quotes and comments.
The lexical rules are Informix's own, measured against 12.10, 14.10
and 15 rather than assumed from standard SQL:
* ``''`` doubling escapes a quote inside ``'...'``. A backslash does
**not** escape anything; ``'a\'b'`` is an unterminated string and
the server answers ``-282``. A Postgres-style scanner that honours
``\'`` would desync here and corrupt everything after it.
* ``"..."`` is a delimited identifier or a string depending on
``DELIMIDENT``. Either way its contents are not ours to touch.
* ``--`` runs to end of line, ``/* */`` does **not** nest (the first
``*/`` closes it; nesting is a syntax error), and ``{ }`` is a
comment too.
* ``::`` is the cast operator and is stepped over as a unit, so it
can never be read as the start of a placeholder.
An unterminated quote or comment consumes the rest of the string and
substitutes nothing further. That is deliberate: under-substituting
leaves the server to reject SQL that was already malformed, while
guessing would corrupt a literal.
"""
return _NUMERIC_PLACEHOLDER_RE.sub("?", sql)
if ":" not in sql:
return sql
out: list[str] = []
i = 0
n = len(sql)
while i < n:
ch = sql[i]
if ch in ("'", '"'):
j = i + 1
while j < n:
if sql[j] == ch:
if j + 1 < n and sql[j + 1] == ch:
j += 2 # doubled quote, still inside
continue
j += 1
break
j += 1
out.append(sql[i:j])
i = j
elif ch == "-" and sql.startswith("--", i):
j = sql.find("\n", i)
j = n if j == -1 else j
out.append(sql[i:j])
i = j
elif ch == "/" and sql.startswith("/*", i):
j = sql.find("*/", i + 2)
j = n if j == -1 else j + 2
out.append(sql[i:j])
i = j
elif ch == "{":
j = sql.find("}", i)
j = n if j == -1 else j + 1
out.append(sql[i:j])
i = j
elif ch == ":":
if sql.startswith("::", i):
out.append("::")
i += 2
continue
j = i + 1
while j < n and sql[j] in _ASCII_DIGITS:
j += 1
if j > i + 1:
out.append("?")
i = j
else:
out.append(ch)
i += 1
else:
out.append(ch)
i += 1
return "".join(out)
def _generate_cursor_name() -> str:
@ -288,6 +374,9 @@ class Cursor:
# manipulation. Two-mode cursor; the same surface API works
# for both.
self._scrollable = scrollable
# Inherited from the connection, overridable per cursor. See
# informix_db.rows for what this costs and why it is opt-in.
self.row_factory = connection.row_factory
self._description: list[tuple] | None = None
self._columns: list[ColumnInfo] = []
self._column_readers: list[tuple] | None = None # Phase 37
@ -328,6 +417,8 @@ class Cursor:
# DESCRIBE's statement-type field. Decides whether a cursor is
# opened -- see _produces_result_set.
self._statement_type: int = 0
# Per-result-set row class when row_factory is set, else None.
self._row_class: type | None = None
# Phase 10: smart-LOB read via ``lotofile(col, path, 'client')``.
# The server orchestrates a SQ_FILE (98) protocol where it tells
# us to "open file X, write these bytes, close". We emulate the
@ -429,6 +520,7 @@ class Cursor:
self._rowcount = -1
self._rows = []
self._row_index = -1 # before-first-row
self._row_class = None
self._statement_type = 0
self._statement_already_done = False
@ -468,6 +560,8 @@ class Cursor:
else:
self._execute_dml()
self._row_class = self._resolve_row_class()
# The statement succeeded. If it was transaction control, the
# server's transaction state just changed and the connection has
# to know, or commit() and rollback() silently do nothing.
@ -481,6 +575,25 @@ class Cursor:
if self._description is not None:
self._row_index = -1
def _resolve_row_class(self) -> type | None:
"""Pick the class this result set's rows are handed back as.
``row_factory`` is either :class:`informix_db.Row` (or a subclass),
in which case the per-shape class is built and cached from the
column names, or any callable taking the name tuple and returning
something that takes a values tuple.
Returns ``None`` when no factory is set, which is the default and
keeps plain tuples on the hot path at zero cost.
"""
factory = self.row_factory
if factory is None or self._description is None:
return None
names = tuple(d[0] for d in self._description)
if isinstance(factory, type) and issubclass(factory, Row):
return make_row_class(names)
return factory(names)
def _note_transaction_control(self) -> None:
"""Sync the connection's transaction flag after a successful execute.
@ -1302,7 +1415,8 @@ class Cursor:
self._row_index = len(self._rows) # past-last
return None
self._row_index = nxt
return self._rows[nxt]
row = self._rows[nxt]
return self._row_class(row) if self._row_class is not None else row
def fetchmany(self, size: int | None = None) -> list[tuple]:
self._check_open()
@ -1333,6 +1447,8 @@ class Cursor:
return []
start = self._row_index + 1
out = self._rows[start:]
if self._row_class is not None:
out = [self._row_class(r) for r in out]
self._row_index = len(self._rows)
return list(out)
@ -1513,7 +1629,7 @@ class Cursor:
if scrolltype == 4 or is_last_probe:
# SFETCH(LAST) — TUPID == total row count
self._scroll_total_rows = self._last_tupid
return row
return self._row_class(row) if self._row_class is not None else row
def close(self) -> None:
"""Close the cursor.

180
src/informix_db/rows.py Normal file
View File

@ -0,0 +1,180 @@
"""Rows that can be read by position, by column name, or by attribute.
PEP 249 only requires a sequence, and a sequence is what the driver
returns by default. That is fine for ``SELECT a, b`` and steadily worse
as the projection grows: ``row[11]`` tells a reader nothing, and stays
correct only until somebody adds a column in the middle.
Opting in with ``row_factory=Row`` gives the shape ``pyodbc`` and
``mssql-python`` provide, all three at once::
conn = informix_db.connect(..., row_factory=informix_db.Row)
cur.execute("SELECT tabid, tabname FROM systables")
row = cur.fetchone()
row[0], row["tabname"], row.tabname
It is opt-in rather than the default because it is not free, and the
driver's whole argument is that pure Python can stay within noise of the
C driver on bulk fetch. Measured on a 20,000-row five-column fetch:
**37.2 ms with tuples, 40.8 ms with Row**, so about 9%. Defaulting it on
would move the published 1.05-1.15x ratio against IfxPy to roughly
1.15-1.25x, which is not a trade to make on everybody's behalf.
Where the 9% goes, against a plain tuple: about 30 ns per row to
construct, about 39 ns per ``row[0]`` because supporting ``row["name"]``
means ``__getitem__`` is a Python method rather than C-level tuple
indexing, and about 13 ns per unpack since CPython's fast path for
``a, b = row`` applies to exact tuples and not to subclasses.
Worth it for readability in application code. Not worth paying in a bulk
export that never looks at a column by name, which is exactly why it is
a choice.
``Row`` subclasses ``tuple``, so ``row == (1, "x")`` is still true and
existing code keeps working unchanged.
**A column always beats a method of the same name.** ``tuple`` defines
``count`` and ``index``; this class adds ``keys``, ``_asdict`` and
``_fields``. A column named any of those would otherwise resolve to the
method and hand back a bound method instead of a value, which is exactly
the shape of the framing bugs this driver spent a fortnight removing: a
plausible-looking wrong answer, in silence. So every such name gets a
descriptor and the column wins, and the reserved set is *computed* from
the class rather than hand-listed, because hand-listing it is what
missed ``keys``, ``_asdict`` and ``_fields`` the first time round. The
methods stay reachable through the class: ``tuple.count(row, x)``,
``Row.keys(row)``.
The machinery itself is name-mangled (``__fields`` / ``__map``) so that
shadowing ``_fields`` cannot break ``repr`` or ``_asdict``.
Column names come from ``cursor.description``. Informix folds unquoted
identifiers to lower case, so ``SELECT Config_Key`` is reachable as
``row.config_key``. Expressions get server-generated names that are not
Python identifiers, like ``(count(*))``; those are reachable by
subscript but not as attributes. Duplicate names resolve to the first
occurrence, matching ``pyodbc``. Dunder names cannot be shadowed and
stay subscript-only, which no real schema should notice.
"""
from __future__ import annotations
import operator
from functools import lru_cache
from typing import ClassVar
__all__ = ["Row", "make_row_class"]
class Row(tuple):
"""A result row addressable by position, name, or attribute.
Used as a ``row_factory``. The concrete class handed to each result
set is a subclass carrying that query's column names, built by
:func:`make_row_class`.
"""
__slots__ = ()
# Name-mangled to ``_Row__fields`` / ``_Row__map`` so that a column
# called "_fields" can be shadowed without breaking the machinery
# that reads it. Set on the per-result-set subclass.
__fields: ClassVar[tuple[str, ...]] = ()
__map: ClassVar[dict[str, int]] = {}
def __getitem__(self, key):
# ``key.__class__ is str`` rather than isinstance: this runs on
# every subscript.
if key.__class__ is str:
try:
return tuple.__getitem__(self, self.__map[key])
except KeyError:
raise KeyError(
f"no column named {key!r}; this row has "
f"{list(self.__fields)}"
) from None
return tuple.__getitem__(self, key)
def __getattr__(self, name):
# Only reached when normal attribute lookup has already failed,
# so this costs nothing for names that do not collide.
try:
return tuple.__getitem__(self, self.__map[name])
except KeyError:
raise AttributeError(
f"no column named {name!r}; this row has "
f"{list(self.__fields)}"
) from None
@property
def _fields(self) -> tuple[str, ...]:
"""Column names, in select order. Mirrors ``namedtuple._fields``."""
return self.__fields
def keys(self) -> tuple[str, ...]:
"""Column names, in select order."""
return self.__fields
def _asdict(self) -> dict:
"""A plain ``dict`` of the row.
On duplicate column names the last occurrence wins here, while
subscript access gives the first. A dict cannot represent both,
and quietly dropping a duplicate is better than raising on a
query that is otherwise fine.
"""
return dict(zip(self.__fields, self, strict=True))
def __repr__(self) -> str:
fields = self.__fields
if len(fields) != len(self):
return tuple.__repr__(self)
body = ", ".join(
f"{name}={value!r}"
for name, value in zip(fields, self, strict=True)
)
return f"Row({body})"
def __reduce__(self):
# The per-result-set class is created at runtime and cannot be
# pickled by reference, so rebuild it from the field names.
return (_rebuild_row, (self.__fields, tuple(self)))
# Every non-dunder attribute a Row already answers to. A column with one
# of these names gets a descriptor so the column wins. Computed, not
# hand-listed: the hand-listed version covered ``count`` and ``index``
# and silently missed ``keys``, ``_asdict`` and ``_fields``.
_RESERVED = frozenset(
name for name in dir(Row) if not name.startswith("__")
) - {"_Row__fields", "_Row__map"}
def _rebuild_row(fields: tuple[str, ...], values: tuple):
return make_row_class(fields)(values)
@lru_cache(maxsize=256)
def make_row_class(fields: tuple[str, ...]) -> type[Row]:
"""Build (and cache) the row class for one column-name shape.
Cached because a class per ``execute()`` would put a ``type()`` call
on the path of every small query, and applications run the same
handful of statement shapes over and over. Keyed on the names alone,
so two queries selecting the same columns share a class.
"""
namespace: dict = {
"__slots__": (),
"_Row__fields": fields,
# First occurrence wins on duplicates, matching pyodbc. Building
# the map in reverse and letting earlier entries overwrite later
# ones is the shortest way to say that.
"_Row__map": {
name: i for i, name in reversed(list(enumerate(fields)))
},
}
for i, name in enumerate(fields):
if name in _RESERVED:
# A column of this name would otherwise resolve to a method.
namespace[name] = property(operator.itemgetter(i))
return type("Row", (Row,), namespace)

View File

@ -214,3 +214,52 @@ async def test_closing_a_connection_stops_its_thread(
f"connection threads outlived their connections "
f"({before} -> {threading.active_count()})"
)
@pytest.mark.asyncio
async def test_row_factory_reaches_the_async_paths(
conn_params: ConnParams,
) -> None:
"""The async layer wraps the sync cursor, so row_factory should flow
through untouched. Each of the four async fetch routes is a separate
call site, and async iteration goes through __anext__ rather than
__next__."""
import informix_db
conn = await aio.connect(row_factory=informix_db.Row, **_kw(conn_params))
try:
cur = await conn.cursor()
sql = "SELECT FIRST 2 tabid, tabname FROM systables ORDER BY tabid"
await cur.execute(sql)
one = await cur.fetchone()
assert one[0] == one["tabid"] == one.tabid
await cur.execute(sql)
assert [r.tabid for r in await cur.fetchall()] == [1, 2]
await cur.execute(sql)
assert all(r.tabname for r in await cur.fetchmany(2))
await cur.execute(sql)
assert [r.tabid async for r in cur] == [1, 2]
finally:
await conn.close()
@pytest.mark.asyncio
async def test_async_pool_forwards_the_row_factory(
conn_params: ConnParams,
) -> None:
import informix_db
pool = await aio.create_pool(
row_factory=informix_db.Row, min_size=1, max_size=2, **_kw(conn_params)
)
try:
async with pool.connection() as conn:
cur = await conn.cursor()
await cur.execute("SELECT FIRST 1 tabid FROM systables")
assert (await cur.fetchone()).tabid == 1
finally:
await pool.close()

View File

@ -0,0 +1,248 @@
"""Rewriting :N placeholders without rewriting the SQL around them.
We advertise ``paramstyle="numeric"`` to match Informix's ESQL/C
convention, and the wire protocol takes ``?``, so placeholders get
rewritten on the way out. That was ``re.sub(r":(\\d+)", "?", sql)``,
which cannot see a string literal and therefore rewrote the inside of
one::
UPDATE jobs SET url = 'http://host:8080/x' WHERE id = ?
stored as 'http://host?/x'
Any ``HH:MM`` time, any URL with a port, any aspect ratio, any
``key:value`` string. It wrote wrong data and said nothing.
Nothing about it was opt-in either. The rewrite runs whenever a
statement has parameters, whatever placeholder style the caller actually
used, so writing ``?`` everywhere and never touching numeric style did
not protect you. And it changed the placeholder *count* while
``num_qmarks`` was still computed from ``len(params)``, leaving driver
and server disagreeing about how many binds exist.
The lexical rules below are Informix's own, measured against 12.10,
14.10 and 15 rather than assumed from standard SQL. The one that matters
most is the backslash: ``'a\\'b'`` is an *unterminated string* to
Informix (``-282``), not an escaped quote. A scanner written to
Postgres habits would desync on it and corrupt everything after.
"""
from __future__ import annotations
import contextlib
import pytest
import informix_db
from informix_db.cursors import _rewrite_numeric_to_qmark as rewrite
from tests.conftest import ConnParams
# ---------------------------------------------------------------------------
# Substitution happens where it should
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
("sql", "expected"),
[
("SELECT * FROM t WHERE a = :1", "SELECT * FROM t WHERE a = ?"),
(
"SELECT * FROM t WHERE a = :1 AND b = :2",
"SELECT * FROM t WHERE a = ? AND b = ?",
),
("SELECT * FROM t WHERE a = :10", "SELECT * FROM t WHERE a = ?"),
# Already-? SQL is returned untouched, and cheaply.
("SELECT * FROM t WHERE a = ?", "SELECT * FROM t WHERE a = ?"),
("", ""),
],
)
def test_placeholders_are_rewritten(sql: str, expected: str) -> None:
assert rewrite(sql) == expected
# ---------------------------------------------------------------------------
# ...and nowhere else
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
("label", "sql", "expected"),
[
(
"url-with-port",
"UPDATE jobs SET url = 'http://host:8080/x' WHERE id = :1",
"UPDATE jobs SET url = 'http://host:8080/x' WHERE id = ?",
),
(
"time-of-day",
"INSERT INTO log VALUES ('started at 09:15:30', :1)",
"INSERT INTO log VALUES ('started at 09:15:30', ?)",
),
(
"aspect-ratio",
"UPDATE t SET ratio = '16:9' WHERE id = :1",
"UPDATE t SET ratio = '16:9' WHERE id = ?",
),
(
"doubled-quote-escape",
"SELECT 'it''s 10:30' FROM t WHERE a = :1",
"SELECT 'it''s 10:30' FROM t WHERE a = ?",
),
(
"delimited-identifier",
'SELECT "col:1" FROM t WHERE a = :1',
'SELECT "col:1" FROM t WHERE a = ?',
),
(
"line-comment",
"-- ticket :99\nSELECT * FROM t WHERE a = :1",
"-- ticket :99\nSELECT * FROM t WHERE a = ?",
),
(
"block-comment",
"SELECT /* not :99 */ * FROM t WHERE a = :1",
"SELECT /* not :99 */ * FROM t WHERE a = ?",
),
(
"brace-comment",
"SELECT { not :99 } * FROM t WHERE a = :1",
"SELECT { not :99 } * FROM t WHERE a = ?",
),
(
"cast-operator",
"SELECT a::INT FROM t WHERE a = :1",
"SELECT a::INT FROM t WHERE a = ?",
),
(
"colon-not-a-placeholder",
"SELECT a FROM t WHERE b = ':x' AND c = :1",
"SELECT a FROM t WHERE b = ':x' AND c = ?",
),
(
"literal-after-placeholder",
"SELECT * FROM t WHERE a = :1 AND b = '10:30'",
"SELECT * FROM t WHERE a = ? AND b = '10:30'",
),
],
)
def test_quotes_and_comments_are_left_alone(
label: str, sql: str, expected: str
) -> None:
assert rewrite(sql) == expected, label
def test_backslash_does_not_escape_a_quote() -> None:
"""Informix answers -282 for ``'a\\'b'``: the backslash is an ordinary
character and the string is unterminated. A scanner that treated it
as an escape would think it was still inside the literal and stop
substituting, or worse, resume in the wrong place."""
sql = "SELECT 'a\\' FROM t WHERE x = :1"
# The quote closes at the character after the backslash, so :1 is
# outside the literal and gets substituted.
assert rewrite(sql) == "SELECT 'a\\' FROM t WHERE x = ?"
def test_unterminated_quote_substitutes_nothing_further() -> None:
"""Under-substituting leaves the server to reject SQL that was
already malformed. Guessing would corrupt a literal."""
assert rewrite("SELECT 'oops :1 FROM t") == "SELECT 'oops :1 FROM t"
def test_unterminated_block_comment_substitutes_nothing_further() -> None:
assert rewrite("SELECT /* oops :1 FROM t") == "SELECT /* oops :1 FROM t"
def test_block_comments_do_not_nest() -> None:
"""Measured: ``/* a /* b */ c */`` is a syntax error on all three
servers, so the first ``*/`` closes the comment. Treating them as
nesting would swallow live SQL."""
assert (
rewrite("SELECT /* a /* b */ :1 FROM t")
== "SELECT /* a /* b */ ? FROM t"
)
# ---------------------------------------------------------------------------
# Against a real server
# ---------------------------------------------------------------------------
def _connect(conn_params: ConnParams) -> informix_db.Connection:
return informix_db.connect(
host=conn_params.host,
port=conn_params.port,
user=conn_params.user,
password=conn_params.password,
database=conn_params.database,
server=conn_params.server,
connect_timeout=10.0,
read_timeout=25.0,
autocommit=True,
)
@pytest.mark.integration
def test_colon_literals_round_trip(conn_params: ConnParams) -> None:
"""The bug as a user meets it: the value stored is not the value
written. Both placeholder styles, because the rewrite ran for both."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("CREATE TEMP TABLE t_rw (id INT, s VARCHAR(60))")
cur.execute(
"INSERT INTO t_rw VALUES (?, 'http://host:8080/path')", (1,)
)
cur.execute("INSERT INTO t_rw VALUES (:1, 'at 09:15:30')", (2,))
cur.execute("SELECT id, s FROM t_rw ORDER BY id")
assert cur.fetchall() == [
(1, "http://host:8080/path"),
(2, "at 09:15:30"),
]
@pytest.mark.integration
def test_colon_literals_round_trip_through_executemany(
conn_params: ConnParams,
) -> None:
"""executemany rewrites unconditionally, so it had the bug too."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("CREATE TEMP TABLE t_rw2 (id INT, s VARCHAR(40))")
cur.executemany(
"INSERT INTO t_rw2 VALUES (?, '16:9')", [(1,), (2,)]
)
cur.execute("SELECT DISTINCT s FROM t_rw2")
assert cur.fetchall() == [("16:9",)]
@pytest.mark.integration
def test_comment_bearing_sql_still_binds(conn_params: ConnParams) -> None:
"""A leading comment reaches PREPARE now that classification asks the
server. Make sure the rewriter agrees and doesn't eat a placeholder
that follows one."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute(
"/* report: daily */ SELECT FIRST 1 tabid FROM systables "
"WHERE tabid > :1",
(0,),
)
assert cur.fetchone() is not None
@pytest.mark.integration
def test_placeholder_count_matches_after_rewrite(
conn_params: ConnParams,
) -> None:
"""The old regex could add placeholders the driver never counted,
leaving num_qmarks (from len(params)) disagreeing with the SQL. A
literal containing three colon-digit sequences is the shape that
used to break it."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("CREATE TEMP TABLE t_rw3 (id INT, s VARCHAR(40))")
with contextlib.suppress(Exception):
cur.execute("DELETE FROM t_rw3")
cur.execute(
"INSERT INTO t_rw3 VALUES (:1, 'a:1 b:2 c:3')", (7,)
)
cur.execute("SELECT id, s FROM t_rw3")
assert cur.fetchone() == (7, "a:1 b:2 c:3")

435
tests/test_rows.py Normal file
View File

@ -0,0 +1,435 @@
"""Named row access, and the cost of it.
A field request: `row[11]` tells a reader nothing on a wide projection,
and `pyodbc` / `mssql-python` both hand back rows that answer to
position, column name, and attribute at once. `row_factory=Row` gives
the same three.
It is opt-in. Defaulting it on would tax the thing this driver is
measured against, since supporting `row["name"]` means `__getitem__`
becomes a Python method rather than C-level tuple indexing, which costs
roughly 39 ns on every subscript. Users who want readable column access
in application code should pay that; a bulk export that never looks at a
column by name should not.
`Row` subclasses `tuple`, so `row == (1, "x")` still holds and nothing
that already worked stops working. That constraint drove the design: the
existing suite compares fetched rows against plain tuples in hundreds of
places, and so does everybody's code.
"""
from __future__ import annotations
import contextlib
import pickle
import pytest
import informix_db
from informix_db.rows import _RESERVED, Row, make_row_class
from tests.conftest import ConnParams
# ---------------------------------------------------------------------------
# The type itself
# ---------------------------------------------------------------------------
def _row(fields, values):
return make_row_class(tuple(fields))(values)
def test_three_ways_to_reach_a_column() -> None:
row = _row(("tabid", "tabname"), (1, "systables"))
assert row[0] == 1
assert row["tabname"] == "systables"
assert row.tabname == "systables"
def test_still_equal_to_a_plain_tuple() -> None:
"""The compatibility constraint. Existing code and the existing test
suite compare fetched rows against tuples everywhere."""
row = _row(("a", "b"), (1, "x"))
plain = (1, "x")
assert row == plain
# Both directions: tuple.__eq__ on the left has to accept a subclass
# on the right, or `expected == fetched` assertions break.
assert plain == row
assert list(row) == [1, "x"]
assert len(row) == 2
a, b = row
assert (a, b) == (1, "x")
assert row in [(1, "x")]
def test_slice_degrades_to_a_plain_tuple() -> None:
"""A slice has no meaningful column mapping, so it should not pretend
to be a Row."""
row = _row(("a", "b", "c"), (1, 2, 3))
assert row[0:2] == (1, 2)
assert type(row[0:2]) is tuple
def test_negative_index_still_works() -> None:
assert _row(("a", "b"), (1, 2))[-1] == 2
def test_keys_and_asdict() -> None:
row = _row(("a", "b"), (1, "x"))
assert row.keys() == ("a", "b")
assert row._asdict() == {"a": 1, "b": "x"}
def test_repr_names_the_columns() -> None:
assert repr(_row(("a", "b"), (1, "x"))) == "Row(a=1, b='x')"
def test_pickles() -> None:
"""The per-shape class is built at runtime, so it cannot be pickled by
reference. Multiprocessing users would hit that immediately."""
row = _row(("a", "b"), (1, "x"))
restored = pickle.loads(pickle.dumps(row))
assert restored == (1, "x")
assert restored.b == "x"
def test_missing_column_says_what_is_there() -> None:
row = _row(("tabid", "tabname"), (1, "x"))
with pytest.raises(KeyError, match="tabid"):
_ = row["nope"]
with pytest.raises(AttributeError, match="tabid"):
_ = row.nope
def test_class_is_cached_per_shape() -> None:
"""A type() call per execute() would land on every small query."""
assert make_row_class(("a", "b")) is make_row_class(("a", "b"))
assert make_row_class(("a", "b")) is not make_row_class(("a", "c"))
def test_duplicate_names_resolve_to_the_first() -> None:
"""``SELECT tabid, tabid`` is legal and both columns are named
``tabid``. pyodbc gives the first; so do we."""
row = _row(("tabid", "tabid"), (1, 2))
assert row["tabid"] == 1
assert row.tabid == 1
assert row[1] == 2, "positional access must still reach the second"
@pytest.mark.parametrize("name", sorted(_RESERVED))
def test_a_column_always_beats_a_method_of_the_same_name(name: str) -> None:
"""Parametrized over the *computed* reserved set rather than a
hand-written list, because the hand-written list is what missed
``keys``, ``_asdict`` and ``_fields`` on the first attempt. Adding a
method to Row later cannot silently reopen the hole: this test grows
with it.
Without the guard these return a bound method, which is the exact
failure shape this driver spent a fortnight removing from its
decoders: a plausible-looking wrong answer, in silence."""
row = _row((name, "other"), (7, 9))
assert getattr(row, name) == 7
assert row[name] == 7
def test_shadowed_methods_stay_reachable_through_the_base_class() -> None:
row = _row(("count", "keys", "_asdict"), (1, 2, 3))
assert tuple.count(row, 1) == 1
assert Row.keys(row) == ("count", "keys", "_asdict")
assert Row._asdict(row) == {"count": 1, "keys": 2, "_asdict": 3}
def test_shadowing_fields_does_not_break_repr_or_asdict() -> None:
"""The machinery reads its own field list, so if a column named
``_fields`` shadowed it, repr and _asdict would report the column
value instead of the names. Mangled attributes keep them separate."""
row = _row(("_fields", "x"), ("not the names", 2))
assert row._fields == "not the names"
assert Row.keys(row) == ("_fields", "x")
assert repr(row) == "Row(_fields='not the names', x=2)"
def test_non_identifier_names_are_subscript_only() -> None:
"""Informix names expression columns things like ``(count(*))``,
which cannot be an attribute."""
row = _row(("(count(*))", "ok"), (5, 1))
assert row["(count(*))"] == 5
assert row.ok == 1
# ---------------------------------------------------------------------------
# Against a real server
# ---------------------------------------------------------------------------
def _connect(conn_params: ConnParams, **kw) -> informix_db.Connection:
return informix_db.connect(
host=conn_params.host,
port=conn_params.port,
user=conn_params.user,
password=conn_params.password,
database=conn_params.database,
server=conn_params.server,
connect_timeout=10.0,
read_timeout=25.0,
autocommit=True,
**kw,
)
@pytest.mark.integration
def test_default_is_still_a_plain_tuple(conn_params: ConnParams) -> None:
"""The opt-in has to be genuinely opt-in. Anyone who does not ask for
Row should not pay for it or see any behaviour change."""
with _connect(conn_params) as conn:
cur = conn.cursor()
cur.execute("SELECT FIRST 1 tabid, tabname FROM systables")
row = cur.fetchone()
assert type(row) is tuple
with pytest.raises(TypeError):
_ = row["tabname"]
@pytest.mark.integration
def test_row_factory_on_the_connection(conn_params: ConnParams) -> None:
"""One line at connect time, then every cursor from it, which is what
'without any additional coding' has to mean in practice."""
with _connect(conn_params, row_factory=informix_db.Row) as conn:
cur = conn.cursor()
cur.execute(
"SELECT FIRST 1 tabid, tabname FROM systables ORDER BY tabid"
)
row = cur.fetchone()
assert row[0] == row["tabid"] == row.tabid
assert row[1] == row["tabname"] == row.tabname
@pytest.mark.integration
def test_every_fetch_path_returns_rows(conn_params: ConnParams) -> None:
"""fetchone, fetchmany, fetchall and iteration each return rows by a
different route through the cursor."""
with _connect(conn_params, row_factory=informix_db.Row) as conn:
sql = "SELECT FIRST 4 tabid, tabname FROM systables ORDER BY tabid"
cur = conn.cursor()
cur.execute(sql)
assert cur.fetchone().tabname is not None
cur.execute(sql)
assert all(r.tabname is not None for r in cur.fetchmany(2))
cur.execute(sql)
assert all(r.tabname is not None for r in cur.fetchall())
cur.execute(sql)
assert all(r.tabname is not None for r in cur)
@pytest.mark.integration
def test_scrollable_cursor_returns_rows(conn_params: ConnParams) -> None:
"""Scrollable cursors return each row straight from the wire rather
than from the materialized list, so they are a separate path."""
with _connect(conn_params, row_factory=informix_db.Row) as conn:
cur = conn.cursor(scrollable=True)
cur.execute("SELECT tabid, tabname FROM systables ORDER BY tabid")
assert cur.fetch_first().tabname is not None
assert cur.fetch_absolute(1).tabname is not None
cur.close()
@pytest.mark.integration
def test_per_cursor_override(conn_params: ConnParams) -> None:
with _connect(conn_params) as conn:
named = conn.cursor()
named.row_factory = informix_db.Row
plain = conn.cursor()
sql = "SELECT FIRST 1 tabid FROM systables"
named.execute(sql)
plain.execute(sql)
assert isinstance(named.fetchone(), Row)
assert type(plain.fetchone()) is tuple
@pytest.mark.integration
def test_informix_lowercases_so_the_obvious_name_works(
conn_params: ConnParams,
) -> None:
"""Informix folds unquoted identifiers, which is why the .lower() in
the workaround people write by hand is a no-op."""
with _connect(conn_params, row_factory=informix_db.Row) as conn:
cur = conn.cursor()
cur.execute("CREATE TEMP TABLE t_rows (Config_Key INT, Cnt INT)")
cur.execute("INSERT INTO t_rows VALUES (1, 2)")
cur.execute("SELECT Config_Key, Cnt FROM t_rows")
row = cur.fetchone()
assert row.config_key == 1
assert row.cnt == 2
@pytest.mark.integration
def test_pool_forwards_the_factory(conn_params: ConnParams) -> None:
pool = informix_db.create_pool(
host=conn_params.host,
port=conn_params.port,
user=conn_params.user,
password=conn_params.password,
database=conn_params.database,
server=conn_params.server,
autocommit=True,
row_factory=informix_db.Row,
min_size=1,
max_size=2,
)
try:
with pool.connection() as conn:
cur = conn.cursor()
cur.execute("SELECT FIRST 1 tabid FROM systables")
assert cur.fetchone().tabid is not None
finally:
pool.close()
def test_zero_column_row() -> None:
row = make_row_class(())(())
assert row == ()
assert row.keys() == ()
assert repr(row) == "Row()"
def test_wide_row() -> None:
"""500 columns: the name map is a dict, so this should be flat, but
it is the shape most likely to expose an off-by-one."""
fields = tuple(f"c{i}" for i in range(500))
row = make_row_class(fields)(tuple(range(500)))
assert row["c499"] == row.c499 == row[499] == 499
assert row["c0"] == row[0] == 0
def test_names_that_cannot_be_attributes_are_subscript_only() -> None:
row = _row(("col with space", "1leading_digit", ""), (1, 2, 3))
assert row["col with space"] == 1
assert row["1leading_digit"] == 2
assert row[""] == 3
def test_unicode_column_name() -> None:
row = _row(("café", "x"), (1, 2))
assert row["café"] == 1
assert row.café == 1
def test_dunder_named_column_is_subscript_only() -> None:
"""A dunder cannot be shadowed without breaking the object protocol,
so the column is reachable by subscript and the attribute keeps its
ordinary meaning. No real schema should notice."""
row = _row(("__class__", "ok"), (1, 2))
assert row["__class__"] == 1
assert row.__class__.__name__ == "Row"
def test_row_outlives_eviction_of_its_class_from_the_cache() -> None:
"""The class cache is bounded. A row already handed to the caller
holds its class alive, so eviction must not affect it."""
row = make_row_class(("z1", "z2"))((9, 8))
for i in range(300):
make_row_class((f"evict{i}", "b"))
assert (row.z1, row["z2"], row[0]) == (9, 8, 9)
def test_class_cache_is_bounded() -> None:
for i in range(400):
make_row_class((f"bounded{i}", "b"))
assert make_row_class.cache_info().currsize <= 256
# ---------------------------------------------------------------------------
# Row must not change a single value
# ---------------------------------------------------------------------------
_WIDE_DDL = """CREATE TABLE t_rowdiff (
c_int INT, c_small SMALLINT, c_big BIGINT, c_int8 INT8, c_serial SERIAL,
c_float FLOAT, c_smallfloat SMALLFLOAT, c_dec DECIMAL(16,4),
c_decu DECIMAL(16), c_money MONEY(12,2), c_char CHAR(10),
c_vchar VARCHAR(40), c_nchar NCHAR(8), c_lvar LVARCHAR(200), c_date DATE,
c_dt DATETIME YEAR TO FRACTION(5), c_ivl INTERVAL YEAR TO MONTH,
c_bool BOOLEAN, c_set SET(INT NOT NULL), c_tail INT)"""
_WIDE_COLS = (
"c_int,c_small,c_big,c_int8,c_serial,c_float,c_smallfloat,c_dec,c_decu,"
"c_money,c_char,c_vchar,c_nchar,c_lvar,c_date,c_dt,c_ivl,c_bool,c_set,"
"c_tail"
)
@pytest.mark.integration
def test_rows_are_value_identical_to_tuples(conn_params: ConnParams) -> None:
"""The strongest statement available: across every awkward type, with
a fully populated row and a fully NULL one, wrapping must change
nothing. If it does, Row is not a presentation layer, it is a bug."""
sql = f"SELECT {_WIDE_COLS} FROM t_rowdiff ORDER BY c_tail"
with _connect(conn_params) as conn:
cur = conn.cursor()
with contextlib.suppress(Exception):
cur.execute("DROP TABLE t_rowdiff")
cur.execute(_WIDE_DDL)
try:
cur.execute(
"INSERT INTO t_rowdiff VALUES (1,2,3,4,0,1.5,2.5,12.34,99,"
"9.99,'ch','vc','nc','lv',TODAY,CURRENT,"
"INTERVAL(1-2) YEAR TO MONTH,'t',SET{1,2},77)"
)
cur.execute(
"INSERT INTO t_rowdiff VALUES (NULL,NULL,NULL,NULL,0,NULL,"
"NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,"
"NULL,NULL,88)"
)
cur.execute(sql)
plain = cur.fetchall()
with _connect(conn_params, row_factory=informix_db.Row) as named_conn:
named = named_conn.cursor()
named.execute(sql)
rows = named.fetchall()
assert rows == plain, "wrapping changed a value"
names = _WIDE_COLS.split(",")
first = rows[0]
for i, name in enumerate(names):
assert first[i] == first[name] == getattr(first, name), name
assert rows[1]["c_int"] is None
assert rows[1].c_tail == 88
finally:
with contextlib.suppress(Exception):
cur.execute("DROP TABLE t_rowdiff")
@pytest.mark.integration
def test_threads_share_one_class_per_shape(conn_params: ConnParams) -> None:
"""The cache is process-wide and the pool hands connections to many
threads, so two threads running the same query must land on the same
class rather than racing to build competing ones."""
import threading
seen: list[type] = []
errors: list[Exception] = []
def worker() -> None:
try:
with _connect(conn_params, row_factory=informix_db.Row) as conn:
cur = conn.cursor()
for _ in range(5):
cur.execute(
"SELECT FIRST 1 tabid, tabname FROM systables"
)
row = cur.fetchone()
assert row.tabname == row["tabname"] == row[1]
seen.append(type(row))
except Exception as exc:
errors.append(exc)
threads = [threading.Thread(target=worker) for _ in range(6)]
for t in threads:
t.start()
for t in threads:
t.join(60)
assert not errors, errors[:2]
assert len({id(c) for c in seen}) == 1, "same shape built more than once"

2
uv.lock generated
View File

@ -34,7 +34,7 @@ wheels = [
[[package]]
name = "informix-driver"
version = "2026.9.2"
version = "2026.9.3"
source = { editable = "." }
[package.optional-dependencies]