Decode the SQ_PROTOCOLS capability negotiation (2026.08.27)
Closes the last open item from the Informix 12 field report. The driver
hardcodes several wire-framing choices that SQLI actually negotiates.
Those choices are correct on every server we've measured, but "correct
as far as we know" and "checked" are different things, and a framing
mismatch corrupts rows silently.
We were already sending SQ_PROTOCOLS with the same 8-byte client offer
IBM's JDBC driver uses, and discarding the reply. Now we decode it.
New informix_db.ServerCapabilities, reachable from any connection:
conn.server_capabilities.four_byte_offset
conn.server_capabilities.varchar_var_len
conn.server_capabilities.violated_assumptions() # [] when we agree
conn.server_version
violated_assumptions() names each place we emit or parse a fixed wire
shape that is actually capability-gated. Non-empty at connect time logs
a warning naming the bit, so an untested server produces a diagnosable
complaint instead of quiet corruption.
Nothing branches on these bits yet — this is observation and validation
only. The hardcoded framing is correct on all three supported servers,
and rewriting working parse paths to be conditional without a server
that needs it trades certainty for risk.
The measurement, and why 12/14/15 are interchangeable:
15.0.1.0.3 bdbe9ffe7fb7ffef ff
14.10.FC7W1 bdbe9ffe7fb7ffef f8
12.10.FC12W1DE bdbe9ffe7fb7ffef f0
^^^^^^^^^^^^^^^^ identical
The first 64 bits are byte-identical. Those releases don't merely behave
alike, they negotiate exactly the same capability set.
Two details worth recording. The reply is NINE bytes; JDBC's
enhancedProtocolMechanism switches on case 0..7 and drops the ninth, so
its BitSet(64) never sees it — yet that dropped byte is the only part
that differs between releases. And Cap_1 in the login response is not a
server version, it's the client's declared protocol level echoed back,
which is why JDBC tests == 316 rather than >=. The version string there
is the internal one: 12.10 reports 9.56, 14.10 reports 9.59. At the
protocol level both really are 9.x servers.
This also retires isUSVER as a red herring: it's one of six bits JDBC
pre-sets for any non-zero Cap_1, and Java's BitSet.set never clears, so
it is true on every modern server regardless of the mask.
Separately, fixed __version__. The distribution was renamed informix-db
-> informix-driver on 2026-05-08 but __init__ kept looking up the old
name. importlib.metadata needs the distribution name, not the module
name, and the miss fails silently — so every install of the renamed
package has reported "0.0.0+local". It escaped notice because a stale
informix-db distribution lingered in the dev venv and answered the
query. tests/test_package_metadata.py now pins the name against
[project].name; verified it catches the bug by reintroducing it.
247/247 integration on 15, 14.10, and 12.10. 120 unit tests.
This commit is contained in:
parent
783b2bec97
commit
36478f02c4
58
CHANGELOG.md
58
CHANGELOG.md
@ -2,6 +2,64 @@
|
|||||||
|
|
||||||
All notable changes to `informix-db`. Versioning is [CalVer](https://calver.org/) — `YYYY.MM.DD` for date-based releases, `YYYY.MM.DD.N` for same-day post-releases per PEP 440.
|
All notable changes to `informix-db`. Versioning is [CalVer](https://calver.org/) — `YYYY.MM.DD` for date-based releases, `YYYY.MM.DD.N` for same-day post-releases per PEP 440.
|
||||||
|
|
||||||
|
## 2026.08.27 — Decode the SQ_PROTOCOLS capability negotiation
|
||||||
|
|
||||||
|
Closes the last open item from the Informix 12 field report. The driver hardcodes several wire-framing choices that SQLI actually *negotiates*; those choices were correct on every server we'd measured, but "correct as far as we know" and "checked" are different things, and the failure mode for a framing mismatch is silently corrupted rows.
|
||||||
|
|
||||||
|
### What changed
|
||||||
|
|
||||||
|
We were already sending `SQ_PROTOCOLS` with the same 8-byte client offer IBM's JDBC driver uses — and throwing the server's reply away. Now we decode it.
|
||||||
|
|
||||||
|
New `informix_db.ServerCapabilities`, reachable from any connection:
|
||||||
|
|
||||||
|
```python
|
||||||
|
conn = informix_db.connect(...)
|
||||||
|
caps = conn.server_capabilities
|
||||||
|
caps.four_byte_offset # describe uses 4-byte string-table/field-index widths
|
||||||
|
caps.varchar_var_len # VARCHAR is length-prefixed, not width-padded
|
||||||
|
caps.remove_64k_limit # 4-byte length prefixes in the fast-path
|
||||||
|
caps.violated_assumptions() # [] when our hardcoded framing matches
|
||||||
|
conn.server_version # from the login response
|
||||||
|
```
|
||||||
|
|
||||||
|
`violated_assumptions()` is the point of the exercise. It names each place we emit or parse a fixed wire shape that is actually capability-gated, and returns empty when the server agrees. On connect, a non-empty result logs a warning naming the specific bit. A server we've never tested now produces a diagnosable complaint instead of quiet corruption.
|
||||||
|
|
||||||
|
Nothing branches on these bits yet — this release is observation and validation only. That's deliberate: the hardcoded framing is correct on all three supported servers, and swapping working code for newly-written conditional code without a server that needs it would add risk for no benefit.
|
||||||
|
|
||||||
|
### The measurement
|
||||||
|
|
||||||
|
All three servers negotiate the same thing:
|
||||||
|
|
||||||
|
| Server | Negotiated mask |
|
||||||
|
|---|---|
|
||||||
|
| 15.0.1.0.3 | `bdbe9ffe7fb7ffef` `ff` |
|
||||||
|
| 14.10.FC7W1 | `bdbe9ffe7fb7ffef` `f8` |
|
||||||
|
| 12.10.FC12W1DE | `bdbe9ffe7fb7ffef` `f0` |
|
||||||
|
|
||||||
|
**The first eight bytes are byte-identical.** That is the root explanation for everything in the 2026.05.08.2 investigation: 12.10, 14.10, and 15 don't merely behave similarly, they negotiate exactly the same 64-bit capability set. `violated_assumptions()` returns empty on all three.
|
||||||
|
|
||||||
|
Two details worth recording:
|
||||||
|
|
||||||
|
The reply is **nine** bytes, not eight. JDBC's `enhancedProtocolMechanism` dispatches on `case 0..7` and silently discards the ninth, so its `BitSet(64)` never sees it. We decode it, because it is the *only* part of the mask that differs between releases — newer feature flags that this JDBC build predates.
|
||||||
|
|
||||||
|
`Cap_1` in the login response is **not** a server version. It's the client's own declared protocol level echoed back, which is why JDBC tests `== 316` rather than `>=`. The actual version string there is the *internal* one: Informix 12.10 reports `9.56`, 14.10 reports `9.59`. At the protocol level both really are 9.x servers, which is why the marketing version jump didn't move the wire format.
|
||||||
|
|
||||||
|
This also retires a red herring. `isUSVER` (bit 2) is one of six bits JDBC pre-sets for any non-zero `Cap_1`, and Java's `BitSet.set` never clears — so it is true on every modern server regardless of the mask. It can't explain version-specific behaviour and isn't worth chasing.
|
||||||
|
|
||||||
|
### Also fixed: `__version__` was reporting a sentinel
|
||||||
|
|
||||||
|
Found incidentally while bumping the version. The distribution was renamed `informix-db` to `informix-driver` on 2026-05-08, but `__init__.py` kept calling `version("informix-db")`. `importlib.metadata` needs the *distribution* name, not the module name, and the miss fails silently — `__version__` degrades to `"0.0.0+local"` rather than raising.
|
||||||
|
|
||||||
|
So every install of the renamed package has been reporting `0.0.0+local`. It escaped notice locally because a stale `informix-db` distribution was still present in the dev environment and answered the lookup with its own old version.
|
||||||
|
|
||||||
|
Added `tests/test_package_metadata.py`, which asserts the name passed to `importlib.metadata.version()` matches `[project].name` in `pyproject.toml`. Verified it actually catches the bug by reintroducing it.
|
||||||
|
|
||||||
|
### Tests
|
||||||
|
|
||||||
|
24 unit tests over the captured masks (no server needed) and 6 integration tests that assert live negotiation. The integration ones run on every server in `make test-matrix`, so the assumption check is re-verified on 12.10, 14.10, and 15 each time.
|
||||||
|
|
||||||
|
Full suite: **247 / 247** on all three.
|
||||||
|
|
||||||
## 2026.05.08.2 — Fix three tuple-framing bugs (BOOLEAN, NCHAR, INT8/SERIAL8)
|
## 2026.05.08.2 — Fix three tuple-framing bugs (BOOLEAN, NCHAR, INT8/SERIAL8)
|
||||||
|
|
||||||
Data-corruption fixes. All three affect **every** Informix version including 15, and two of them silently corrupt columns *after* the offending one. If your schema uses `BOOLEAN`, `NCHAR`, `INT8`, or `SERIAL8`, upgrade.
|
Data-corruption fixes. All three affect **every** Informix version including 15, and two of them silently corrupt columns *after* the offending one. If your schema uses `BOOLEAN`, `NCHAR`, `INT8`, or `SERIAL8`, upgrade.
|
||||||
|
|||||||
15
README.md
15
README.md
@ -172,7 +172,20 @@ Beyond the suite passing, a 28-type round-trip (every scalar type we support, in
|
|||||||
|
|
||||||
Earlier releases of this README claimed the protocol was "stable across modern versions" and should "work against 12.10+ unmodified." That claim turned out to be correct, but it was written before anyone had run it against 12.10, and a user lost debugging time to a version-compatibility problem that didn't exist. It's measured now. Apologies for the earlier guess.
|
Earlier releases of this README claimed the protocol was "stable across modern versions" and should "work against 12.10+ unmodified." That claim turned out to be correct, but it was written before anyone had run it against 12.10, and a user lost debugging time to a version-compatibility problem that didn't exist. It's measured now. Apologies for the earlier guess.
|
||||||
|
|
||||||
One caveat remains, and it's a real one. SQLI negotiates some wire framing through a capability exchange (`SQ_PROTOCOLS`) that this driver does not yet perform — we hardcode the modern framing. That is correct on all three servers above, but it is an assumption rather than a negotiation. If you hit corrupted rows on a server not in that table, this is the first thing to suspect; please open an issue with your server version and the `cursor.description` of the offending query.
|
### Capability negotiation
|
||||||
|
|
||||||
|
SQLI settles some wire framing through a capability exchange (`SQ_PROTOCOLS`) rather than fixing it by version. This driver decodes that exchange and checks it against the framing it emits:
|
||||||
|
|
||||||
|
```python
|
||||||
|
conn = informix_db.connect(...)
|
||||||
|
conn.server_version # 'IBM Informix Dynamic Server Version 15.0.1.0.3'
|
||||||
|
conn.server_capabilities.four_byte_offset # True
|
||||||
|
conn.server_capabilities.violated_assumptions() # [] — hardcoded framing matches
|
||||||
|
```
|
||||||
|
|
||||||
|
All three servers above negotiate an identical 64-bit capability set, and `violated_assumptions()` is empty on each. If it ever returns something, the driver logs a warning naming the specific bit at connect time — a framing mismatch otherwise shows up as silently corrupted rows, which is a miserable thing to debug.
|
||||||
|
|
||||||
|
The framing itself is still hardcoded rather than driven by those bits. That's a deliberate limit: it's correct everywhere we've measured, and rewriting working parse paths to be conditional without a server that needs it would trade certainty for risk. If you hit corrupted rows on a server outside that table, check `violated_assumptions()` first, then please open an issue with your server version, the capability mask, and the `cursor.description` of the offending query.
|
||||||
|
|
||||||
For features that need server-side configuration (smart-LOBs, logged transactions), see [`docs/DECISION_LOG.md`](https://git.supported.systems/warehack.ing/informix-db/src/branch/main/docs/DECISION_LOG.md):
|
For features that need server-side configuration (smart-LOBs, logged transactions), see [`docs/DECISION_LOG.md`](https://git.supported.systems/warehack.ing/informix-db/src/branch/main/docs/DECISION_LOG.md):
|
||||||
- Phase 7 — logged-DB transactions
|
- Phase 7 — logged-DB transactions
|
||||||
|
|||||||
@ -1,6 +1,6 @@
|
|||||||
[project]
|
[project]
|
||||||
name = "informix-driver"
|
name = "informix-driver"
|
||||||
version = "2026.05.08.2"
|
version = "2026.08.27"
|
||||||
description = "Pure-Python driver for IBM Informix IDS — speaks the SQLI wire protocol over raw sockets. No CSDK, no JVM, no native libraries."
|
description = "Pure-Python driver for IBM Informix IDS — speaks the SQLI wire protocol over raw sockets. No CSDK, no JVM, no native libraries."
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
license = { text = "MIT" }
|
license = { text = "MIT" }
|
||||||
|
|||||||
@ -23,6 +23,7 @@ from __future__ import annotations
|
|||||||
import ssl
|
import ssl
|
||||||
from importlib.metadata import PackageNotFoundError, version
|
from importlib.metadata import PackageNotFoundError, version
|
||||||
|
|
||||||
|
from ._capabilities import ServerCapabilities
|
||||||
from .connections import Connection
|
from .connections import Connection
|
||||||
from .converters import (
|
from .converters import (
|
||||||
BlobLocator,
|
BlobLocator,
|
||||||
@ -56,9 +57,14 @@ threadsafety = 1 # threads may share the module but not connections
|
|||||||
paramstyle = "numeric" # locked in DECISION_LOG.md — matches Informix ESQL/C
|
paramstyle = "numeric" # locked in DECISION_LOG.md — matches Informix ESQL/C
|
||||||
|
|
||||||
try:
|
try:
|
||||||
__version__ = version("informix-db")
|
# NOTE: the *distribution* is ``informix-driver``; the *module* is
|
||||||
|
# ``informix_db``. They differ, so this string can't be derived from
|
||||||
|
# ``__name__``. This looked up the old ``informix-db`` distribution
|
||||||
|
# until 2026.08.27, which meant __version__ silently reported
|
||||||
|
# "0.0.0+local" for anyone who installed the renamed package.
|
||||||
|
__version__ = version("informix-driver")
|
||||||
except PackageNotFoundError:
|
except PackageNotFoundError:
|
||||||
# Editable install or running uninstalled; fall back to a sentinel.
|
# Running from a source checkout without an install.
|
||||||
__version__ = "0.0.0+local"
|
__version__ = "0.0.0+local"
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
@ -80,6 +86,7 @@ __all__ = [
|
|||||||
"PoolTimeoutError",
|
"PoolTimeoutError",
|
||||||
"ProgrammingError",
|
"ProgrammingError",
|
||||||
"RowValue",
|
"RowValue",
|
||||||
|
"ServerCapabilities",
|
||||||
"Warning",
|
"Warning",
|
||||||
"__version__",
|
"__version__",
|
||||||
"apilevel",
|
"apilevel",
|
||||||
|
|||||||
269
src/informix_db/_capabilities.py
Normal file
269
src/informix_db/_capabilities.py
Normal file
@ -0,0 +1,269 @@
|
|||||||
|
"""Server capability decoding — the SQ_PROTOCOLS feature bitmap.
|
||||||
|
|
||||||
|
SQLI does not have a single "protocol version". It has a 64-bit feature
|
||||||
|
bitmap that client and server negotiate at connect time, and several bits
|
||||||
|
in it change **wire framing**, not just feature availability. Get one of
|
||||||
|
those wrong and the row decoder desyncs.
|
||||||
|
|
||||||
|
We already perform the negotiation (``Connection._init_session`` sends
|
||||||
|
``SQ_PROTOCOLS`` with the same 8-byte client offer the IBM JDBC driver
|
||||||
|
uses). Until now we discarded the server's reply and hardcoded the modern
|
||||||
|
framing. This module decodes the reply so the assumptions can at least be
|
||||||
|
*checked* instead of merely believed.
|
||||||
|
|
||||||
|
Two independent sources feed the bitmap, mirroring
|
||||||
|
``IfxSqliConnect.getServerVer`` / ``enhancedProtocolMechanism``:
|
||||||
|
|
||||||
|
1. ``Cap_1`` from the login (SLTYPE_CONACC) response. This is the client's
|
||||||
|
own protocol level echoed back — 316 means "speaks the enhanced
|
||||||
|
protocol". It is **not** a server version number, which is why JDBC
|
||||||
|
tests ``== 316`` rather than ``>=``.
|
||||||
|
2. The 8-byte mask from the SQ_PROTOCOLS reply.
|
||||||
|
|
||||||
|
When ``Cap_1`` is non-zero, JDBC unconditionally pre-sets bits
|
||||||
|
{0, 2, 3, 4, 49, 51} *before* applying the mask, and a Java ``BitSet.set``
|
||||||
|
only ever sets — never clears. So those six bits are true on any modern
|
||||||
|
server regardless of what the mask says. ``isUSVER`` (bit 2) is one of
|
||||||
|
them, which is why it never varies in practice and is a dead end when
|
||||||
|
chasing version-specific behaviour.
|
||||||
|
|
||||||
|
Bit numbering is MSB-first within each byte: byte 0 bit 0x80 is bit 0,
|
||||||
|
byte 0 bit 0x01 is bit 7, byte 1 bit 0x80 is bit 8, and so on.
|
||||||
|
|
||||||
|
A note on length. The client offer is 8 bytes, but every server we have
|
||||||
|
tested replies with **nine**. JDBC's ``enhancedProtocolMechanism``
|
||||||
|
dispatches on ``switch (i) case 0..7`` and silently discards anything
|
||||||
|
past byte 7, so its ``BitSet(64)`` never sees the ninth byte. We decode
|
||||||
|
it — bits 64-71 — because it turns out to be the only part of the mask
|
||||||
|
that differs between releases::
|
||||||
|
|
||||||
|
Informix 15.0.1.0.3 bdbe9ffe7fb7ffef ff
|
||||||
|
Informix 14.10.FC7W1 bdbe9ffe7fb7ffef f8
|
||||||
|
Informix 12.10.FC12W1DE bdbe9ffe7fb7ffef f0
|
||||||
|
^^^^^^^^^^^^^^^^ identical
|
||||||
|
|
||||||
|
The first 64 bits being byte-identical across those three releases is
|
||||||
|
why they speak an indistinguishable SQLI dialect. Bits 64+ have no known
|
||||||
|
meaning here; they are surfaced for diagnostics, not branched on.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
|
||||||
|
# The client's capability offer. Byte-for-byte the value IBM's JDBC driver
|
||||||
|
# sends (``IfxSqliConnect.clientProtocols``), replayed verbatim — the bits
|
||||||
|
# are a fixed constant there too, never computed.
|
||||||
|
CLIENT_PROTOCOLS_MASK = bytes.fromhex("fffc7ffc3c8caa97")
|
||||||
|
|
||||||
|
# ``Cap_1`` value meaning "enhanced protocol negotiation applies". JDBC
|
||||||
|
# compares with equality, not >=, because the server echoes the client's
|
||||||
|
# own declared level rather than reporting its own.
|
||||||
|
ENHANCED_PROTOCOL_CAP = 316
|
||||||
|
|
||||||
|
# Bits JDBC pre-sets for any non-zero Cap_1, before the mask is applied.
|
||||||
|
_PRESET_BITS = frozenset({0, 2, 3, 4, 49, 51})
|
||||||
|
|
||||||
|
# Named bits. Only the ones we can justify from the decompiled JDBC are
|
||||||
|
# listed; the mask has 64 and most are irrelevant to us.
|
||||||
|
BIT_USVER = 2 # isUSVER — always true on modern servers
|
||||||
|
BIT_LONG_ID_A = 10 # isLongID is bit 10 OR bit 18
|
||||||
|
BIT_LONG_ID_B = 18
|
||||||
|
BIT_VARCHAR_VAR_LEN = 13 # gated further by bit 40 + client ifxPADVARCHAR
|
||||||
|
BIT_DESCRIBE_INPUT = 34
|
||||||
|
BIT_LVARCHAR_GT_2K = 37
|
||||||
|
BIT_PAD_VARCHAR_GATE = 40
|
||||||
|
BIT_FP_DESCRIBE = 45
|
||||||
|
BIT_AUTO_GENERATED_KEYS = 48
|
||||||
|
BIT_CUR_SESS_INFO = 49
|
||||||
|
BIT_FOUR_BYTE_OFFSET = 50 # describe: string-table + field-index widths
|
||||||
|
BIT_GLS = 51
|
||||||
|
BIT_NAMED_PARAMETERS = 52
|
||||||
|
BIT_BIGINT = 54
|
||||||
|
BIT_SAVEPOINT = 56
|
||||||
|
BIT_SQ_BATCH = 61
|
||||||
|
BIT_REMOVE_64K_LIMIT = 62 # 4-byte vs 2-byte length prefixes
|
||||||
|
BIT_TWO_GB_FETCH_BUFFER = 63
|
||||||
|
|
||||||
|
|
||||||
|
def _decode_bits(mask: bytes) -> set[int]:
|
||||||
|
"""Expand a big-endian, MSB-first bit mask into a set of bit numbers."""
|
||||||
|
bits: set[int] = set()
|
||||||
|
for byte_index, value in enumerate(mask):
|
||||||
|
for offset in range(8):
|
||||||
|
if value & (0x80 >> offset):
|
||||||
|
bits.add(byte_index * 8 + offset)
|
||||||
|
return bits
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ServerCapabilities:
|
||||||
|
"""Decoded SQ_PROTOCOLS feature bitmap plus login-response metadata.
|
||||||
|
|
||||||
|
``bits`` is the authoritative set; the named properties are readability
|
||||||
|
sugar over it. ``raw_mask`` is kept so a bug report can include the
|
||||||
|
exact bytes without needing a packet capture.
|
||||||
|
"""
|
||||||
|
|
||||||
|
bits: frozenset[int] = field(default_factory=frozenset)
|
||||||
|
raw_mask: bytes = b""
|
||||||
|
cap_1: int = 0
|
||||||
|
cap_2: int = 0
|
||||||
|
cap_3: int = 0
|
||||||
|
server_version: str = ""
|
||||||
|
serial_number: str = ""
|
||||||
|
applid_name: str = ""
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_wire(
|
||||||
|
cls,
|
||||||
|
mask: bytes,
|
||||||
|
*,
|
||||||
|
cap_1: int = 0,
|
||||||
|
cap_2: int = 0,
|
||||||
|
cap_3: int = 0,
|
||||||
|
server_version: str = "",
|
||||||
|
serial_number: str = "",
|
||||||
|
applid_name: str = "",
|
||||||
|
) -> ServerCapabilities:
|
||||||
|
bits = _decode_bits(mask)
|
||||||
|
if cap_1 != 0:
|
||||||
|
# Mirrors JDBC: pre-set bits are OR-ed in and never cleared.
|
||||||
|
bits |= _PRESET_BITS
|
||||||
|
return cls(
|
||||||
|
bits=frozenset(bits),
|
||||||
|
raw_mask=bytes(mask),
|
||||||
|
cap_1=cap_1,
|
||||||
|
cap_2=cap_2,
|
||||||
|
cap_3=cap_3,
|
||||||
|
server_version=server_version,
|
||||||
|
serial_number=serial_number,
|
||||||
|
applid_name=applid_name,
|
||||||
|
)
|
||||||
|
|
||||||
|
def has(self, bit: int) -> bool:
|
||||||
|
return bit in self.bits
|
||||||
|
|
||||||
|
# -- named capabilities ------------------------------------------------
|
||||||
|
|
||||||
|
@property
|
||||||
|
def usver(self) -> bool:
|
||||||
|
return self.has(BIT_USVER)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def long_id(self) -> bool:
|
||||||
|
return self.has(BIT_LONG_ID_A) or self.has(BIT_LONG_ID_B)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def varchar_var_len(self) -> bool:
|
||||||
|
"""VARCHAR carries a 1-byte length prefix rather than being padded
|
||||||
|
to its full declared width.
|
||||||
|
|
||||||
|
JDBC additionally consults the client-side ``ifxPADVARCHAR``
|
||||||
|
setting; we never enable padding, so the client half is constant.
|
||||||
|
"""
|
||||||
|
return self.has(BIT_VARCHAR_VAR_LEN)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def four_byte_offset(self) -> bool:
|
||||||
|
"""Describe records use 4-byte string-table size and field indices
|
||||||
|
rather than 2-byte."""
|
||||||
|
return self.has(BIT_FOUR_BYTE_OFFSET)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def bigint(self) -> bool:
|
||||||
|
return self.has(BIT_BIGINT)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def lvarchar_gt_2k(self) -> bool:
|
||||||
|
return self.has(BIT_LVARCHAR_GT_2K)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def remove_64k_limit(self) -> bool:
|
||||||
|
"""4-byte length prefixes in the fast-path/SQ_FILE paths."""
|
||||||
|
return self.has(BIT_REMOVE_64K_LIMIT)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def two_gb_fetch_buffer(self) -> bool:
|
||||||
|
return self.has(BIT_TWO_GB_FETCH_BUFFER)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def gls(self) -> bool:
|
||||||
|
return self.has(BIT_GLS)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def describe_input(self) -> bool:
|
||||||
|
return self.has(BIT_DESCRIBE_INPUT)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def named_parameters(self) -> bool:
|
||||||
|
return self.has(BIT_NAMED_PARAMETERS)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def savepoint(self) -> bool:
|
||||||
|
return self.has(BIT_SAVEPOINT)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def enhanced_protocol(self) -> bool:
|
||||||
|
return self.cap_1 == ENHANCED_PROTOCOL_CAP
|
||||||
|
|
||||||
|
# -- assumption checking ----------------------------------------------
|
||||||
|
|
||||||
|
def violated_assumptions(self) -> list[str]:
|
||||||
|
"""Framing assumptions this driver hardcodes that the server
|
||||||
|
contradicts.
|
||||||
|
|
||||||
|
Each entry names a place where we emit or parse a fixed wire shape
|
||||||
|
that is actually capability-gated. An empty list means every
|
||||||
|
hardcoded choice matches what the server negotiated.
|
||||||
|
|
||||||
|
This is a diagnostic, not a guarantee: it only covers bits we know
|
||||||
|
to be framing-relevant from the decompiled JDBC. It exists so that
|
||||||
|
a server we have never tested produces a loud, specific warning
|
||||||
|
instead of silently corrupted rows.
|
||||||
|
"""
|
||||||
|
problems: list[str] = []
|
||||||
|
if not self.four_byte_offset:
|
||||||
|
problems.append(
|
||||||
|
"server did not negotiate 4-byte describe offsets "
|
||||||
|
f"(bit {BIT_FOUR_BYTE_OFFSET}), but parse_describe reads "
|
||||||
|
"4-byte string-table size and field indices; column "
|
||||||
|
"metadata will be misparsed"
|
||||||
|
)
|
||||||
|
if not self.varchar_var_len:
|
||||||
|
problems.append(
|
||||||
|
f"server did not negotiate variable-length VARCHAR "
|
||||||
|
f"(bit {BIT_VARCHAR_VAR_LEN}); VARCHAR values are padded to "
|
||||||
|
"the declared column width on the wire, but the row decoder "
|
||||||
|
"expects a 1-byte length prefix"
|
||||||
|
)
|
||||||
|
if not self.remove_64k_limit:
|
||||||
|
problems.append(
|
||||||
|
f"server did not negotiate the 64K limit removal "
|
||||||
|
f"(bit {BIT_REMOVE_64K_LIMIT}); fast-path routine signatures "
|
||||||
|
"are emitted with a 4-byte length prefix that the server "
|
||||||
|
"expects to be 2-byte"
|
||||||
|
)
|
||||||
|
return problems
|
||||||
|
|
||||||
|
def __repr__(self) -> str:
|
||||||
|
named = [
|
||||||
|
name
|
||||||
|
for name, value in (
|
||||||
|
("usver", self.usver),
|
||||||
|
("4byte_offset", self.four_byte_offset),
|
||||||
|
("varchar_var_len", self.varchar_var_len),
|
||||||
|
("bigint", self.bigint),
|
||||||
|
("long_id", self.long_id),
|
||||||
|
("lvarchar>2k", self.lvarchar_gt_2k),
|
||||||
|
("remove_64k", self.remove_64k_limit),
|
||||||
|
("gls", self.gls),
|
||||||
|
)
|
||||||
|
if value
|
||||||
|
]
|
||||||
|
return (
|
||||||
|
f"ServerCapabilities(cap_1={self.cap_1}, "
|
||||||
|
f"mask={self.raw_mask.hex()}, "
|
||||||
|
f"version={self.server_version!r}, "
|
||||||
|
f"flags={'|'.join(named) or 'none'})"
|
||||||
|
)
|
||||||
@ -12,6 +12,7 @@ reference in ``docs/CAPTURES/01-connect-only.socat.log``.
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import contextlib
|
import contextlib
|
||||||
|
import logging
|
||||||
import os
|
import os
|
||||||
import socket as socket_mod
|
import socket as socket_mod
|
||||||
import ssl
|
import ssl
|
||||||
@ -21,6 +22,7 @@ from io import BytesIO
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from . import _auth
|
from . import _auth
|
||||||
|
from ._capabilities import CLIENT_PROTOCOLS_MASK, ServerCapabilities
|
||||||
from ._messages import (
|
from ._messages import (
|
||||||
APPL_ID,
|
APPL_ID,
|
||||||
APPL_TYPE,
|
APPL_TYPE,
|
||||||
@ -69,6 +71,9 @@ _LOCALE_ENCODING_MAP = {
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
_log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
def _extract_server_error_text(payload: bytes) -> str | None:
|
def _extract_server_error_text(payload: bytes) -> str | None:
|
||||||
"""Pull the longest printable run out of an opaque rejection payload.
|
"""Pull the longest printable run out of an opaque rejection payload.
|
||||||
|
|
||||||
@ -113,6 +118,74 @@ def _python_encoding_from_locale(locale: str) -> str:
|
|||||||
return _LOCALE_ENCODING_MAP.get(suffix, "iso-8859-1")
|
return _LOCALE_ENCODING_MAP.get(suffix, "iso-8859-1")
|
||||||
|
|
||||||
|
|
||||||
|
def _decode_conacc(rest: bytes) -> dict | None:
|
||||||
|
"""Decode the SLTYPE_CONACC server-metadata block.
|
||||||
|
|
||||||
|
Mirrors ``com.informix.asf.Connection.DecodeAscBinary``. ``rest`` is
|
||||||
|
the login response with its 2-byte total-length prefix already
|
||||||
|
stripped, i.e. starting at the SLType byte.
|
||||||
|
|
||||||
|
Layout, verified byte-for-byte against Informix 12.10, 14.10 and 15::
|
||||||
|
|
||||||
|
byte SLType (2 = CONACC)
|
||||||
|
3 skip
|
||||||
|
short 100 marker
|
||||||
|
short 101 marker
|
||||||
|
4 skip
|
||||||
|
short len; skip len (codeset id, e.g. "IEEEI")
|
||||||
|
short 108 marker — JDBC hard-fails if this isn't 108
|
||||||
|
12 skip
|
||||||
|
short len; bytes server version string
|
||||||
|
short len; bytes serial number
|
||||||
|
short len; bytes applid name
|
||||||
|
int Cap_1 client protocol level, echoed (316 = enhanced)
|
||||||
|
int Cap_2
|
||||||
|
int Cap_3
|
||||||
|
|
||||||
|
Returns ``None`` if the block doesn't match that shape. Callers must
|
||||||
|
treat that as "no capability info", never as a connection failure —
|
||||||
|
every server we support connects fine without any of this being
|
||||||
|
decodable, and a future server variant should degrade, not break.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
offset = 1 + 3 # SLType + 3 skipped bytes
|
||||||
|
marker_a, marker_b = struct.unpack_from("!hh", rest, offset)
|
||||||
|
offset += 4
|
||||||
|
if marker_a != 100 or marker_b != 101:
|
||||||
|
return None
|
||||||
|
offset += 4
|
||||||
|
(codeset_len,) = struct.unpack_from("!h", rest, offset)
|
||||||
|
offset += 2 + codeset_len
|
||||||
|
(marker_c,) = struct.unpack_from("!h", rest, offset)
|
||||||
|
offset += 2
|
||||||
|
if marker_c != 108:
|
||||||
|
return None
|
||||||
|
offset += 12
|
||||||
|
strings: list[str] = []
|
||||||
|
for _ in range(3):
|
||||||
|
(length,) = struct.unpack_from("!h", rest, offset)
|
||||||
|
offset += 2
|
||||||
|
if length < 0:
|
||||||
|
return None
|
||||||
|
strings.append(
|
||||||
|
rest[offset : offset + length].rstrip(b"\x00").decode(
|
||||||
|
"iso-8859-1", "replace"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
offset += length
|
||||||
|
cap_1, cap_2, cap_3 = struct.unpack_from("!iii", rest, offset)
|
||||||
|
except (struct.error, IndexError, UnicodeDecodeError):
|
||||||
|
return None
|
||||||
|
return {
|
||||||
|
"server_version": strings[0],
|
||||||
|
"serial_number": strings[1],
|
||||||
|
"applid_name": strings[2],
|
||||||
|
"cap_1": cap_1,
|
||||||
|
"cap_2": cap_2,
|
||||||
|
"cap_3": cap_3,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
# Default environment variables sent in the login PDU (SQ_ASCENV section).
|
# Default environment variables sent in the login PDU (SQ_ASCENV section).
|
||||||
# These match what the JDBC driver sends for a vanilla en_US.8859-1
|
# These match what the JDBC driver sends for a vanilla en_US.8859-1
|
||||||
# connection. Anything missing makes the server fall back to defaults.
|
# connection. Anything missing makes the server fall back to defaults.
|
||||||
@ -205,6 +278,13 @@ class Connection:
|
|||||||
# SQ_GETROUTINE; subsequent calls skip that round-trip.
|
# SQ_GETROUTINE; subsequent calls skip that round-trip.
|
||||||
self._fp_handle_cache: dict[str, tuple[str, int]] = {}
|
self._fp_handle_cache: dict[str, tuple[str, int]] = {}
|
||||||
|
|
||||||
|
# Server metadata from the login response and the SQ_PROTOCOLS
|
||||||
|
# negotiation. Both are populated during connect; both degrade to
|
||||||
|
# None rather than failing the connection.
|
||||||
|
self._conacc: dict | None = None
|
||||||
|
self._server_protocols: bytes | None = None
|
||||||
|
self._capabilities: ServerCapabilities | None = None
|
||||||
|
|
||||||
# Build the env-var dict sent in the login PDU.
|
# Build the env-var dict sent in the login PDU.
|
||||||
self._env = dict(_DEFAULT_ENV)
|
self._env = dict(_DEFAULT_ENV)
|
||||||
self._env["CLIENT_LOCALE"] = client_locale
|
self._env["CLIENT_LOCALE"] = client_locale
|
||||||
@ -589,9 +669,9 @@ class Connection:
|
|||||||
# The 8-byte protocols mask is the JDBC reference value from
|
# The 8-byte protocols mask is the JDBC reference value from
|
||||||
# docs/CAPTURES/02-select-1.socat.log; we replay it verbatim
|
# docs/CAPTURES/02-select-1.socat.log; we replay it verbatim
|
||||||
# since the bits are opaque (server-recognized features).
|
# since the bits are opaque (server-recognized features).
|
||||||
protocols_mask = bytes.fromhex("fffc7ffc3c8caa97")
|
self._send_protocols(CLIENT_PROTOCOLS_MASK)
|
||||||
self._send_protocols(protocols_mask)
|
self._drain_to_eot() # captures the reply into self._server_protocols
|
||||||
self._drain_to_eot()
|
self._build_capabilities()
|
||||||
|
|
||||||
# Step 2: SQ_INFO with INFO_ENV subtype + session env vars.
|
# Step 2: SQ_INFO with INFO_ENV subtype + session env vars.
|
||||||
# The actual on-wire format (from JDBC's sendEnv at IfxSqli.java
|
# The actual on-wire format (from JDBC's sendEnv at IfxSqli.java
|
||||||
@ -626,6 +706,54 @@ class Connection:
|
|||||||
self._send_dbopen(self._database)
|
self._send_dbopen(self._database)
|
||||||
self._drain_to_eot()
|
self._drain_to_eot()
|
||||||
|
|
||||||
|
def _build_capabilities(self) -> None:
|
||||||
|
"""Assemble ``ServerCapabilities`` from the login response and the
|
||||||
|
SQ_PROTOCOLS reply, and warn about contradicted assumptions.
|
||||||
|
|
||||||
|
This is observation only — nothing in the parse paths branches on
|
||||||
|
these bits yet. The value is diagnostic: this driver hardcodes
|
||||||
|
several wire-framing choices that SQLI actually negotiates, and
|
||||||
|
those choices are correct on every server we have measured
|
||||||
|
(Informix 12.10, 14.10, 15). On some server we have not measured
|
||||||
|
they might not be, and the failure mode for a framing mismatch is
|
||||||
|
silently corrupted rows, which is about the worst way for a
|
||||||
|
database driver to fail. A log warning naming the specific bit
|
||||||
|
turns that into something diagnosable from a bug report.
|
||||||
|
|
||||||
|
Deliberately never raises: a server that declines to negotiate is
|
||||||
|
not necessarily broken, and refusing the connection over a
|
||||||
|
diagnostic would be a worse outcome than proceeding.
|
||||||
|
"""
|
||||||
|
if self._server_protocols is None:
|
||||||
|
return
|
||||||
|
conacc = self._conacc or {}
|
||||||
|
try:
|
||||||
|
caps = ServerCapabilities.from_wire(
|
||||||
|
self._server_protocols,
|
||||||
|
cap_1=conacc.get("cap_1", 0),
|
||||||
|
cap_2=conacc.get("cap_2", 0),
|
||||||
|
cap_3=conacc.get("cap_3", 0),
|
||||||
|
server_version=conacc.get("server_version", ""),
|
||||||
|
serial_number=conacc.get("serial_number", ""),
|
||||||
|
applid_name=conacc.get("applid_name", ""),
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
_log.debug("could not decode server capabilities", exc_info=True)
|
||||||
|
return
|
||||||
|
self._capabilities = caps
|
||||||
|
|
||||||
|
problems = caps.violated_assumptions()
|
||||||
|
if problems:
|
||||||
|
_log.warning(
|
||||||
|
"Informix server %r negotiated capabilities that contradict "
|
||||||
|
"this driver's hardcoded wire framing (mask=%s). Result rows "
|
||||||
|
"may be decoded incorrectly. Please report this with the "
|
||||||
|
"mask and your server version. Details: %s",
|
||||||
|
caps.server_version or "<unknown version>",
|
||||||
|
caps.raw_mask.hex(),
|
||||||
|
"; ".join(problems),
|
||||||
|
)
|
||||||
|
|
||||||
def _send_protocols(self, protocols: bytes) -> None:
|
def _send_protocols(self, protocols: bytes) -> None:
|
||||||
"""Emit a SQ_PROTOCOLS PDU per ``IfxSqli.sendProtocols``.
|
"""Emit a SQ_PROTOCOLS PDU per ``IfxSqli.sendProtocols``.
|
||||||
|
|
||||||
@ -668,9 +796,13 @@ class Connection:
|
|||||||
elif tag == MessageType.SQ_PROTOCOLS:
|
elif tag == MessageType.SQ_PROTOCOLS:
|
||||||
# ``[short payloadLen][bytes payload][byte 0 if odd-len pad]``
|
# ``[short payloadLen][bytes payload][byte 0 if odd-len pad]``
|
||||||
# Then the loop continues and consumes the next tag (usually SQ_EOT).
|
# Then the loop continues and consumes the next tag (usually SQ_EOT).
|
||||||
|
# The payload is the negotiated 64-bit feature bitmap; stash
|
||||||
|
# it for _capabilities decoding. Reading it was always
|
||||||
|
# necessary for stream alignment — we just used to throw it
|
||||||
|
# away instead of looking at it.
|
||||||
payload_len = struct.unpack("!h", self._sock.read_exact(2))[0]
|
payload_len = struct.unpack("!h", self._sock.read_exact(2))[0]
|
||||||
if payload_len > 0:
|
if payload_len > 0:
|
||||||
self._sock.read_exact(payload_len)
|
self._server_protocols = self._sock.read_exact(payload_len)
|
||||||
if payload_len & 1:
|
if payload_len & 1:
|
||||||
self._sock.read_exact(1) # writePadded's even-alignment pad
|
self._sock.read_exact(1) # writePadded's even-alignment pad
|
||||||
elif tag == MessageType.SQ_DONE:
|
elif tag == MessageType.SQ_DONE:
|
||||||
@ -883,6 +1015,28 @@ class Connection:
|
|||||||
|
|
||||||
# -- response parsing -------------------------------------------------
|
# -- response parsing -------------------------------------------------
|
||||||
|
|
||||||
|
@property
|
||||||
|
def server_capabilities(self):
|
||||||
|
"""The negotiated ``SQ_PROTOCOLS`` feature bitmap, or ``None``.
|
||||||
|
|
||||||
|
``None`` means the negotiation reply couldn't be decoded — the
|
||||||
|
connection still works, we just have no capability information.
|
||||||
|
See ``informix_db._capabilities.ServerCapabilities``.
|
||||||
|
"""
|
||||||
|
return self._capabilities
|
||||||
|
|
||||||
|
@property
|
||||||
|
def server_version(self) -> str:
|
||||||
|
"""Server version string from the login response.
|
||||||
|
|
||||||
|
Note this is the *internal* protocol version, which is not the
|
||||||
|
marketing version. Informix 12.10 reports ``9.56``, 14.10 reports
|
||||||
|
``9.59``, and 15 reports ``15.0.1.0.3``. That is why the two older
|
||||||
|
releases speak an identical SQLI dialect: at the protocol level
|
||||||
|
they are both 9.x servers.
|
||||||
|
"""
|
||||||
|
return self._conacc.get("server_version", "") if self._conacc else ""
|
||||||
|
|
||||||
def _parse_login_response(self) -> None:
|
def _parse_login_response(self) -> None:
|
||||||
"""Read and parse the server's login response.
|
"""Read and parse the server's login response.
|
||||||
|
|
||||||
@ -916,8 +1070,12 @@ class Connection:
|
|||||||
)
|
)
|
||||||
elif sl_type != SLHeader.SLTYPE_CONACC:
|
elif sl_type != SLHeader.SLTYPE_CONACC:
|
||||||
raise ProtocolError(f"unknown SLType in login response: {sl_type}")
|
raise ProtocolError(f"unknown SLType in login response: {sl_type}")
|
||||||
# SLTYPE_CONACC — connection accepted. We don't (yet) decode the
|
|
||||||
# full server-side metadata. Phase 1 just needs to know "we got in".
|
# SLTYPE_CONACC — connection accepted. Decode the server metadata
|
||||||
|
# block for the capability fields. Best-effort: a decode failure
|
||||||
|
# here must not fail an otherwise-good connection, since every
|
||||||
|
# server we've tested connects fine without any of this.
|
||||||
|
self._conacc = _decode_conacc(rest)
|
||||||
|
|
||||||
def _raise_from_rejection(self, reader: IfxStreamReader) -> None:
|
def _raise_from_rejection(self, reader: IfxStreamReader) -> None:
|
||||||
"""Best-effort decode of the connection-rejection error block.
|
"""Best-effort decode of the connection-rejection error block.
|
||||||
|
|||||||
98
tests/test_capabilities.py
Normal file
98
tests/test_capabilities.py
Normal file
@ -0,0 +1,98 @@
|
|||||||
|
"""Integration tests for live SQ_PROTOCOLS negotiation.
|
||||||
|
|
||||||
|
These run against whatever server ``IFX_PORT`` points at, so `make
|
||||||
|
test-matrix` exercises them on 12.10, 14.10, and 15 in turn.
|
||||||
|
|
||||||
|
The important one is ``test_no_violated_assumptions``: this driver
|
||||||
|
hardcodes several wire-framing choices that SQLI actually negotiates, and
|
||||||
|
a mismatch corrupts rows silently. That test turns "we assume this" into
|
||||||
|
"we check this on every supported server".
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
import informix_db
|
||||||
|
from informix_db._capabilities import ENHANCED_PROTOCOL_CAP, ServerCapabilities
|
||||||
|
from tests.conftest import ConnParams
|
||||||
|
|
||||||
|
pytestmark = pytest.mark.integration
|
||||||
|
|
||||||
|
|
||||||
|
def _connect(conn_params: ConnParams) -> informix_db.Connection:
|
||||||
|
return informix_db.connect(
|
||||||
|
host=conn_params.host,
|
||||||
|
port=conn_params.port,
|
||||||
|
user=conn_params.user,
|
||||||
|
password=conn_params.password,
|
||||||
|
database=conn_params.database,
|
||||||
|
server=conn_params.server,
|
||||||
|
connect_timeout=10.0,
|
||||||
|
read_timeout=10.0,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_capabilities_are_decoded(conn_params: ConnParams) -> None:
|
||||||
|
with _connect(conn_params) as conn:
|
||||||
|
caps = conn.server_capabilities
|
||||||
|
assert isinstance(caps, ServerCapabilities)
|
||||||
|
assert caps.raw_mask, "server sent an empty protocols reply"
|
||||||
|
assert caps.bits
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_violated_assumptions(conn_params: ConnParams) -> None:
|
||||||
|
"""Every wire-framing shape we hardcode is one the server negotiated.
|
||||||
|
|
||||||
|
If this fails, rows are being decoded against the wrong framing and
|
||||||
|
the fix is to branch on the capability rather than assume it.
|
||||||
|
"""
|
||||||
|
with _connect(conn_params) as conn:
|
||||||
|
caps = conn.server_capabilities
|
||||||
|
assert caps is not None
|
||||||
|
assert caps.violated_assumptions() == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_enhanced_protocol_negotiated(conn_params: ConnParams) -> None:
|
||||||
|
"""Cap_1 is the client's declared protocol level echoed back. Every
|
||||||
|
supported server accepts 316."""
|
||||||
|
with _connect(conn_params) as conn:
|
||||||
|
caps = conn.server_capabilities
|
||||||
|
assert caps is not None
|
||||||
|
assert caps.cap_1 == ENHANCED_PROTOCOL_CAP
|
||||||
|
assert caps.enhanced_protocol
|
||||||
|
|
||||||
|
|
||||||
|
def test_framing_capabilities_present(conn_params: ConnParams) -> None:
|
||||||
|
with _connect(conn_params) as conn:
|
||||||
|
caps = conn.server_capabilities
|
||||||
|
assert caps is not None
|
||||||
|
assert caps.four_byte_offset
|
||||||
|
assert caps.varchar_var_len
|
||||||
|
assert caps.remove_64k_limit
|
||||||
|
assert caps.usver
|
||||||
|
|
||||||
|
|
||||||
|
def test_server_version_is_exposed(conn_params: ConnParams) -> None:
|
||||||
|
"""The version here is the *internal* protocol version, not the
|
||||||
|
marketing one: 12.10 reports 9.56 and 14.10 reports 9.59. Assert the
|
||||||
|
shape rather than a value so this holds across the matrix."""
|
||||||
|
with _connect(conn_params) as conn:
|
||||||
|
version = conn.server_version
|
||||||
|
assert "Informix" in version
|
||||||
|
assert "Version" in version
|
||||||
|
|
||||||
|
|
||||||
|
def test_capabilities_survive_multiple_connections(
|
||||||
|
conn_params: ConnParams,
|
||||||
|
) -> None:
|
||||||
|
"""Negotiation happens per-connection; two connections to the same
|
||||||
|
server must agree."""
|
||||||
|
with _connect(conn_params) as first, _connect(conn_params) as second:
|
||||||
|
assert first.server_capabilities is not None
|
||||||
|
assert second.server_capabilities is not None
|
||||||
|
assert (
|
||||||
|
first.server_capabilities.raw_mask
|
||||||
|
== second.server_capabilities.raw_mask
|
||||||
|
)
|
||||||
|
assert first.server_capabilities.bits == second.server_capabilities.bits
|
||||||
200
tests/test_capabilities_unit.py
Normal file
200
tests/test_capabilities_unit.py
Normal file
@ -0,0 +1,200 @@
|
|||||||
|
"""Unit tests for SQ_PROTOCOLS capability decoding — no server required.
|
||||||
|
|
||||||
|
The masks below were captured from live servers on 2026-08-27. Their most
|
||||||
|
interesting property is that the first eight bytes are identical across
|
||||||
|
Informix 12.10, 14.10, and 15 — which is why those three releases speak an
|
||||||
|
indistinguishable SQLI dialect.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import dataclasses
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from informix_db._capabilities import (
|
||||||
|
BIT_FOUR_BYTE_OFFSET,
|
||||||
|
BIT_REMOVE_64K_LIMIT,
|
||||||
|
BIT_USVER,
|
||||||
|
BIT_VARCHAR_VAR_LEN,
|
||||||
|
CLIENT_PROTOCOLS_MASK,
|
||||||
|
ENHANCED_PROTOCOL_CAP,
|
||||||
|
ServerCapabilities,
|
||||||
|
_decode_bits,
|
||||||
|
)
|
||||||
|
|
||||||
|
MASK_15 = bytes.fromhex("bdbe9ffe7fb7ffefff")
|
||||||
|
MASK_1410 = bytes.fromhex("bdbe9ffe7fb7ffeff8")
|
||||||
|
MASK_1210 = bytes.fromhex("bdbe9ffe7fb7ffeff0")
|
||||||
|
|
||||||
|
ALL_MASKS = [
|
||||||
|
pytest.param(MASK_15, id="15.0.1.0.3"),
|
||||||
|
pytest.param(MASK_1410, id="14.10.FC7W1"),
|
||||||
|
pytest.param(MASK_1210, id="12.10.FC12W1DE"),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# Bit expansion
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_bit_numbering_is_msb_first() -> None:
|
||||||
|
assert _decode_bits(b"\x80") == {0}
|
||||||
|
assert _decode_bits(b"\x01") == {7}
|
||||||
|
assert _decode_bits(b"\x00\x80") == {8}
|
||||||
|
assert _decode_bits(b"\x00\x01") == {15}
|
||||||
|
|
||||||
|
|
||||||
|
def test_bit_expansion_of_first_captured_byte() -> None:
|
||||||
|
"""0xBD is 1011 1101 -> bits 0, 2, 3, 4, 5, 7."""
|
||||||
|
assert _decode_bits(b"\xbd") == {0, 2, 3, 4, 5, 7}
|
||||||
|
|
||||||
|
|
||||||
|
def test_empty_mask_yields_no_bits() -> None:
|
||||||
|
assert _decode_bits(b"") == set()
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# The captured masks
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_first_eight_bytes_identical_across_versions() -> None:
|
||||||
|
"""The finding this whole module exists to record: 12.10, 14.10, and 15
|
||||||
|
negotiate exactly the same 64-bit capability set. Only the 9th byte —
|
||||||
|
which IBM's own JDBC driver discards — differs."""
|
||||||
|
assert MASK_15[:8] == MASK_1410[:8] == MASK_1210[:8]
|
||||||
|
assert len({MASK_15[8], MASK_1410[8], MASK_1210[8]}) == 3
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("mask", ALL_MASKS)
|
||||||
|
def test_framing_bits_set_on_every_tested_server(mask: bytes) -> None:
|
||||||
|
"""Every wire-framing choice this driver hardcodes is one the server
|
||||||
|
actually negotiated. If this ever fails, the row decoder is wrong."""
|
||||||
|
caps = ServerCapabilities.from_wire(mask, cap_1=ENHANCED_PROTOCOL_CAP)
|
||||||
|
assert caps.four_byte_offset, "describe offsets would be misparsed"
|
||||||
|
assert caps.varchar_var_len, "VARCHAR framing would be misparsed"
|
||||||
|
assert caps.remove_64k_limit, "fast-path length prefix would be wrong"
|
||||||
|
assert caps.bigint
|
||||||
|
assert caps.long_id
|
||||||
|
assert caps.lvarchar_gt_2k
|
||||||
|
assert caps.gls
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("mask", ALL_MASKS)
|
||||||
|
def test_no_violated_assumptions_on_tested_servers(mask: bytes) -> None:
|
||||||
|
caps = ServerCapabilities.from_wire(mask, cap_1=ENHANCED_PROTOCOL_CAP)
|
||||||
|
assert caps.violated_assumptions() == []
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("mask", ALL_MASKS)
|
||||||
|
def test_ninth_byte_is_decoded(mask: bytes) -> None:
|
||||||
|
"""JDBC drops bits 64+; we keep them because they're the only part
|
||||||
|
that varies between releases."""
|
||||||
|
caps = ServerCapabilities.from_wire(mask, cap_1=ENHANCED_PROTOCOL_CAP)
|
||||||
|
assert any(b >= 64 for b in caps.bits)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# Pre-set bits — the reason isUSVER is a dead end
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_preset_bits_applied_when_cap1_nonzero() -> None:
|
||||||
|
"""JDBC pre-sets {0,2,3,4,49,51} for any non-zero Cap_1 and BitSet.set
|
||||||
|
never clears, so these are true on any modern server regardless of the
|
||||||
|
mask. That's why isUSVER never varies and can't explain version-specific
|
||||||
|
behaviour."""
|
||||||
|
caps = ServerCapabilities.from_wire(b"\x00" * 9, cap_1=ENHANCED_PROTOCOL_CAP)
|
||||||
|
assert {0, 2, 3, 4, 49, 51}.issubset(caps.bits)
|
||||||
|
assert caps.usver
|
||||||
|
|
||||||
|
|
||||||
|
def test_preset_bits_not_applied_when_cap1_zero() -> None:
|
||||||
|
caps = ServerCapabilities.from_wire(b"\x00" * 9, cap_1=0)
|
||||||
|
assert caps.bits == frozenset()
|
||||||
|
assert not caps.usver
|
||||||
|
|
||||||
|
|
||||||
|
def test_preset_cannot_clear_a_bit_the_mask_set() -> None:
|
||||||
|
caps = ServerCapabilities.from_wire(MASK_15, cap_1=ENHANCED_PROTOCOL_CAP)
|
||||||
|
assert caps.has(BIT_USVER)
|
||||||
|
assert caps.has(BIT_FOUR_BYTE_OFFSET)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# Assumption violations — the diagnostic path
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_missing_four_byte_offset_is_reported() -> None:
|
||||||
|
# Everything set except bit 50.
|
||||||
|
bits = bytearray(b"\xff" * 9)
|
||||||
|
bits[BIT_FOUR_BYTE_OFFSET // 8] &= ~(0x80 >> (BIT_FOUR_BYTE_OFFSET % 8)) & 0xFF
|
||||||
|
caps = ServerCapabilities.from_wire(bytes(bits), cap_1=ENHANCED_PROTOCOL_CAP)
|
||||||
|
problems = caps.violated_assumptions()
|
||||||
|
assert len(problems) == 1
|
||||||
|
assert "4-byte describe offsets" in problems[0]
|
||||||
|
|
||||||
|
|
||||||
|
def test_missing_varchar_var_len_is_reported() -> None:
|
||||||
|
bits = bytearray(b"\xff" * 9)
|
||||||
|
bits[BIT_VARCHAR_VAR_LEN // 8] &= ~(0x80 >> (BIT_VARCHAR_VAR_LEN % 8)) & 0xFF
|
||||||
|
caps = ServerCapabilities.from_wire(bytes(bits), cap_1=ENHANCED_PROTOCOL_CAP)
|
||||||
|
problems = caps.violated_assumptions()
|
||||||
|
assert len(problems) == 1
|
||||||
|
assert "variable-length VARCHAR" in problems[0]
|
||||||
|
|
||||||
|
|
||||||
|
def test_missing_remove_64k_is_reported() -> None:
|
||||||
|
bits = bytearray(b"\xff" * 9)
|
||||||
|
bits[BIT_REMOVE_64K_LIMIT // 8] &= ~(0x80 >> (BIT_REMOVE_64K_LIMIT % 8)) & 0xFF
|
||||||
|
caps = ServerCapabilities.from_wire(bytes(bits), cap_1=ENHANCED_PROTOCOL_CAP)
|
||||||
|
problems = caps.violated_assumptions()
|
||||||
|
assert len(problems) == 1
|
||||||
|
assert "64K limit" in problems[0]
|
||||||
|
|
||||||
|
|
||||||
|
def test_all_zero_mask_reports_every_assumption() -> None:
|
||||||
|
caps = ServerCapabilities.from_wire(b"\x00" * 9, cap_1=ENHANCED_PROTOCOL_CAP)
|
||||||
|
assert len(caps.violated_assumptions()) == 3
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# Misc
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_client_offer_matches_jdbc_reference() -> None:
|
||||||
|
"""IfxSqliConnect.clientProtocols, verbatim. Changing this changes what
|
||||||
|
the server negotiates, so it is pinned."""
|
||||||
|
jdbc_reference = bytes([0xFF, 0xFC, 0x7F, 0xFC, 0x3C, 0x8C, 0xAA, 0x97])
|
||||||
|
assert jdbc_reference == CLIENT_PROTOCOLS_MASK
|
||||||
|
assert len(CLIENT_PROTOCOLS_MASK) == 8
|
||||||
|
|
||||||
|
|
||||||
|
def test_enhanced_protocol_requires_exact_cap1() -> None:
|
||||||
|
"""JDBC tests == 316, not >=, because the server echoes the client's own
|
||||||
|
declared level rather than reporting its own version."""
|
||||||
|
assert ServerCapabilities.from_wire(MASK_15, cap_1=316).enhanced_protocol
|
||||||
|
assert not ServerCapabilities.from_wire(MASK_15, cap_1=315).enhanced_protocol
|
||||||
|
assert not ServerCapabilities.from_wire(MASK_15, cap_1=317).enhanced_protocol
|
||||||
|
|
||||||
|
|
||||||
|
def test_repr_is_readable_and_includes_mask() -> None:
|
||||||
|
caps = ServerCapabilities.from_wire(
|
||||||
|
MASK_15, cap_1=316, server_version="IBM Informix Dynamic Server Version 15"
|
||||||
|
)
|
||||||
|
text = repr(caps)
|
||||||
|
assert "bdbe9ffe7fb7ffefff" in text
|
||||||
|
assert "usver" in text
|
||||||
|
assert "cap_1=316" in text
|
||||||
|
|
||||||
|
|
||||||
|
def test_capabilities_are_frozen() -> None:
|
||||||
|
"""Immutable so a connection's negotiated state can't be edited after
|
||||||
|
the fact and quietly disagree with what's on the wire."""
|
||||||
|
caps = ServerCapabilities.from_wire(MASK_15, cap_1=316)
|
||||||
|
with pytest.raises(dataclasses.FrozenInstanceError):
|
||||||
|
caps.cap_1 = 1 # type: ignore[misc]
|
||||||
76
tests/test_package_metadata.py
Normal file
76
tests/test_package_metadata.py
Normal file
@ -0,0 +1,76 @@
|
|||||||
|
"""Guards on the package's own metadata.
|
||||||
|
|
||||||
|
The distribution is ``informix-driver``; the import module is
|
||||||
|
``informix_db``. That mismatch is a live trap: ``importlib.metadata``
|
||||||
|
needs the *distribution* name, so the string can't be derived from
|
||||||
|
``__name__``, and getting it wrong fails silently — ``__version__``
|
||||||
|
degrades to the ``0.0.0+local`` sentinel instead of raising.
|
||||||
|
|
||||||
|
That is exactly what shipped between the 2026-05-08 rename and
|
||||||
|
2026.08.27: ``__init__`` still asked for ``informix-db``, so anyone who
|
||||||
|
installed the renamed package saw ``0.0.0+local``. It went unnoticed
|
||||||
|
because a stale ``informix-db`` distribution lingered in the dev
|
||||||
|
environment and answered the query.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import tomllib
|
||||||
|
|
||||||
|
import informix_db
|
||||||
|
|
||||||
|
PYPROJECT = Path(__file__).resolve().parent.parent / "pyproject.toml"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(scope="module")
|
||||||
|
def pyproject() -> dict:
|
||||||
|
if not PYPROJECT.is_file():
|
||||||
|
pytest.skip("pyproject.toml not present (installed-package test run)")
|
||||||
|
with PYPROJECT.open("rb") as handle:
|
||||||
|
return tomllib.load(handle)
|
||||||
|
|
||||||
|
|
||||||
|
def test_version_is_not_the_unresolved_sentinel() -> None:
|
||||||
|
"""If this fails, the distribution name in ``__init__`` no longer
|
||||||
|
matches ``[project].name`` and every user sees a bogus version."""
|
||||||
|
assert informix_db.__version__ != "0.0.0+local", (
|
||||||
|
"__version__ fell back to its sentinel — the distribution name "
|
||||||
|
"looked up in informix_db/__init__.py does not match the installed "
|
||||||
|
"package. Check [project].name in pyproject.toml."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_version_matches_pyproject(pyproject: dict) -> None:
|
||||||
|
declared = pyproject["project"]["version"]
|
||||||
|
# PEP 440 normalisation drops leading zeros: 2026.08.27 -> 2026.8.27.
|
||||||
|
normalised = ".".join(
|
||||||
|
str(int(part)) if part.isdigit() else part
|
||||||
|
for part in declared.split(".")
|
||||||
|
)
|
||||||
|
assert informix_db.__version__ in {declared, normalised}
|
||||||
|
|
||||||
|
|
||||||
|
def test_init_looks_up_the_declared_distribution_name(pyproject: dict) -> None:
|
||||||
|
"""Catch the rename trap directly: the name passed to
|
||||||
|
``importlib.metadata.version()`` must equal ``[project].name``."""
|
||||||
|
source = (
|
||||||
|
Path(informix_db.__file__).resolve().parent / "__init__.py"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
declared_name = pyproject["project"]["name"]
|
||||||
|
assert f'version("{declared_name}")' in source, (
|
||||||
|
f"informix_db/__init__.py should call version({declared_name!r}) to "
|
||||||
|
"match [project].name in pyproject.toml"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_version_is_importable_and_stringy() -> None:
|
||||||
|
assert isinstance(informix_db.__version__, str)
|
||||||
|
assert informix_db.__version__
|
||||||
|
|
||||||
|
|
||||||
|
def test_public_surface_is_exported() -> None:
|
||||||
|
for name in informix_db.__all__:
|
||||||
|
assert hasattr(informix_db, name), f"{name} in __all__ but not defined"
|
||||||
Loading…
x
Reference in New Issue
Block a user