diff --git a/CLAUDE.md b/CLAUDE.md index 9e4b9aa..3862dc6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -349,7 +349,7 @@ firmware/ **IbusHandler** is the BMW I/K-Bus FSM — reads from transport's RX ring, validates XOR checksum, applies source filter, delivers packets via callback. -**KLineObd2** is the OBD-II handler — 5-baud slow init (bit-bang), fast init (TiniPulse), request/response framing with half-duplex echo clearing, PID convenience wrapper. +**KLineObd2** is the OBD-II handler — 5-baud slow init (bit-bang), fast init (TiniPulse), request/response with checksum validation, echo verification, structural ISO 14230 frame parsing, TesterPresent keepalive, and session re-initialization. **IbusEsp32** is preserved as a thin facade so existing BMW sniffer code needs zero changes. @@ -380,6 +380,7 @@ pio device monitor # serial monitor at 115200 ### Defensive Design (Post-Review Fixes) +**Transport layer (KLineTransport):** - **Atomic timestamps**: `_lastRxTransitionUs` is `uint32_t` (not `int64_t`) — atomic reads on 32-bit CPUs, prevents torn reads between GPIO ISR and timer callback - **TX loopback race**: idle timer reset BEFORE clearing `_isTransmitting`, with compiler memory barrier (`__asm__ volatile("" ::: "memory")`) - **Atomic TX writes**: `write()` checks free space before writing any bytes — entire message fits or entire message is dropped, no partial packets @@ -387,6 +388,17 @@ pio device monitor # serial monitor at 115200 - **FSM timeout**: `FIND_MESSAGE` state has 100ms timeout to prevent parser stall on partial messages (bus glitch, sender crash) - **TX corruption recovery**: corrupt length in TX ring flushes entire buffer; underrun mid-packet aborts TX cleanly +**OBD-II handler (KLineObd2) — hardened after Apollo code review:** +- **UART pin detach**: `pinMatrixOutDetach()` properly disconnects UART TX from GPIO before bit-bang init — `uart_set_pin(UART_PIN_NO_CHANGE)` is a no-op, not a disconnect +- **Timeout underflow guard**: `remainingMs()` helper clamps unsigned subtraction to 0 instead of wrapping to ~65s hang +- **Checksum validation**: `readResponse()` validates mod256 checksum on every received frame — corrupt data rejected, not silently accepted +- **Echo verification**: `clearEcho()` compares each echo byte against expected data — bus contention detected immediately +- **RX buffer flush**: `flushRx()` clears both UART hardware FIFO and ring buffer after init sequences — stale framing errors from bit-bang period discarded +- **Structural frame parsing**: `parseFrameData()` decodes ISO 14230 format byte (address mode bits 7:6, data length bits 5:0) — service ID found at known offset, not by byte-scanning +- **Negative response handling**: Service `0x7F` detected and NRC code logged, distinguished from timeout +- **Session keepalive**: TesterPresent (service `0x3E`) sent automatically when idle exceeds P3max (4s), preventing ECU session timeout +- **Session recovery**: Scanner tracks consecutive failures and re-initializes (slow init → fast init) after 5 total failures + ## What's Been Done 1. Optocoupler schematic analyzed (RX path, TX path, isolation topology) @@ -398,6 +410,7 @@ pio device monitor # serial monitor at 115200 7. **IbusSerial ported to ESP32** — PlatformIO project with IbusEsp32 library. Bus sniffer sketch included. Builds for ESP32, ESP32-C3, ESP32-S3. Code reviewed for ISR safety, race conditions, buffer overflows. 8. **Multi-protocol refactor (AutoWire)** — split monolithic IbusEsp32 into KLineTransport (hardware) + IbusHandler (BMW FSM). Library renamed from IbusEsp32 to AutoWire. IbusEsp32 preserved as backward-compatible facade. Zero changes to existing BMW sniffer sketch. 9. **OBD-II K-line support** — KLineObd2 handler with ISO 9141 5-baud slow init, ISO 14230 fast init (TiniPulse), request/response with half-duplex echo clearing. Obd2Pids.h with ~20 common PID decode helpers (SAE J1979). Scanner example sketch polls RPM, speed, coolant temp, throttle, voltage. +10. **OBD-II code review + hardening** — Apollo code review identified 4 critical and 7 important issues. All fixed: proper UART pin detach (`pinMatrixOutDetach`), timeout underflow guards, checksum validation, echo verification, RX buffer flush after init, structural ISO 14230 frame parsing, negative response handling, TesterPresent keepalive, and session re-initialization. ## What's Next diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..3ae5e05 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,355 @@ +# AutoWire Library Architecture + +Multi-protocol automotive bus library for ESP32. Handles BMW I/K-Bus and OBD-II K-line (ISO 9141 / ISO 14230) over a shared single-wire bus interface with PC817 optocoupler isolation. + +## Composition Diagram + +``` + Application Layer + +-----------------------------------------+ + | main.cpp (sniffer) | + | | + | IbusEsp32 ibus; | + | ibus.begin(Serial1, RX, TX, LED); | + | ibus.onPacket(callback); | + | ibus.run(); | + +-----------------------------------------+ + + | (facade: zero-change migration) + v + + +-------------------+ +-------------------+ + | IbusHandler | | KLineObd2 | + | | | | + | BMW I/K-Bus FSM | | ISO 9141 slow init| + | XOR checksum | | ISO 14230 fast | + | Source filtering | | init | + | Packet callback | | Echo clearing | + | | | Frame parsing | + | | | TesterPresent | + +--------+----------+ +---------+---------+ + | | + | reads RX ring | reads RX ring + | delegates TX | sends via serial() + | | flushes RX + v v + +-----------------------------------------+ + | KLineTransport | + | | + | UART (HardwareSerial) | + | GPIO ISR (IRAM_ATTR, RX pin) | + | esp_timer (periodic idle check) | + | RX ring buffer (256 bytes) | + | TX ring buffer (128 bytes) | + | Bus idle detection | + | TX inversion (uart_set_line_inverse) | + | Configurable: KLineConfig struct | + +-----------------------------------------+ + + | + v + + +-----------------------------------------+ + | RingBuffer | + | | + | malloc'd circular buffer | + | Bounds-checked peek(n), remove(n) | + | capacity = size - 1 (standard ring) | + +-----------------------------------------+ +``` + +IbusHandler and KLineObd2 sit as parallel protocol handlers on top of a shared KLineTransport. Neither knows about the other. IbusEsp32 is a facade that composes one transport and one handler, preserving the original API for existing sketches. + +## File Inventory + +``` +firmware/lib/AutoWire/ + library.json PlatformIO metadata (v2026.02.13) + KLineTransport.h/cpp Shared hardware layer + IbusHandler.h/cpp BMW I/K-Bus protocol + KLineObd2.h/cpp OBD-II K-line protocol + IbusEsp32.h/cpp Backward-compatible facade + RingBuffer.h/cpp Circular byte buffer + E46Codes.h ~100 BMW E46 command arrays (namespace ibus{}) + Obd2Pids.h OBD-II PID constants + decode helpers (namespace obd2{}) +``` + +## Why Composition (not Inheritance) + +Transport and protocol are independent concerns. KLineTransport handles UART configuration, pin management, interrupt routing, ring buffers, and bus idle timing. IbusHandler parses BMW message framing. KLineObd2 runs ISO 9141/14230 init sequences and request/response cycles. These share no protocol logic with each other. + +Both handlers need ring buffer access, but they use it differently. IbusHandler peeks ahead into the RX ring without consuming bytes until a complete message with valid checksum is found. KLineObd2 reads bytes one at a time with timeouts, blocking until a response arrives. Inheritance would force a common interface where none exists. + +No virtual functions. On ESP32 (Xtensa/RISC-V, single-core Arduino loop), vtable dispatch adds overhead and indirection for no benefit. The handler type is known at compile time. References are resolved directly. + +The facade (IbusEsp32) preserves the original API so existing sketches like `main.cpp` need zero changes. It owns the transport and handler as member variables and delegates every call: + +```cpp +// IbusEsp32.cpp — the entire implementation +void IbusEsp32::run() { + _transport.drainUart(); + _handler.process(); + _transport.run(); +} +``` + +New code that needs both protocols (or just OBD-II) can use KLineTransport and KLineObd2 directly, bypassing the facade entirely. + +## KLineTransport + +The shared hardware layer. Owns all physical resources: UART peripheral, GPIO interrupt, esp_timer, and both ring buffers. + +### Configuration + +Everything protocol-specific is parameterized through `KLineConfig`: + +```cpp +struct KLineConfig { + uint32_t baud = 9600; // 9600 for BMW, 10400 for OBD-II + uint32_t framing = SERIAL_8E1; // 8E1 for BMW, 8N1 for OBD-II + uint16_t idleTimeoutUs = 1500; // 0 = no idle detect (master/slave) + uint16_t idleCheckUs = 250; // timer resolution + uint16_t packetGapMs = 10; // min gap between own packets + bool txInvert = true; // true for optocoupler, false for transistor + ChecksumType checksumType = CHECKSUM_XOR; // XOR or MOD256 +}; +``` + +Setting `idleTimeoutUs = 0` disables bus idle detection entirely. The transport sets `_clearToSend = true` at init and keeps it there. This is the correct mode for OBD-II K-line, which is master/slave -- the tester owns the bus after init, there is no contention to detect. + +Setting `idleTimeoutUs = 1500` enables multi-master mode for BMW I/K-Bus. The transport monitors the RX pin for bus activity and only permits TX after 1.5ms of silence. + +### TX Inversion + +The PC817 optocoupler inverts the TX signal (TX HIGH drives the bus LOW). KLineTransport calls `uart_set_line_inverse(UART_SIGNAL_TXD_INV)` when `txInvert = true`, so the UART peripheral itself handles the inversion. No software bit-flipping needed. + +For direct transistor circuits (no optocoupler), set `txInvert = false`. + +### Ring Buffer Access + +Protocol handlers access the RX ring through transport methods: + +- `rxAvailable()` -- bytes waiting in the ring +- `rxPeek(n)` -- look at byte n without consuming (IbusHandler uses this heavily) +- `rxRemove(n)` -- discard n bytes (after bad checksum or filter reject) +- `rxRead()` -- consume one byte (KLineObd2 reads byte-by-byte with timeouts) +- `flushRx()` -- drain both UART FIFO and ring (KLineObd2 calls this after init sequences) +- `drainUart()` -- move bytes from UART FIFO into ring buffer + +The transport also exposes `serial()` for direct UART access. KLineObd2 needs this for init sequences where it writes individual bytes and reads echoes outside the normal ring buffer flow. + +### TX Queue + +`write()` appends a framed message to the TX ring: a length prefix byte, the message bytes, and a checksum byte (computed per `config.checksumType`). The entire message must fit or the entire message is dropped -- no partial writes. + +`sendRaw()` bypasses the TX ring and writes directly to the UART. KLineObd2 uses this for init pulse sequences that do not follow normal message framing. + +TX scheduling happens in `run()` -> `trySend()`: + +1. Check `_clearToSend` (bus idle) +2. Check TX ring has data +3. Check `packetGapMs` has elapsed since last TX +4. Send next packet from ring + +### Pin Exposure + +KLineObd2 needs to detach the UART from the TX pin during 5-baud slow init (bit-bang at 200ms per bit) and fast init (25ms LOW/HIGH pulse). The transport exposes `txPin()` and `uartNum()` for this. After init, KLineObd2 re-attaches the UART with `uart_set_pin()`. + +## IbusHandler + +BMW I/K-Bus protocol parser. Takes a reference to KLineTransport and reads from its RX ring buffer. + +### State Machine + +``` +FIND_SOURCE --> FIND_LENGTH --> FIND_MESSAGE --> GOOD_CHECKSUM --> (callback, back to FIND_SOURCE) + ^ | | | + | | | v + +----- bad ----+ timeout (100ms) BAD_CHECKSUM ----> remove 1 byte, retry + | remove 1 byte + +<-----------------------------+ +``` + +The FSM peeks into the RX ring without consuming bytes until it has a complete, checksum-validated message. On `GOOD_CHECKSUM`, it reads the bytes out of the ring and fires the callback. On `BAD_CHECKSUM`, it removes one byte (the presumed bad source byte) and restarts -- this slides the window forward through noise or framing errors. + +The 100ms timeout in `FIND_MESSAGE` prevents the parser from stalling on partial messages. At 9600 baud, a maximum-length I-Bus message (36 data bytes + overhead = ~40 bytes) takes approximately 44ms. If the remaining bytes never arrive (sender crashed, bus glitch), the timeout fires, removes the source byte, and the FSM restarts. + +### Checksum + +XOR of all bytes from source through the last data byte. The result must equal the final byte of the message. This is computed by peeking through the ring without consuming: + +```cpp +uint8_t checksum = 0; +for (int i = 0; i <= _length; i++) { + checksum ^= _transport.rxPeek(i); +} +if (_transport.rxPeek(_length + 1) == checksum) { ... } +``` + +### Source Filtering + +Optional. When enabled, only messages from specified source addresses pass through to the callback. Others are silently discarded (bytes removed from ring). Up to 16 filter addresses. Off by default (sniffer mode -- all traffic passes through). + +### Packet Callback + +```cpp +using PacketCallback = std::function; +``` + +The callback receives the complete raw message including source, length, destination, data, and checksum bytes. The `length` parameter is total byte count (not the length field from byte 1). The callback runs synchronously inside `process()` -- keep it short. + +The activity LED toggles around the callback: `ledOn()` before, `ledOff()` after. + +## KLineObd2 + +OBD-II K-line handler for ISO 9141-2 (5-baud slow init) and ISO 14230 / KWP2000 (fast init). Takes a reference to KLineTransport. Operates in a blocking request/response style -- call `requestPid()` and it blocks until the ECU responds or times out. + +### 5-Baud Slow Init (ISO 9141-2) + +Full sequence: + +1. Detach UART TX from GPIO pin via `pinMatrixOutDetach()` +2. Hold TX HIGH for 300ms (W0 idle period) +3. Bit-bang target address (default 0x33) at 5 baud: 200ms per bit, LSB first, start/stop framing. Total: 2 seconds for one byte +4. Re-attach UART TX via `uart_set_pin()` +5. Flush RX (UART received garbage framing errors during the 2-second bit-bang) +6. Read sync byte (0x55) -- ECU confirms baud rate +7. Read keyword bytes (KW1, KW2) -- ECU declares protocol capabilities +8. Send inverted KW2 back to ECU -- tester confirms handshake +9. Verify echo of inverted KW2 (half-duplex bus) +10. Read inverted target address from ECU -- final acknowledgment + +After success, `_initialized = true` and the session is active. + +### Fast Init (ISO 14230, KWP2000) + +1. Detach UART TX, hold HIGH 300ms +2. TiniPulse: 25ms LOW + 25ms HIGH on TX pin +3. Re-attach UART, flush RX +4. Send StartCommunication request (service 0x81, functional addressing) +5. Verify echo (5 bytes) +6. Read and parse StartComm positive response (0xC1 = 0x81 + 0x40) +7. Extract keyword bytes from response + +### Half-Duplex Echo + +K-line is a single wire. Everything you transmit appears on the RX line as well. `clearEcho()` reads back each transmitted byte and compares against the expected value. A mismatch means bus contention -- another device was transmitting simultaneously. + +### ISO 14230 Frame Parsing + +The format byte encodes both address mode and data length: + +``` +Bits 7-6: Address mode (0=none, 2=physical, 3=functional) +Bits 5-0: Data length (0 = separate length byte follows address bytes) +``` + +`parseFrameData()` decodes this structure and returns the index of the first data byte (service ID position). Both `readResponse()` and `requestPid()` use this to locate response data regardless of header format. + +### TesterPresent Keepalive + +ECUs drop the diagnostic session after the P3 timeout (~5 seconds). `requestPid()` checks elapsed time since the last request and sends TesterPresent (service 0x3E) automatically if more than 4 seconds have passed. This is transparent to the caller. + +### Negative Response Handling + +If the ECU returns service 0x7F (negative response), `requestPid()` returns 0. In debug mode, it prints the NRC (Negative Response Code). No automatic retry -- the caller decides what to do. + +### Session Recovery + +Call `reset()` to clear the `_initialized` flag. Then call `slowInit()` or `fastInit()` again. This is the correct response after a bus error, timeout, or P3 session expiration. + +## IbusEsp32 (Facade) + +Backward-compatible wrapper. Owns a `KLineTransport` and an `IbusHandler` as member variables: + +```cpp +class IbusEsp32 { +private: + KLineTransport _transport; + IbusHandler _handler; +}; +``` + +The constructor initializes `_handler` with a reference to `_transport`. `begin()` populates a `KLineConfig` from the `IBUS_*` defines and calls `_transport.begin()`. Every other method is a one-line delegation. + +Existing code like the bus sniffer in `main.cpp` uses `IbusEsp32` and needs no changes: + +```cpp +IbusEsp32 ibus; +ibus.begin(Serial1, RX_PIN, TX_PIN, LED_PIN); +ibus.onPacket(callback); +// loop: +ibus.run(); +``` + +The `IBUS_*` defines in `config.h` are re-exported by `IbusEsp32.h` with defaults, so `#include "IbusEsp32.h"` still works without `#include "config.h"`. The `KLINE_*` buffer size defines in `KLineTransport.h` fall through to `IBUS_*` defines if present, maintaining backward compatibility with existing `platformio.ini` build flags. + +## Protocol Comparison + +| Feature | BMW I/K-Bus | OBD-II K-line | +|---|---|---| +| Baud | 9600 | 10400 | +| Framing | 8E1 (even parity) | 8N1 (no parity) | +| Checksum | XOR of all bytes | Sum mod 256 | +| Bus model | Multi-master | Master/slave | +| Init sequence | None (always-on bus) | 5-baud slow or fast init pulse | +| Idle detection | 1.5ms quiet before TX | Not needed (tester owns bus) | +| Message format | Source + Length + Dest + Data + XOR | Format + [Addr] + [Len] + Data + Sum | +| TX style | Queued (ring buffer, async) | Direct (blocking request/response) | +| Session | None (stateless bus) | Maintained (P3 keepalive required) | +| TX inversion | Yes (optocoupler) | No (transistor circuit) | + +## Concurrency Model + +Three execution contexts share state: + +1. **GPIO ISR** (`onRxPinChange`, IRAM_ATTR) -- runs on any RX pin edge. Records `_lastRxTransitionUs` and sets `_clearToSend = false`. Returns immediately if `_isTransmitting` is true (suppresses loopback echo from own TX). + +2. **esp_timer callback** (`onIdleTimerCallback`) -- runs every 250us from timer interrupt context. Checks if `(now - _lastRxTransitionUs) >= idleTimeoutUs`. If so, sets `_clearToSend = true`. Skips check if already clear or currently transmitting. + +3. **Arduino loop** (`run()` / `trySend()` / `sendNextPacket()`) -- reads `_clearToSend` to decide whether to transmit. Sets `_isTransmitting = true` before writing to UART, clears it after `flush()` + 200us settling delay. + +### Atomicity + +`_lastRxTransitionUs` is `uint32_t`, not `int64_t`. On 32-bit ESP32 (Xtensa LX6/LX7, RISC-V), 32-bit aligned reads and writes are atomic. This prevents torn reads where the ISR updates the timestamp while the timer callback is reading it. + +`_clearToSend` and `_isTransmitting` are `volatile bool`. Single-byte writes are atomic on all ESP32 variants. + +### TX Loopback Race + +When transmitting, the UART TX data appears on the RX pin (optocoupler loopback or half-duplex bus echo). Without protection, the GPIO ISR would interpret this as bus activity from another module and clear `_clearToSend`, blocking subsequent transmissions. + +The fix: set `_isTransmitting = true` before writing, and the ISR checks it first: + +```cpp +void IRAM_ATTR KLineTransport::onRxPinChange(void* arg) { + KLineTransport* self = static_cast(arg); + if (self->_isTransmitting) return; // ignore own loopback + ... +} +``` + +After the last byte is flushed, the transport resets the idle timer and clears `_isTransmitting` with a compiler memory barrier between them: + +```cpp +_lastRxTransitionUs = (uint32_t)esp_timer_get_time(); +_clearToSend = false; +__asm__ __volatile__("" ::: "memory"); +_isTransmitting = false; +``` + +The barrier ensures the idle timer reset is visible to the timer callback before `_isTransmitting` goes false. Without it, the compiler could reorder the stores and the timer callback might see `_isTransmitting = false` while `_clearToSend` is still stale, potentially granting a premature clear-to-send. + +## Defensive Design + +- **Atomic TX writes**: `write()` checks free space in the TX ring before writing anything. If the entire message (length prefix + body + checksum) does not fit, the whole message is dropped. No partial packets in the ring. + +- **TX corruption recovery**: `sendNextPacket()` reads the length prefix from the TX ring. If it is zero, negative, or exceeds `KLINE_MAX_MSG`, the entire ring is flushed. Same if the ring has fewer bytes than the length claims (underrun). This prevents one corrupt message from poisoning all subsequent transmissions. + +- **Ring buffer bounds**: `peek(n)` validates n against `available()`, returns -1 if out of range. `remove(n)` clamps to `available()`. `malloc` failure sets `_size = 0`, disabling all operations rather than dereferencing null. + +- **FSM timeout**: The `FIND_MESSAGE` state has a 100ms watchdog. If the expected bytes never arrive, the FSM drops the candidate source byte and restarts. This prevents a permanent stall from a truncated message (bus glitch, sender crash mid-packet). + +- **RX flush after init**: KLineObd2 calls `flushRx()` after both slow and fast init. During the 2-second 5-baud bit-bang, the UART RX is receiving framing errors at whatever the bit-bang edges look like at 10400 baud. These garbage bytes must be discarded before reading the ECU response. + +- **Echo verification**: `clearEcho()` reads back every transmitted byte and compares against expected. Mismatch indicates bus contention (another device was transmitting). Init sequences abort on echo mismatch rather than trying to parse a corrupted response. diff --git a/docs/circuit-design.md b/docs/circuit-design.md new file mode 100644 index 0000000..93c459f --- /dev/null +++ b/docs/circuit-design.md @@ -0,0 +1,317 @@ +# PC817 Optocoupler Interface — Circuit Design and SPICE Validation + +BMW I/K-Bus interface for ESP32, galvanically isolated via PC817 optocouplers. Based on the [muki01/I-K_Bus](https://github.com/muki01/I-K_Bus) optocoupler schematic, modified for 3.3V ESP32 operation (R2: 470 to 220 ohm). + +--- + +## Circuit Overview + +The interface sits between a 12V automotive bus (BMW I-Bus or K-Bus) and a 3.3V ESP32 microcontroller. Four key elements: + +- **U1 (PC817)** -- RX optocoupler. Converts 12V bus signals to 3.3V logic for the ESP32's UART RX pin. Emitter-follower configuration preserves signal polarity. +- **U2 (PC817)** -- TX optocoupler. Converts 3.3V logic from the ESP32's UART TX pin to open-collector drive on the 12V bus. Signal is inverted. +- **Q1 (BC547)** -- NPN transistor. Drives U2's infrared LED. The ESP32 cannot source enough current through its GPIO to drive the LED directly; Q1 provides the current gain. +- **D1 (1N4007)** -- Bridges bus-side 12V to the MCU power rail (through a 3.3V regulator). Blocks reverse current when the MCU is USB-powered during development. + +Galvanic isolation is maintained across both optocouplers. The only electrical connection between bus-side and MCU-side ground planes is through the power supply path (D1 + regulator). Signal paths are optically coupled. + +### Schematic + +``` + I/K-Bus Side (12V) MCU Side (3.3V) + ================== ================ + + 12V ────── D1 (1N4007) ──────────────── 12V (to regulator) + (reverse protection) + + I/K-Bus ─── R1 (2k) ──> U1 pin 1 U1 pin 4 <── 3.3V + (LED anode) (PC817) (collector) + GND <── U1 pin 2 U1 pin 3 ──> RX_Pin + (LED cathode) (emitter) + R4 (1k) pull-down to GND + + : photons : + + I/K-Bus <── U2 pin 4 U2 pin 1 <── R2 (220) + (collector) (LED anode) | + U2 pin 3 U2 pin 2 <── Q1 collector (BC547) + (emitter) (LED cathode) + | Q1 base <── R3 (10k) <── R5 (470) <── TX_Pin + GND Q1 emitter ──> GND +``` + +Reference images in `reference/`: +- `muki01-optocoupler-schematic.png` -- original schematic from muki01 +- `muki01-transistor-schematic.png` -- non-isolated alternative (not used here) + +--- + +## RX Path (Bus to MCU via U1) + +U1 converts 12V bus signals to 3.3V logic. The phototransistor is wired in emitter-follower configuration: collector tied to VCC (3.3V), emitter to the RX pin, with R4 (1k) pulling the emitter to ground. + +**Bus HIGH (12V idle):** +Current flows through R1 (2k) into U1's LED. I_LED = (12V - 1.2V) / 2k = 5.4mA. At this drive level the PC817's CTR (minimum 80% for grade A) produces sufficient base current to saturate the phototransistor. The emitter follows the collector voltage, pulling RX toward VCC. Result: RX = HIGH. + +**Bus LOW (module transmitting):** +No voltage across R1, no LED current, phototransistor turns off. R4 (1k) pulls RX to ground. Result: RX = LOW. + +**Signal polarity is preserved.** The emitter-follower does not invert. Bus HIGH maps to RX HIGH, bus LOW maps to RX LOW. No software inversion needed on the RX path. + +### RX Simulation Results + +Netlist: `reference/ibus_rx_path.cir` +Test stimulus: byte `0x50` (MFL address) at 9600 baud, 8E1 framing. Bus modeled as 12V source through 100 ohm source impedance. + +| Measurement | Value | Threshold | Status | +|---|---|---|---| +| V(RX) HIGH | 3.128V | > 2.475V (ESP32 VIH) | Pass | +| V(RX) LOW | ~0V | < 0.825V (ESP32 VIL) | Pass | +| LED current (peak) | 5.21mA | -- | -- | +| Rise time (10-90%) | 5.8us | << 104.17us bit time | Pass | + +The 5.8us rise time consumes 5.6% of the 104.17us bit period at 9600 baud. Sampling at bit center (52us into the bit) sees a fully settled signal. + +--- + +## TX Path (MCU to Bus via U2 + Q1) + +U2's phototransistor is wired in common-emitter configuration: collector connected to the bus line, emitter to ground. When the phototransistor conducts, it pulls the bus LOW against whatever external pull-up impedance exists on the bus. + +Q1 (BC547) drives U2's LED. The ESP32 TX pin sources current through R5 (470 ohm) and R3 (10k) into Q1's base. + +**TX HIGH (UART idle):** +Q1 base is driven. Q1 saturates (Vce_sat = 0.073V from simulation). Current flows from VCC through R2 (220 ohm) into U2's LED, through Q1 to ground. I_LED = (3.3V - 1.2V - 0.073V) / 220 = 9.21mA (simulated: 9.71mA). U2's phototransistor saturates, pulling the bus line toward ground. + +**TX LOW (transmitting a UART bit):** +Q1 turns off. No LED current. U2's phototransistor turns off. The bus is released and returns HIGH via the bus pull-up resistor. + +**Signal is inverted.** TX HIGH produces bus LOW. TX LOW produces bus HIGH. The firmware compensates with `uart_set_line_inverse(UART_SIGNAL_TXD_INV)` on the ESP32's UART peripheral, which inverts the TX signal in hardware before it reaches the GPIO pin. + +### TX Simulation Results + +Netlist: `reference/tx_validated_r2_220.cir` +Test conditions: 3.3V VCC, R2=220 ohm, 1k ohm bus pull-up to 12V (worst realistic BMW bus loading). Byte `0x68` (RAD address) at 9600 baud. + +| Measurement | Value | Notes | +|---|---|---| +| V(IBUS) LOW | 0.266V | U2 pulls bus to near ground | +| V(IBUS) HIGH | 8.35V | Bus sags from 12V due to U1's RX LED load (5.4mA through R1) | +| V(IBUS) swing | 8.08V | 0.27V to 8.35V | +| U2 LED current | 9.71mA | Through R2 (220 ohm) when Q1 is on | +| Q1 Vce(sat) | 0.073V | Fully saturated | +| Bus rise time (10-90%) | 9.1us | 8.7% of 104.17us bit time | +| RX loopback HIGH | ~3.1V | U1 sees its own TX through the bus | +| RX loopback LOW | ~0V | Confirms end-to-end signal path | + +The 9.1us bus rise time is passive -- set by the RC time constant of the bus pull-up and parasitic capacitance. On a real BMW bus, the TH3122 transceivers in other modules provide low-impedance active drive. Actual rise times will be similar or faster. + +--- + +## The R2 Fix: 470 ohm to 220 ohm for 3.3V ESP32 + +This is the single most important modification from the original muki01 design. + +### The Problem + +The muki01 schematic specifies R2=470 ohm, designed for 5V Arduino. With a 5V supply, the LED current through U2 is: + +``` +I_LED = (5.0V - 1.2V - Vce_sat) / 470 = ~7.9mA +``` + +At 3.3V, this drops to: + +``` +I_LED = (3.3V - 1.2V - Vce_sat) / 470 = ~4.3mA +``` + +The phototransistor's collector current is limited by CTR * I_LED. At 4.3mA LED drive and worst-case CTR (80% for PC817A), the maximum collector current is only 3.4mA. Against a 1k ohm bus pull-up (12V / 1k = 12mA needed to pull bus to ground), the phototransistor cannot sink enough current. The bus voltage stays well above the LOW threshold of downstream modules. + +### Three Parameter Sweeps + +Three `.step` simulations explored the operating envelope. Each sweep varies one parameter while holding others at worst-case or typical values. + +#### Sweep 1: R2 Value (Worst-Case CTR, 4.7k Bus) + +Netlist: `reference/tx_sweep_r2.cir` +Conditions: Igain=0.5m (PC817A minimum CTR, ~50%), R_PULL=4.7k ohm. +Stepped: R2 = 100, 150, 220, 270, 330, 390, 470 ohm. + +| R2 (ohm) | V(IBUS) LOW | LED Current | Status | +|---|---|---|---| +| 100 | 0.128V | 13.0mA | Pass | +| 150 | 0.147V | 9.4mA | Pass | +| 220 | 0.176V | 7.97mA | Pass | +| 270 | 0.189V | 5.91mA | Pass | +| 330 | 0.211V | 5.59mA | Pass | +| 390 | 0.514V | 4.66mA | Marginal | +| 470 | ~0.7V+ | 4.26mA | Fail | + +At worst-case CTR, R2=330 ohm is the threshold. R2=390 ohm and above cannot pull the bus low enough. R2=220 ohm provides margin. + +#### Sweep 2: Bus Impedance (R2=470, Typical CTR) + +Netlist: `reference/tx_sweep_rpull.cir` +Conditions: R2=470 ohm (original), Igain=1m (PC817A typical CTR, ~100%). +Stepped: R_PULL = 510, 1000, 2200, 3300, 4700, 6800, 10000 ohm. + +| R_PULL (ohm) | V(IBUS) LOW | Status | +|---|---|---| +| 510 | 4.58V | Fail | +| 1000 | 0.268V | Pass | +| 2200 | 0.123V | Pass | +| 3300 | 0.099V | Pass | +| 4700 | 0.082V | Pass | +| 6800 | 0.070V | Pass | +| 10000 | 0.062V | Pass | + +With the original R2=470 ohm, the design fails at 510 ohm bus impedance even with typical (not worst-case) CTR. A loaded BMW bus can present impedances in the low kilohm range. + +#### Sweep 3: CTR Grade (R2=470, 1k Bus) + +Netlist: `reference/tx_sweep_ctr.cir` +Conditions: R2=470 ohm (original), R_PULL=1k ohm (loaded bus). +Stepped: Igain = 0.3m, 0.5m, 0.8m, 1.0m, 1.5m, 2.3m, 3.4m (mapping to PC817 grades A through D). + +| Igain | Approx. Grade | V(IBUS) LOW | Status | +|---|---|---|---| +| 0.3m | Below A | 6.89V | Fail | +| 0.5m | A (min) | 4.41V | Fail | +| 0.8m | A (typ) | 2.51V | Marginal | +| 1.0m | A (max) / B (min) | 0.18V | Pass | +| 1.5m | B (typ) | 0.10V | Pass | +| 2.3m | C | 0.07V | Pass | +| 3.4m | D | 0.05V | Pass | + +With R2=470 ohm against a 1k bus, even a typical PC817A (Igain=0.8m) only manages V_LOW=2.51V -- marginal at best. You'd need to cherry-pick PC817B or better parts. + +### The Fix + +**R2=220 ohm.** This doubles the LED current from ~4.3mA to ~9.7mA. The increased drive current produces roughly double the phototransistor base current, significantly extending the collector current capability. + +Validated in `reference/tx_validated_r2_220.cir` against worst-case realistic conditions (1k bus pull-up, PC817A typical CTR): +- V(IBUS) LOW = 0.266V (well below any logic threshold) +- LED current = 9.71mA (19% of the PC817's 50mA absolute max rating) +- Supports bus pull-ups down to approximately 530 ohm + +No other component changes required. + +--- + +## K-Line Compatibility Analysis (510 ohm Pull-Up) + +The optocoupler circuit was also evaluated for potential use on OBD-II K-line (a sibling project that uses a transistor interface). K-line has a 510 ohm pull-up to 12V per ISO 9141, requiring the transmitter to sink approximately 23.5mA. + +Two additional netlists tested this scenario: + +**R2 sweep at 510 ohm pull-up** (`reference/opto_vs_kline_510.cir`): +Stepped R2 from 47 to 470 ohm at 10400 baud (K-line rate), typical CTR (Igain=1m). + +At R2=220 ohm (the BMW fix value), the phototransistor maxes out at approximately 22.4mA of collector current -- right at the edge of the 23.5mA needed. The bus voltage sits around 0.5-1V, which may or may not be interpreted as LOW depending on the ECU's input threshold. + +Only R2=100 ohm (LED current ~20.5mA) reliably pulls the bus below 0.3V at 510 ohm. + +**CTR grade sweep at 510 ohm, R2=100** (`reference/opto_vs_kline_ctr.cir`): +Even with R2=100 ohm, worst-case PC817A (Igain=0.5m) fails at 510 ohm bus impedance. Typical CTR and above passes. + +**Conclusion:** The optocoupler circuit is marginal for K-line's 510 ohm pull-up. A direct transistor interface (BC547 or MOSFET open-drain) is the better choice for OBD-II K-line -- it does not have the CTR limitation and can sink arbitrary current up to its collector rating. The optocoupler design is correct for BMW I/K-Bus, where bus impedances are in the kilohm range. + +--- + +## Bill of Materials + +| Ref | Value | Package | Qty | Notes | +|---|---|---|---|---| +| U1, U2 | PC817A+ | DIP-4 | 2 | Optocoupler, any grade (A/B/C/D). PC817A works; B+ adds margin | +| Q1 | BC547 | TO-92 | 1 | Or 2N3904. TX LED driver | +| R1 | 2k | 0805/TH | 1 | RX LED current limiter | +| R2 | 220 | 0805/TH | 1 | TX LED current limiter (reduced from 470 for 3.3V ESP32) | +| R3 | 10k | 0805/TH | 1 | Q1 base resistor | +| R4 | 1k | 0805/TH | 1 | RX pull-down (defines LOW when opto OFF) | +| R5 | 470 | 0805/TH | 1 | TX input series resistor | +| D1 | 1N4007 | DO-41 | 1 | Reverse polarity protection, 12V bridge | +| U3 | ESP32-C3 | Module | 1 | Any ESP32 with hardware UART | +| J1 | Bus connector | -- | 1 | OBD-II, roundel, or direct wire to CD changer connector | + +**Optional for permanent install:** +- AMS1117-3.3 or MP1584 buck converter: 12V to 3.3V for ESP32 +- 10uF + 100nF ceramic caps on 3.3V rail +- TVS diode (SMBJ16A) on bus line for transient protection +- Status LED on spare GPIO (not in signal path) + +--- + +## Design Notes + +### 1. Bus Rise Time Depends on External Impedance + +The optocoupler's phototransistor is an open-collector (common-emitter) output. It can actively pull the bus LOW, but the return to HIGH is passive -- set by the bus pull-up resistor and parasitic capacitance. The simulated 9.1us rise time (with 1k pull-up, 100pF) is 8.7% of the 104.17us bit time at 9600 baud. + +On a real BMW bus, other modules have TH3122 transceivers with active push-pull outputs. These provide low-impedance drive in both directions. The actual rise time on the vehicle bus will be similar to or faster than the simulation. + +### 2. Bench Testing Requires a Low-Impedance 12V Source + +U1's RX LED draws 5.4mA continuously through R1 whenever the bus is at 12V (which is always during idle). If the bench test setup uses only a passive pull-up resistor (e.g., 4.7k to 12V), the bus voltage sags significantly under this load: + +- 4.7k pull-up: V_bus drops to ~4.3V (most of the current goes through R1) +- 1k pull-up: V_bus drops to ~8.4V + +Use a 12V power supply through 100-510 ohm to simulate a real bus source impedance during development. Or better, a dedicated 12V rail with a series resistor matching the expected bus impedance. + +### 3. PC817 Grade Selection + +PC817A (CTR 80-160%) is the cheapest and most widely available grade. With R2=220 ohm, it works for BMW I/K-Bus across the full range of realistic bus impedances. PC817B (CTR 130-260%) or higher grades provide additional margin with no circuit changes -- drop-in replacement, same pinout, same package. + +The sweep netlists (`reference/tx_sweep_*.cir`) document the exact operating envelope for each grade. + +### 4. Parasitic Drain + +The interface draws approximately 5.4mA continuously from the bus 12V through R1. This is the U1 RX LED current, always on when the bus is at idle voltage. U2's LED only draws current during TX (9.7mA pulses, duty cycle dependent on message frequency). + +For a vehicle parked long-term, 5.4mA from a ~60Ah battery is roughly 460 days to full discharge (in theory). In practice, the vehicle's own parasitic drain is much higher. Still, for multi-week parking, either disconnect the interface or add a sleep circuit (MOSFET switch on R1, controlled by a GPIO or the vehicle's terminal 15 signal). + +### 5. PC817 SPICE Model + +All simulations use the PC817 subcircuit inlined from LTspice's library (`lib/sub/PC817.sub`). The model uses a voltage-controlled current source (VCCS, `G1`) with a gain parameter `{Igain}` to represent the optical coupling. Pin order: 1=Anode, 2=Cathode, 3=Collector, 4=Emitter. + +The `Igain` parameter maps to CTR as follows: +- `Igain=0.5m` -- PC817A minimum (~50% effective CTR in the model) +- `Igain=1m` -- PC817A typical (~100%) +- `Igain=1.5m` -- PC817B typical (~150%) +- `Igain=2.3m` -- PC817C typical (~230%) +- `Igain=3.4m` -- PC817D typical (~340%) + +These are approximate. The SPICE model includes a phototransistor (NPN with Bf=1200) driven by the VCCS, so the actual transfer ratio is nonlinear at high drive levels. + +--- + +## SPICE Netlist Inventory + +All files in `reference/`. Simulations target 9600 baud (BMW I/K-Bus) unless noted. + +| File | Description | +|---|---| +| `ibus_rx_path.cir` | RX path: bus byte 0x50 at 9600 baud through U1 (PC817) to ESP32 RX. Measures V(RX) levels, LED current, rise/fall times. 3.3V VCC, 100 ohm bus source impedance. | +| `ibus_tx_path.cir` | TX path: ESP32 TX through Q1 (BC547B) and U2 (PC817) to bus. Alternating pulses, 4.7k bus pull-up. Includes U1 RX loopback. R2=220 ohm, Igain=1m. | +| `tx_validated_r2_220.cir` | TX validation: R2=220 ohm fix against worst realistic loading (1k bus pull-up). Byte 0x68 at 9600 baud. Measures bus swing, rise/fall times, Q1 Vce_sat, RX loopback. The primary validation netlist for the R2 fix. | +| `tx_sweep_r2.cir` | Parameter sweep: R2 from 100 to 470 ohm. Worst-case CTR (Igain=0.5m), 4.7k bus pull-up. Single TX pulse. Identifies R2=330 ohm as the pass/fail threshold. | +| `tx_sweep_rpull.cir` | Parameter sweep: bus pull-up from 510 to 10k ohm. R2=470 ohm (original), typical CTR (Igain=1m). Shows failure at 510 ohm, pass at 1k and above. Demonstrates why R2=470 is insufficient for loaded buses. | +| `tx_sweep_ctr.cir` | Parameter sweep: CTR grade (Igain 0.3m to 3.4m). R2=470 ohm, 1k bus pull-up. Shows PC817A (typical) is marginal, PC817B+ passes. Motivated the R2 reduction as an alternative to grade selection. | +| `opto_vs_kline_510.cir` | K-line compatibility: R2 sweep (47 to 470 ohm) against 510 ohm pull-up at 10400 baud. Typical CTR. Shows R2=220 ohm is marginal, R2=100 ohm is the minimum for K-line. | +| `opto_vs_kline_ctr.cir` | K-line compatibility: CTR grade sweep at 510 ohm pull-up, R2=100 ohm, 10400 baud. Shows worst-case PC817A fails even with aggressive R2. Confirms transistor circuit is preferred for K-line. | + +Waveform plots (SVG, from initial simulations): + +| File | Description | +|---|---| +| `rx_path_vibus.svg` | Bus voltage waveform during RX simulation | +| `rx_path_vrx.svg` | ESP32 RX pin voltage during RX simulation | +| `tx_path_vibus.svg` | Bus voltage waveform during TX simulation | +| `tx_path_vrx_loopback.svg` | RX loopback voltage during TX simulation | + +--- + +## Attribution + +Circuit design from [muki01/I-K_Bus](https://github.com/muki01/I-K_Bus) (MIT license). R2 modification, SPICE validation, and 3.3V analysis are original work for this project. diff --git a/docs/development-journal.md b/docs/development-journal.md new file mode 100644 index 0000000..91c485c --- /dev/null +++ b/docs/development-journal.md @@ -0,0 +1,461 @@ +# Development Journal - BMW I/K-Bus Interface Board + +This is a working log of the BMW I/K-Bus interface board project. It started as a straightforward port of muki01's optocoupler design for an ESP32 and ended up becoming a multi-protocol automotive bus library. The entries here are roughly chronological, split across four working sessions. I'm keeping track of what actually happened - including the dumb mistakes and dead ends - because that's the stuff you need when you come back to a project six months later and wonder "why did I do it that way?" + +The hardware side is based on [muki01/I-K_Bus](https://github.com/muki01/I-K_Bus), which is an optocoupler-isolated interface for BMW's proprietary body/instrumentation bus. The original design targets a 5V Arduino Nano. We're targeting a 3.3V ESP32. That voltage difference turned out to matter more than I expected. + +--- + +## Session 1: SPICE Simulation of the Optocoupler Circuit + +### Starting Point + +The project kicked off with a design brief already written up - the full CLAUDE.md had the schematic, protocol spec, module address map, the works. What it didn't have was any proof the circuit actually works at 3.3V. The muki01 design was built for 5V Arduino, and we're running on an ESP32 at 3.3V. Time to fire up the simulator. + +The tool here is LTspice via the `mcp-ltspice` MCP server, which lets us build and simulate SPICE netlists programmatically. I was half expecting to need to build a PC817 model from scratch - optocoupler models can be weird because you're dealing with an LED optically coupled to a phototransistor, and not every SPICE library bothers to include them. + +Turns out LTspice already ships with a PC817 subcircuit in `lib/sub/PC817.sub`. Nice. It uses a voltage-controlled current source (VCCS, the `G1` element) to model the optical coupling, with an `Igain` parameter that sets the current transfer ratio. `Igain=1m` gives you roughly PC817A behavior (~100% CTR). Not a perfect model of the real nonlinear CTR curve, but good enough for what we need. + +### The Pin Order Trap + +Created two separate netlists: one for the RX path (bus to MCU) and one for the TX path (MCU to bus). Both failed on the first run. + +The problem? The PC817 SPICE subcircuit pin order. The subcircuit defines its pins as: + +``` +.subckt PC817 1 2 3 4 +* 1=Anode, 2=Cathode, 3=Collector, 4=Emitter +``` + +That's **not** the physical DIP pin order. On the actual IC, pin 3 is the collector and pin 4 is the emitter, but in the SPICE netlist you instantiate it with positional arguments: + +``` +XU1 anode_net cathode_net collector_net emitter_net PC817 Igain=1m +``` + +I had the collector and emitter swapped. The phototransistor was backwards, so the emitter-follower output was doing nothing useful. Swapped them, both sims ran clean. + +This is the kind of thing that eats an hour if you don't know to look for it. The SPICE pin order is whatever the subcircuit author decided, and it might or might not match the physical package. Always check the `.subckt` definition. + +### RX Path Results + +The RX path was the easy one. Bus HIGH (12V idle) drives 5.4mA through R1 (2k) into U1's LED. The phototransistor saturates, and in emitter-follower configuration (collector tied to VCC, emitter is the output), you get V(RX) pulled up toward VCC. + +The bus signal for testing was byte `0x50` (the MFL steering wheel address) at 9600 baud, 8E1. That's `01010000` in binary, but UART sends LSB first, so the actual bit pattern on the wire is `00001010` with even parity bit = 0. + +| Measurement | Value | Notes | +|-------------|-------|-------| +| V(RX) HIGH | 3.128V | Needs to be > 2.475V (ESP32 VIH) | +| V(RX) LOW | ~0V | Needs to be < 0.825V (ESP32 VIL) | +| LED current | 5.21mA | Through R1 (2k) at 12V bus | +| Rise time (10-90%) | 5.8us | 5.6% of 104.17us bit time | +| Signal polarity | Preserved | Bus HIGH = RX HIGH | + +Clean. 3.13V is 653mV above the ESP32 VIH threshold, and the rise time at 5.8us is well under the 104.17us bit period. The emitter-follower configuration preserves polarity (no signal inversion), which is convenient for the UART. + +### TX Path and the Asymmetry Problem + +The TX path is more interesting. It goes: ESP32 TX pin -> R5 (470) -> R3 (10k) -> Q1 base (BC547) -> Q1 drives U2 LED through R2 -> U2 phototransistor pulls bus LOW. + +With the original R2=470 at 5V VCC, the LED gets about 7.9mA. Plenty to saturate the phototransistor. But at 3.3V? Only 4.66mA. That's where things got dicey. + +But before we got to the R2 problem, the TX simulation revealed something interesting about the fundamental asymmetry of optocoupler-based bus interfaces: + +- **Pulling the bus LOW**: The phototransistor actively sinks current. Fast. About 5.8us fall time. +- **Releasing the bus HIGH**: The phototransistor just turns off. The bus goes HIGH only because of the external pull-up resistor. Slow. 19us rise time with a 1k pull-up. + +The pull-up has to charge whatever bus capacitance exists through whatever impedance the pull-up provides. That's fundamentally slower than an active drive. This is why BMW designed the TH3122 transceiver IC - it has active drive in both directions. The optocoupler approach works fine at 9600 baud (19us is still only 18% of the bit period), but you can see why it wouldn't scale to higher baud rates. + +### The R2 Investigation + +So the TX LED only gets 4.66mA at 3.3V with R2=470. Is that enough? Depends on the bus impedance. + +If the bus pull-up is 10k? Sure, the phototransistor barely needs to sink any current. If the pull-up is 1k or lower? Now you need real drive current, and 4.66mA through the LED might not generate enough collector current to pull the bus below the LOW threshold. + +Time for parameter sweeps. Three of them. + +**Sweep 1: R2 value (100-470 ohm) at worst-case CTR** + +Kept the bus at 1k pull-up (realistic worst case for a loaded BMW bus) and swept R2. The results told the story: + +| R2 (ohm) | LED Current (mA) | V(IBUS) LOW (V) | Status | +|-----------|------------------|------------------|--------| +| 100 | 15.0 | 0.17 | Solid | +| 150 | 11.9 | 0.19 | Solid | +| 220 | 9.7 | 0.27 | Solid | +| 330 | 7.0 | 0.24 | OK | +| 390 | 6.1 | 1.84 | Marginal | +| 470 | 4.7 | 2.51 | FAIL | + +The threshold is right around R2=330. At R2=390, the bus LOW voltage is already at 1.84V - getting uncomfortably close to logic-level ambiguity. At R2=470 (the original), V(IBUS) LOW is 2.51V. That's not LOW. That's "maybe LOW, maybe not, depends on who's asking." + +**Sweep 2: Bus pull-up impedance (510-10k ohm) at R2=470** + +Keeping the original R2=470 and sweeping the bus pull-up resistance: + +| R_PULL (ohm) | V(IBUS) LOW (V) | Status | +|--------------|------------------|--------| +| 510 | 4.52 | FAIL | +| 1k | 2.51 | FAIL | +| 2k | 0.42 | OK | +| 4.7k | 0.22 | Solid | +| 10k | 0.15 | Solid | + +At 510 ohm pull-up, V(IBUS) LOW is 4.52V. That's basically still HIGH. The phototransistor can't sink enough current to overcome the pull-up. Even at 1k it's still failing. You need the bus pull-up to be at least 2k for the original R2=470 to work at 3.3V. + +**Sweep 3: CTR grade (PC817A through D) at 1k bus** + +PC817 comes in four CTR grades. A is the cheapest (80-160%), D is the highest (200-400%). With R2=470 and 1k bus: + +| Grade | Igain | V(IBUS) LOW (V) | Status | +|-------|-------|------------------|--------| +| PC817A | 1m | 2.51 | FAIL | +| PC817B | 2m | 0.38 | OK | +| PC817C | 3m | 0.24 | Solid | +| PC817D | 4m | 0.19 | Solid | + +So if you use PC817B or better, the original R2=470 works even at 3.3V. But that's adding a component constraint that doesn't need to exist. Just change the resistor. + +### The Fix: R2=220 + +R2=220 doubles the LED current from 4.66mA to 9.71mA. That's still only 19% of the PC817's 50mA absolute maximum rating. Not even close to stressing the part. + +Created a single-point validation netlist (`reference/tx_validated_r2_220.cir`) with worst-case conditions: PC817A grade, 1k bus pull-up, 3.3V VCC. Results: + +| Measurement | Value | +|-------------|-------| +| V(IBUS) LOW | 0.266V | +| V(IBUS) swing | 8.08V (0.27V to 8.35V) | +| U2 LED current | 9.71mA | +| Q1 Vce(sat) | 0.073V | +| Bus rise time | 9.1us (8.7% of bit time) | + +0.266V. That's definitively LOW on any bus. And it works with the cheapest PC817A grade, which means you can grab whatever's in the parts bin. + +### K-line Compatibility Check (Spoiler: Nope) + +While we had the simulator warmed up, I also checked whether this optocoupler circuit would work for the Tucker project's OBD-II K-line interface. K-line has a 510 ohm pull-up to battery voltage - much stiffer than the BMW I/K-Bus. + +With R2=220 and 510 ohm pull-up, the phototransistor needs to sink about 23.5mA. A PC817A at 9.71mA LED current generates maybe 22.4mA of collector current. That's right at the edge - it might work, it might not, and "might work" is not a design spec for something plugged into a car. + +Even dropping R2 to 100 ohm (20.5mA LED current) only barely gets there. The conclusion: optocouplers are the wrong tool for K-line's low-impedance bus. The transistor design in the Tucker project is the right call for that use case. Different bus impedances, different hardware. Makes sense in retrospect. + +The relevant netlists are saved at `reference/opto_vs_kline_510.cir` and `reference/opto_vs_kline_ctr.cir` for reference. + +--- + +## Session 2: ESP32 Port + Architecture Design + +### Porting IbusSerial to ESP32 + +The muki01 IbusSerial library was written for AVR (Arduino Nano). It's a solid piece of work - a proper state machine protocol handler with ring buffers for async message handling, bus contention timing, and sleep management. But it's deeply tied to AVR hardware. + +The major changes: + +| AVR Thing | ESP32 Replacement | Why | +|-----------|-------------------|-----| +| Timer2 CTC mode (OCR2A=94, prescaler=256) | `esp_timer` periodic callback at 250us | AVR timer registers don't exist on ESP32 | +| TH3122 SEN/STA pin interrupt | GPIO interrupt on UART RX pin, CHANGE trigger | We don't have a TH3122 - reading the RX line directly | +| SoftwareSerial (pins 7/8) for debug | Hardware UART0 (USB) at 115200 | ESP32 has 3 hardware UARTs, no need for software serial | +| `PROGMEM` / `pgm_read_byte()` | Plain `const` arrays | ESP32 flash is memory-mapped, no special access needed | +| `digitalPinToInterrupt(3)` | `attachInterrupt(pin, isr, CHANGE)` on any GPIO | ESP32 supports interrupts on any GPIO | +| TH3122 EN pin for sleep mode | Not ported | No transceiver IC to put to sleep | +| Global volatile state | Class members with `volatile` where ISR-shared | Cleaner encapsulation | +| TX signal polarity (for TH3122) | `uart_set_line_inverse(UART_SIGNAL_TXD_INV)` | Optocoupler inverts TX, so we invert it back in hardware | + +The TX inversion thing is worth explaining. The optocoupler design inverts the TX signal: when the ESP32 TX pin is HIGH (idle), Q1 turns ON, U2 LED fires, the phototransistor conducts, and the bus gets pulled LOW. That's backwards from what the bus expects (bus idle should be HIGH). The muki01 AVR library works around this in software. On ESP32, we can just tell the UART peripheral to invert the TX output: `uart_set_line_inverse(UART_NUM, UART_SIGNAL_TXD_INV)`. Hardware fix, no software gymnastics needed. + +The bus idle detection was the trickiest part to port. The original uses AVR Timer2 in CTC (Clear Timer on Compare) mode. The timer fires an interrupt every ~1.5ms. If the SEN/STA pin hasn't triggered during that window, the bus is idle and it's safe to transmit. + +Without the TH3122 chip, we don't have a SEN/STA pin. Instead: GPIO interrupt on the RX pin (CHANGE trigger) to detect any bus activity, combined with an `esp_timer` periodic callback at 250us. Each time the timer fires, it checks if the time since the last RX transition exceeds 1500us. If yes, the bus is idle. Worst-case detection latency is idle_timeout + 250us = 1750us, which is well within the ~10ms inter-packet budget on a busy BMW bus. + +### The PlatformIO Setup + +Set up a PlatformIO project with three build environments: ESP32 (classic), ESP32-C3 (RISC-V), and ESP32-S3. Each has different pin assignments because the GPIO numbering varies across variants: + +- **ESP32**: GPIO 16/17 (UART2 defaults, free on most devkits) +- **ESP32-C3**: GPIO 4/5 (general purpose), LED on GPIO 8 +- **ESP32-S3**: GPIO 15/16 (free on S3-DevKitM), LED on GPIO 48 (RGB) + +All config is through `-D` build flags in `platformio.ini`, with `config.h` providing defaults. The main sketch (`main.cpp`) is a bus sniffer - it prints every I/K-Bus message it sees with module name lookup: + +``` +50 MFL -> 68 RAD [32 11] chk=1F +``` + +That's a volume-up command from the steering wheel to the radio. The module lookup table maps addresses like `0x50` to "MFL" (Multi-Function steering wheel) and `0x68` to "RAD" (Radio). There are 17 modules in the lookup table. Anything that doesn't match gets "???" which is probably fine for initial testing. + +All three environments build clean. Zero warnings. + +### Code Review: Concurrency Bugs + +Ran the code through review and it caught three real concurrency issues. These aren't hypothetical - they're the kind of bugs that work fine 99.9% of the time and then corrupt your data on a busy bus when you're trying to demo something. + +**Bug 1: 64-bit Torn Reads** + +The original timestamp `_lastRxTransitionUs` was `int64_t`. On a 32-bit ESP32, reading a 64-bit value isn't atomic. The GPIO ISR writes this variable on every RX edge. The timer callback reads it every 250us. If the timer reads between the two 32-bit halves of a write, you get garbage. The value could jump forward or backward by billions of microseconds. + +Fix: Changed to `uint32_t`. Atomic on 32-bit CPUs. Wraps every ~71 minutes, which is fine - we only care about differences of 1500us. + +**Bug 2: TX Loopback Race** + +When transmitting, the code sets `_isTransmitting = true` to prevent the idle detection from triggering on our own TX bytes (which loop back on a half-duplex bus). After the last byte goes out, the code was clearing `_isTransmitting` *before* resetting the idle timer. This creates a tiny window where: + +1. Last TX byte finishes +2. ISR fires from the TX loopback +3. `_isTransmitting` gets cleared +4. Timer fires, sees "idle" (because the idle timer wasn't reset yet) +5. Starts another TX while our loopback bytes are still on the bus + +Fix: Reset the idle timer FIRST, then clear `_isTransmitting`, with a compiler memory barrier (`__asm__ volatile("" ::: "memory")`) between them to prevent reordering. + +**Bug 3: Partial TX Writes** + +The `write()` method was adding bytes to the TX ring buffer one at a time. If the buffer was almost full, you could get the source byte and length byte in the buffer but not the rest of the message. Now you have a partial packet in the ring, and the TX state machine will try to send it, interpret whatever comes after as data, and put garbage on the bus. + +Fix: Check if the ENTIRE message fits in the ring buffer before writing any of it. All-or-nothing. Either the complete message goes in, or nothing does. + +### Multi-Protocol Architecture + +Then came a question that changed the project scope: can this library support both BMW I/K-Bus AND OBD-II K-line? + +The two protocols share the same physical layer - single-wire, half-duplex, open-collector/drain. But everything above that differs: baud rate (9600 vs 10400), framing (8E1 vs 8N1), checksum (XOR vs additive mod 256), bus access model (multi-master vs master/slave), and initialization (none vs 5-baud wake-up). + +The hardware layer is almost identical though. UART, GPIO interrupt, ring buffers, idle detection, TX inversion - all the same. So the design fell out naturally: + +``` + +-----------------+ + | KLineTransport | UART, GPIO ISR, ring buffers, idle detect + +-----------------+ + / \ + +------------+ +------------+ + | IbusHandler| | KLineObd2 | Protocol-specific FSMs + +------------+ +------------+ + | + +------------+ + | IbusEsp32 | Backward-compatible facade + +------------+ +``` + +**KLineTransport** owns all the hardware. Protocol-agnostic. Configurable baud, framing, checksum style. + +**IbusHandler** is the BMW I/K-Bus FSM extracted from IbusEsp32. It reads from the transport's RX ring, validates XOR checksums, applies source filtering, and delivers packets via callback. + +**KLineObd2** is new - OBD-II protocol handler with slow init, fast init, request/response. + +**IbusEsp32** stays as a thin facade wrapping Transport + Handler, so the existing BMW sniffer sketch doesn't need any changes. Drop-in compatible. + +--- + +## Session 3: Multi-Protocol Implementation + +This was the big build session. Three phases, done sequentially because each depended on the last. + +### Phase 1: Refactor (No New Features) + +The rule here was: extract KLineTransport and IbusHandler from the existing IbusEsp32 code, rewire everything, and verify that all three ESP32 environments still build with zero changes to `main.cpp`. If the sniffer sketch works identically before and after, the refactor is clean. + +Created `lib/AutoWire/` with the new file structure: + +``` +lib/AutoWire/ + KLineTransport.h / .cpp <-- extracted from IbusEsp32 + IbusHandler.h / .cpp <-- extracted from IbusEsp32 + IbusEsp32.h / .cpp <-- now just a facade + KLineObd2.h / .cpp <-- (Phase 2) + Obd2Pids.h <-- (Phase 2) + RingBuffer.h / .cpp <-- unchanged + E46Codes.h <-- unchanged + library.json +``` + +The extraction was surgical. ISR code, timer code, ring buffer management, UART init, TX inversion - all went into KLineTransport. The BMW protocol state machine (FIND_SOURCE -> FIND_LENGTH -> FIND_MESSAGE -> checksum validation), source filtering, and callback dispatch went into IbusHandler. IbusEsp32 got rewritten to literally just hold a Transport pointer and a Handler pointer. + +One wrinkle: the debug macros. The original code used `IBUS_DEBUG` to gate `Serial.printf()` calls. Since KLineTransport is shared between protocols, I renamed the macro to `KLINE_DEBUG` internally but kept `IBUS_DEBUG` as an alias in the build flags so existing platformio.ini configs don't break. Same with buffer size macros - `IBUS_RX_BUFFER_SIZE` still works, it just maps to the transport's buffer. + +All three environments built. `main.cpp` untouched. Phase 1 done. + +### Phase 2: OBD-II K-line Support + +Now the new stuff. KLineObd2 needed to handle two initialization methods (because different ECUs support different ones) and the request/response cycle. + +**5-Baud Slow Init (ISO 9141)** + +This is the weird one. To wake up an ECU, you send a target address at 5 baud. Five. Baud. That's 200ms per bit. A single byte takes 2 seconds to transmit. You can't use a UART for this - no UART peripheral supports 5 baud. You have to bit-bang it. + +The sequence: +1. Detach the UART TX from the GPIO pin (so we can drive it manually) +2. Drive TX LOW for 200ms (start bit) +3. For each data bit: drive TX HIGH or LOW for 200ms +4. Drive TX HIGH for 200ms (stop bit) +5. Re-attach the UART TX +6. Switch UART to 10400 baud, 8N1 +7. Wait for ECU to respond with sync byte (0x55) and two keyword bytes +8. Send the inverted keyword2 back within 25ms (the "handshake") + +The default target address is 0x33, which is the ISO 9141 "all ECUs" address. + +**Fast Init (ISO 14230)** + +Much more civilized. Pull the TX line LOW for 25ms, then HIGH for 25ms (the "TiniPulse"), then send a StartCommunication request at 10400 baud. The ECU responds with its keywords. Faster but not universally supported. + +**Request/Response** + +OBD-II is half-duplex on K-line, which means you hear your own transmissions echoed back. The echo has to be cleared before you can read the real response. Standard stuff, but it means every send has to be followed by a read-and-discard of exactly the bytes you just sent. + +**Obd2Pids.h** + +Created a header with ~20 PID decode helpers from SAE J1979. Things like: + +- `decodeRpm(a, b)` -> `(256 * a + b) / 4.0` +- `decodeCoolantTemp(a)` -> `a - 40` (in celsius) +- `decodeSpeed(a)` -> `a` (km/h, it's that simple) +- `decodeThrottlePos(a)` -> `a * 100.0 / 255.0` (percent) + +Plus the PID constants: `PID_RPM = 0x0C`, `PID_SPEED = 0x0D`, etc. + +**The Scanner Sketch** + +`obd2_scanner.cpp` is a simple loop that: +1. Tries slow init, falls back to fast init +2. Polls RPM, speed, coolant temp, throttle position, and battery voltage +3. Prints results to serial monitor +4. Sends TesterPresent every 4 seconds to keep the session alive + +Hit a namespace collision here. I had a variable named `obd2` and a `namespace obd2` for the PID helpers. C++ didn't appreciate that. Renamed the variable to `scanner`. Moving on. + +Added `build_src_filter` directives to `platformio.ini` so each environment only compiles its sketch: + +```ini +[env:esp32dev] +build_src_filter = + + +[env:obd2-scanner] +build_src_filter = + +``` + +All four environments (3 BMW sniffer + 1 OBD-II scanner) built clean. + +### Phase 3: Documentation + +Updated `library.json` metadata with the new library name (AutoWire) and file list. Updated CLAUDE.md with the architecture diagram, new file structure, OBD-II section, and the build instructions for the fourth environment. Not glamorous work but someone has to do it. + +--- + +## Session 4: Code Review + Hardening + +Ran the Apollo code review agent on the new OBD-II code. This is where it gets humbling. + +### Critical Findings (4) + +**C1: uart_set_pin() doesn't actually detach the pin** + +The slow init sequence needs to bit-bang the TX pin at 5 baud. The code was calling `uart_set_pin(UART_NUM, UART_PIN_NO_CHANGE, ...)` thinking that `UART_PIN_NO_CHANGE` would disconnect the UART from the TX GPIO. It doesn't. That constant means "don't change this pin assignment" - it's a no-op. The UART peripheral stays connected to the pin the entire time you're trying to bit-bang it. So `digitalWrite()` and the UART are fighting over the same GPIO. + +Fix: `pinMatrixOutDetach(txPin, false, false)` to actually disconnect the UART output matrix from the pin. Then `reattachUartTx()` reconnects it when bit-bang is done. + +This one's embarrassing because it's the kind of thing you'd only catch by reading the ESP-IDF source code or by scoping the TX pin during init and seeing the UART peripheral trampling your bit-bang output. + +**C2: Timeout arithmetic underflow** + +`readResponse()` had code like: + +```cpp +uint16_t remaining = timeoutMs - (millis() - startMs); +``` + +If `millis() - startMs` exceeds `timeoutMs`, that unsigned subtraction wraps around to ~65535. Instead of timing out, the function waits another 65 seconds. On a real car, that means your scanner hangs for a minute every time an ECU doesn't respond to a PID. + +Fix: Added a `remainingMs()` helper that does the subtraction with signed comparison and clamps to 0: + +```cpp +static uint16_t remainingMs(unsigned long startMs, uint16_t timeoutMs) { + unsigned long elapsed = millis() - startMs; + if (elapsed >= timeoutMs) return 0; + return timeoutMs - (uint16_t)elapsed; +} +``` + +**C3: readResponse() never validates checksum** + +The original code received the response, extracted the data bytes, and returned them. It never checked the checksum. On K-line, a corrupt byte is indistinguishable from a valid one unless you verify the additive mod-256 checksum at the end of each frame. Without this check, bus noise or partial collisions produce silently wrong data. + +Fix: Compute `checksumMod256` over all received bytes except the last, compare to the last byte. Reject the frame if they don't match. + +**C4: clearEcho() doesn't verify echo content** + +Half-duplex K-line means you hear your own TX echoed back on RX. The original `clearEcho()` just read N bytes and discarded them. It never compared them to what was actually sent. If another device was transmitting at the same time (bus contention), the echo bytes would be corrupted, and you'd never know. You'd then try to read a "response" that's actually a mix of your echo and the other device's data. + +Fix: Compare each received echo byte against the expected byte. If they don't match, return false and let the caller know the bus was busy. + +### Important Findings (7) + +**I1: RX buffers not flushed after init** + +During 5-baud slow init, the UART is configured at 10400/8N1 but the pin is being bit-banged at 5 baud. The UART peripheral sees the 200ms-wide bit-bang pulses and tries to interpret them as 10400 baud bytes. It generates framing errors and fills the RX buffer with garbage. After init completes and real communication starts, this stale garbage is sitting in the buffer ahead of the real data. + +Fix: `flushRx()` - clears both the UART hardware FIFO (via `uart_flush_input()`) and the ring buffer after init completes. + +**I2/I3/I5: Fixed-offset response parsing** + +The original code used hardcoded byte offsets to find the service ID in responses: `buf[3]` for one format, `buf[2]` for another. This works for the common case but ISO 14230 has multiple frame formats determined by the "format byte": + +- Bits 7:6 = address mode (00 = no address, 01 = CARB mode, 10 = with address, 11 = with address) +- Bits 5:0 = data length (0 = length in separate byte, 1-63 = length from format byte) + +Depending on the address mode, the actual data (service ID) could be at offset 1, 3, or 4. The hardcoded offsets would break on ECUs that use different addressing modes. + +Fix: Wrote `parseFrameData()` - a structural parser that reads the format byte, determines the address mode, finds the length (inline or separate byte), and returns the offset of the first data byte. Also sets `dataLen` so you know how many data bytes follow. + +**I4: Response buffer too small** + +Buffer was 16 bytes. Some OBD-II responses (especially multi-PID responses and VIN queries) can be longer. Bumped to 32 bytes. + +**I6: No TesterPresent keepalive, no session recovery** + +K-line diagnostic sessions have a timeout (P3max, typically 5 seconds). If the tester doesn't send anything within that window, the ECU drops the session and you have to re-initialize. The original code had no keepalive mechanism and no way to detect or recover from a lost session. + +Fix: `P3_KEEPALIVE_MS = 4000` - automatically sends TesterPresent (service 0x3E) when the idle time approaches P3max. Also added `consecutiveFailures` tracking in the scanner sketch - after 5 consecutive failures, it calls `reset()` and re-runs the init sequence (slow init first, then fast init if slow fails). + +**I7: Missing null check after new** + +`new KLineObd2()` could return nullptr on memory exhaustion. The scanner sketch didn't check. On an ESP32 with limited heap, this isn't purely theoretical. Added the check. + +### After All Fixes + +All four PlatformIO environments still build clean. The commit message was straightforward: "Harden OBD-II K-line handler (code review findings)". + +--- + +## Lessons Learned + +Looking back across all four sessions, a few things stand out: + +**Simulate before you solder.** The R2=220 fix came entirely from SPICE. If we'd built the prototype first with R2=470, we'd have had an intermittently-working TX path and spent hours with a scope trying to figure out why. The sweep netlists (`tx_sweep_r2.cir`, `tx_sweep_rpull.cir`, `tx_sweep_ctr.cir`) documented the entire operating envelope without touching a soldering iron. + +**Pin order is not pin numbering.** SPICE subcircuit pin order is defined by the `.subckt` line. Physical IC pin numbering is defined by the package. They are not the same thing. Check the subcircuit definition. Every time. + +**Voltage matters more than you think.** The jump from 5V to 3.3V cut the TX LED current almost in half (7.9mA to 4.66mA). That took the design from "works great" to "fails at normal bus loading." A simple resistor change fixed it, but only because we caught it in simulation. On real hardware, it would've manifested as "sometimes my TX messages don't work" which is one of the worst failure modes to debug. + +**Code review catches what testing misses.** The `uart_set_pin()` no-op bug (C1) would've been invisible in a loopback test. The code would appear to work because the UART TX output and the bit-bang output happen to agree during the stop bit (both HIGH). It would only fail intermittently during the data bits when the UART tries to send its idle pattern while you're pulling the line LOW for bit-bang. Good luck finding that with `Serial.println()` debugging. + +**Half-duplex is harder than it looks.** Echo clearing, bus contention detection, checksum validation, session keepalive, timeout handling - each one is straightforward in isolation, but the combination creates a lot of edge cases. The original code handled the happy path. The hardening pass handled everything else. + +--- + +## Current State + +As of the last commit, the project has: + +- Two validated SPICE netlists (RX and TX paths) plus four parameter sweep netlists +- A multi-protocol PlatformIO library (AutoWire) that builds for ESP32, ESP32-C3, and ESP32-S3 +- BMW I/K-Bus sniffer sketch (tested in simulation, not yet on hardware) +- OBD-II K-line scanner sketch (code-reviewed, not yet on hardware) +- 13 source files in the AutoWire library +- 4 PlatformIO build environments, all clean + +What hasn't been done yet: breadboard prototype, loopback testing, bench testing with a 12V supply, and actual vehicle testing. That's the next chapter. + +The hardware is simple enough - two PC817s, a BC547, five resistors, and a diode. The circuit has been validated in simulation down to specific voltages and currents at worst-case conditions. The firmware has been through two rounds of code review. At this point, the biggest unknown is what happens when you plug it into a real BMW and start listening to the K-Bus chatter from a dozen modules talking over each other. + +Should be fun. diff --git a/firmware/.gitignore b/firmware/.gitignore new file mode 100644 index 0000000..968a41b --- /dev/null +++ b/firmware/.gitignore @@ -0,0 +1,2 @@ +.pio/ +.vscode/