ArdaandClaude Opus 4.6
42c93bdea7
Deep architectural traces spanning kit management, schema registry, file save/open pipelines, live device connection, DASP transport, SOX protocol commands, and invoke/action slot flow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1438 lines
97 KiB
Markdown
1438 lines
97 KiB
Markdown
# Sedona Editor - Architecture Traces
|
||
|
||
Deep architectural traces of the Sedona Editor codebase, each following a specific feature or subsystem from UI entry point through every layer to its final effect. These traces were produced through systematic analysis of the code graph and source files.
|
||
|
||
---
|
||
|
||
## Traces 1-6 (Prior Session)
|
||
|
||
The following traces were produced in an earlier session:
|
||
|
||
- **Trace 1 -- SoxClient Bridge Analysis** (from prior session)
|
||
- **Trace 2 -- Live Property Edit Flow** (from prior session)
|
||
- **Trace 3 -- COV Event Flow** (from prior session)
|
||
- **Trace 4 -- Live Add/Delete Component Flow** (from prior session)
|
||
- **Trace 5 -- Wire Routing Pathfinding** (from prior session)
|
||
- **Trace 6 -- Undo Architecture** (from prior session)
|
||
|
||
---
|
||
|
||
## Trace 7 — Kit Management Flow
|
||
|
||
The kit management system spans **5 communities** in the graph: the KitsPanel UI (`manage_kits.py`), the schema DOM layer (`sax_file.py`), the palette resolution layer (`palette.py`), the manifest/dependency layer (`schema.py`), and the undo system (`undo.py`). Its job is to keep the `<schema>` block of a `.sax` file in sync with the kits its components actually need — a constraint that matters because a Sedona device won't run an app whose schema doesn't declare every kit in its dependency closure.
|
||
|
||
### Layer 1 — The Palette: What's Installed
|
||
|
||
Everything starts at `palette.py`, which reads the local Sedona home directory. It has two filesystem sources:
|
||
|
||
- **`manifests/<kit>/<kit>-<checksum>.xml`** — the XML manifest describing a kit's types and their slots
|
||
- **`kits/<kit>/<kit>-<checksum>-<version>.kit`** — the binary kit file whose filename carries the version number
|
||
|
||
`resolve_manifest_path()` ([palette.py:76](palette.py:76)) picks which manifest describes a kit. It follows a three-tier preference: (1) the checksum the open file's `<schema>` pins, (2) the highest version number from .kit filenames, (3) the last manifest alphabetically. This is how the palette always shows types from the build the app is actually using, not the newest one.
|
||
|
||
`kit_builds()` ([palette.py:120](palette.py:120)) enumerates every build of a kit (every manifest under `manifests/<kit>/`), cross-referencing the `.kit` filenames for version labels, and sorts newest-first. `resolve_checksum()` ([palette.py:178](palette.py:178)) returns the checksum of whichever manifest `resolve_manifest_path` chose.
|
||
|
||
The caches (`_versions_cache`, `_manifest_cache`, `_builds_cache`) are module-level dicts. `invalidate()` clears all of them, forcing the next access to re-read the filesystem — called after any schema change so the palette reflects the new pins.
|
||
|
||
### Layer 2 — Dependency Resolution: What's Required
|
||
|
||
`schema.py` handles manifest parsing and dependency tracking. `read_kit_dependencies()` ([schema.py:47](schema.py:47)) parses a manifest's `<depends>` block and returns the kit names it was built against (e.g., `<depend on="sys 1+"/>` yields `"sys"`). Results are cached by manifest path.
|
||
|
||
`required_kit_names()` ([sax_file.py:421](sax_file.py:421)) performs a **BFS over the dependency graph**: it starts with `used_kit_names()` (every kit named by a `<comp>` element's `type` attribute), then iteratively expands through `read_kit_dependencies()` until no new names appear. This transitive closure is what the schema block must declare — not just the kits components name, but every kit those kits depend on.
|
||
|
||
### Layer 3 — The Deferred Declaration Pattern
|
||
|
||
When the user drags a component from the palette onto the wiresheet, `note_kit_used()` ([sax_file.py:388](sax_file.py:388)) records the kit **without touching the DOM**:
|
||
|
||
```
|
||
app_state.pending_schema_kits[kit_name] = palette.resolve_checksum(kit_name)
|
||
```
|
||
|
||
This is a deliberate design choice. The schema block isn't updated on every placement because: (a) the user might undo the placement, (b) multiple components from the same kit might be added in quick succession, and (c) interrupting the user with a kit dialog on every drag-and-drop would be hostile.
|
||
|
||
The pending list only supplies checksums. The DOM is the authority on what kits are actually needed — `used_kit_names()` ([sax_file.py:408](sax_file.py:408)) walks every `<comp>` in the tree, so an undone placement can't leave a phantom kit behind.
|
||
|
||
### Layer 4 — Save-Time Reconciliation
|
||
|
||
When the user saves, `reconcile_schema_kits_before_save()` ([sax_file.py:728](sax_file.py:728)) runs as a gate before the file is written:
|
||
|
||
1. **Snapshot** the `<schema>` block's children (for rollback)
|
||
2. Call `declare_used_kits()` ([sax_file.py:444](sax_file.py:444)), which computes the full transitive closure via `required_kit_names()`, diffs it against what the schema already declares, and inserts `<kit>` elements for anything missing — using the checksum from `pending_schema_kits` or falling back to `palette.resolve_checksum()`
|
||
3. If nothing was added, return `True` (save proceeds)
|
||
4. If kits were added, build a report via `describe_schema_kits()` ([sax_file.py:554](sax_file.py:554)) showing what was added and the full post-save kit list, then show a confirmation dialog
|
||
5. **If the user cancels**: `restore_children()` puts the schema block back, `active_schema_kits` entries are rolled back, and the save is abandoned — `return False`
|
||
6. **If the user confirms**: clear the pending entries and `return True` — the save writes the file with the new kits
|
||
|
||
This is the **all-or-nothing** pattern seen throughout the codebase: snapshot, mutate, validate, rollback-on-reject.
|
||
|
||
### Layer 5 — The Manage Kits Dialog
|
||
|
||
`KitsPanel` ([manage_kits.py:223](manage_kits.py:223)) is the explicit management UI, opened from the menu. It presents a Treeview table with four groups of rows, built by `collect_rows()` ([manage_kits.py:131](manage_kits.py:131)):
|
||
|
||
1. **Installed kits** — declared and required (checkbox greyed, can't uncheck)
|
||
2. **Installed but not used** — declared but no component needs them (can uncheck to remove)
|
||
3. **Available on computer** — in Sedona home but not in schema (can check to add)
|
||
4. **Not supported** — incompatible kits (shown with a cross, not actionable)
|
||
|
||
Each row carries an Action dropdown built by `action_options()` ([manage_kits.py:85](manage_kits.py:85)). The first option is always a no-op ("Keep at…"), and subsequent options are labeled "Upgrade to", "Downgrade to", "Add at", or "Set to" depending on version comparison.
|
||
|
||
The panel has two commit paths:
|
||
|
||
**"Upgrade all"** ([manage_kits.py:509](manage_kits.py:509)): scans every declared kit, finds those where `latest_checksum != current_checksum`, shows a preview, and on confirmation calls `_apply_changes()` with the full selection list.
|
||
|
||
**"Update kits"** ([manage_kits.py:538](manage_kits.py:538)): reads the table state — unchecked declared kits become removals, checked kits with non-default action choices become selections — and passes both to `_apply_changes()`.
|
||
|
||
Both paths funnel through `_apply_changes()` ([manage_kits.py:478](manage_kits.py:478)), which calls `sax_file.apply_kit_selections()`.
|
||
|
||
### Layer 6 — Atomic Apply with Rollback
|
||
|
||
`apply_kit_selections()` ([sax_file.py:623](sax_file.py:623)) is the transactional core. It:
|
||
|
||
1. **Snapshots** the schema block's children
|
||
2. Walks selections: inserts missing `<kit>` elements, updates checksums on existing ones
|
||
3. Walks removals: removes `<kit>` elements
|
||
4. If nothing actually changed, returns `([], None)`
|
||
5. **Invalidates** the palette cache (so `resolve_manifest_path` re-reads with new pins)
|
||
6. **Recompiles** the schema registry via `schema.load_schema_kits_and_manifests()`
|
||
7. **Validates** — if a manifest is missing (`MissingManifestError`), calls `abandon()` which restores the snapshot and reloads the registry: returns `([], reason)`
|
||
8. **Validates** — if any component types became unresolvable (a repin dropped a type the app uses), same rollback
|
||
9. Clears pending entries for the changed kits
|
||
10. **Pushes** a `SchemaBlockCommand` onto the undo stack with the before/after snapshots
|
||
11. Marks dirty and refreshes the treeview and wiresheet
|
||
|
||
The `abandon()` closure ([sax_file.py:690](sax_file.py:690)) is elegant: `restore_children(schema_elem, before)` + `undo.reload_schema_registry()`, then return the error message. The caller gets `([], reason)` and can show the error without needing to know anything about the rollback internals.
|
||
|
||
### Layer 7 — Undo Integration
|
||
|
||
`SchemaBlockCommand` (from `undo.py`) stores before/after snapshots of the `<schema>` element's children. Its `undo()` restores the before-snapshot and recompiles the registry; its `redo()` restores the after-snapshot and recompiles. This is the same snapshot/restore mechanism that `apply_kit_selections` uses for its own rollback — the undo system is literally replaying the same operation the transactional validation used.
|
||
|
||
### The Full Loop
|
||
|
||
```
|
||
User drags component → note_kit_used() → pending_schema_kits grows
|
||
(no DOM change)
|
||
User saves → reconcile_…_before_save()
|
||
→ declare_used_kits()
|
||
→ required_kit_names() (BFS transitive closure)
|
||
→ insert_kit_element() (DOM mutation)
|
||
→ describe_schema_kits() (report)
|
||
→ user confirms or cancels (rollback if cancel)
|
||
|
||
User opens Manage Kits → collect_rows() → reads DOM + palette + dependencies
|
||
→ _upgrade_all() or _update_marked()
|
||
→ _apply_changes()
|
||
→ apply_kit_selections()
|
||
→ snapshot → mutate → invalidate palette
|
||
→ recompile registry → validate
|
||
→ rollback on error, or push undo + mark dirty
|
||
```
|
||
|
||
The key insight: there are **two reconciliation moments** — passive (save-time, for kits that crept in through component placement) and active (the Manage Kits dialog, for explicit version management) — but both converge on the same atomic apply-or-rollback mechanism.
|
||
|
||
---
|
||
|
||
## Trace 8 — Schema Registry Compilation
|
||
|
||
The schema registry is the central type system of the editor. It maps every `kit::Type` string to a flat list of resolved slots — the properties, inputs, and outputs a component of that type exposes. Every box on the wiresheet, every row in the properties panel, every wire endpoint, and every live device interaction depends on this registry being correct and current. It lives in `app_state.final_components_registry`, a plain dict populated by `schema.py` and read everywhere else.
|
||
|
||
### Entry Points — When Does Compilation Happen?
|
||
|
||
There are **three distinct entry points**, each reaching the registry through a different path:
|
||
|
||
**1. File open** ([sax_file.py:175](sax_file.py:175)): `schema.load_schema_kits_and_manifests()` is called right after `ET.parse()` loads the XML. This is the primary path — it reads the `<schema>` block's `<kit>` entries, resolves each to a manifest file, and compiles the full registry. A `MissingManifestError` here aborts the file open entirely.
|
||
|
||
**2. Live device connect** ([main.py:2077](main.py:2077)): `_load_live_schema()` takes a different route. Instead of reading `<kit>` elements from a DOM, it reads `ds.sox_client.schema_parts` — the kit names and checksums the device reported over SOX. It populates `active_schema_kits` from those, finds manifest paths itself, and calls `compile_schema_registry()` directly (not through `load_schema_kits_and_manifests`). Missing kits here are **tolerated** — a warning in the status bar, not an error — because the device may run kits the local machine doesn't have manifests for.
|
||
|
||
**3. Kit changes** (multiple callers): `apply_kit_selections()` ([sax_file.py:699](sax_file.py:699)), `add_component()` ([sax_file.py:792](sax_file.py:792), [sax_file.py:803](sax_file.py:803)), and `SchemaBlockCommand.undo()/redo()` ([undo.py:112](undo.py:112)) all recompile after mutating the `<schema>` block. Each calls through `load_schema_kits_and_manifests()` after invalidating the palette cache.
|
||
|
||
### Phase 1 — Manifest Discovery
|
||
|
||
`load_schema_kits_and_manifests()` ([schema.py:239](schema.py:239)) starts by clearing `active_schema_kits` and rebuilding it from the DOM:
|
||
|
||
```python
|
||
for kit in schema_elem.findall("kit"):
|
||
app_state.active_schema_kits[name] = kit.get("checksum")
|
||
```
|
||
|
||
Then it merges in pending kits — those recorded by `note_kit_used()` when components were placed but the schema hasn't been saved yet:
|
||
|
||
```python
|
||
kits_to_load = dict(app_state.active_schema_kits)
|
||
for name, checksum in app_state.pending_schema_kits.items():
|
||
kits_to_load.setdefault(name, checksum)
|
||
```
|
||
|
||
The `setdefault` is important: a declared kit's checksum always wins over its pending checksum. Pending kits only fill gaps.
|
||
|
||
For each kit, `find_manifest_path()` ([schema.py:202](schema.py:202)) locates the XML manifest under `SEDONA_HOME/manifests/<kit>/<kit>-<checksum>.xml`. If the declared checksum matches a file, that file is used. If not — because the checksum is absent or points to a build this machine doesn't hold — it falls back to the last manifest alphabetically. This tolerance is what lets files with no checksums (or checksums from another machine) still open.
|
||
|
||
Missing manifests are **collected, not failed on immediately**. The loop finishes, then raises a single `MissingManifestError` naming every missing kit. This matters: fixing one-at-a-time is painful when five kits are missing.
|
||
|
||
### Phase 2 — Type Extraction
|
||
|
||
`compile_schema_registry()` ([schema.py:167](schema.py:167)) receives the resolved manifest paths and builds the type index:
|
||
|
||
```python
|
||
kit_schemas = {}
|
||
for path in manifest_paths:
|
||
root = ET.parse(path).getroot()
|
||
kit_name = root.get("name")
|
||
for type_tag in root.findall("type"):
|
||
type_name = f"{kit_name}::{type_tag.get('name')}"
|
||
kit_schemas[type_name] = type_tag
|
||
```
|
||
|
||
This produces a flat namespace of `"kit::Type"` → XML element mappings. Every type from every loaded manifest goes in, regardless of whether any component in the app uses it. The index is cheap (it's just element references) and having the full set is essential for inheritance resolution — a type in kit A may inherit from a type in kit B.
|
||
|
||
### Phase 3 — Slot Inheritance Resolution
|
||
|
||
The heart of the system. `get_slots_with_inheritance()` ([schema.py:136](schema.py:136)) resolves each type's complete slot list by walking the `base` chain:
|
||
|
||
```python
|
||
def get_slots_with_inheritance(type_name, kit_schemas):
|
||
type_element = kit_schemas[type_name]
|
||
base_type = type_element.get("base")
|
||
|
||
resolved_slots = []
|
||
if base_type:
|
||
resolved_slots = get_slots_with_inheritance(base_type, kit_schemas)
|
||
|
||
current_slots = [... for slot in type_element.findall("slot")]
|
||
current_slots.sort(key=lambda s: s["declared_id"])
|
||
resolved_slots.extend(current_slots)
|
||
return resolved_slots
|
||
```
|
||
|
||
This is classic recursive inheritance: walk up to the root of the type hierarchy, collect that ancestor's slots, then append each descendant's slots on the way back down. The `declared_id` sort within each level preserves the manifest's own ordering. The final list is the **complete set of slots** a component of that type has, from the most ancestral (e.g., `sys::Component`'s meta slot) through every intermediate type down to the concrete one.
|
||
|
||
If the base type references a kit that isn't loaded — say `control::Ramp` inherits from `sys::Component` but `sys` wasn't in the schema — a `MissingManifestError` is raised with the `unresolved_type` field set. The error message then explains not just "kit X is missing" but "type Y inherits from kit X, so its slots can't be resolved."
|
||
|
||
Each slot is extracted with its full metadata:
|
||
- `name`, `type`, `default`, `flags` — from the `<slot>` attributes
|
||
- `declared_id` — the manifest's own slot numbering (used for wire endpoint alignment)
|
||
- `readonly` — from the `@readonly` facet
|
||
- `summary` — from the `@summary` facet (controls wiresheet visibility)
|
||
- `facets` — the complete facet dict (`read_all_facets()` at [schema.py:90](schema.py:90)) carrying unit, precision, min, max, trueText/falseText, range
|
||
|
||
### Phase 4 — Flattening and ID Assignment
|
||
|
||
Back in `compile_schema_registry()`, the inherited slot list gets re-indexed:
|
||
|
||
```python
|
||
for sequence_id, slot_data in enumerate(flat_slots):
|
||
resolved_type_structure.append({
|
||
"slot_id": sequence_id,
|
||
...
|
||
})
|
||
app_state.final_components_registry[type_name] = resolved_type_structure
|
||
```
|
||
|
||
The `slot_id` is a **zero-based sequence number** across the full inheritance chain, not the manifest's `declared_id`. This is the ID used everywhere downstream: it's the SOX slot index for live property reads/writes, the row number for wire endpoint anchoring, and the key for property panel rendering. The registry stores both — `declared_id` for display formatting (the "00. meta", "01. status" labels on wiresheet boxes), `slot_id` for protocol-level addressing.
|
||
|
||
### The Three Consumers
|
||
|
||
The registry serves three different views, each calling a different accessor:
|
||
|
||
**Wiresheet boxes** call `get_slots_for_type()` ([schema.py:303](schema.py:303)) at [wiresheet.py:166](wiresheet.py:166). This is the most filtered view: it hides operator-flagged slots (`"o"` in flags), config-only slots (`"c"` in flags) unless they have a wire, and slots with `@summary false`. The result is the rows painted inside each box.
|
||
|
||
**Properties panel** calls `get_full_slots_for_type()` ([schema.py:292](schema.py:292)) at [properties.py:512](properties.py:512). This shows everything except nameless slots — config slots need to be editable somewhere, and this is that somewhere. It returns copies (`dict(slot)`) so the panel can annotate them without corrupting the registry.
|
||
|
||
**Wire creation** calls `get_all_slots_for_type()` ([schema.py:281](schema.py:281)) at [main.py:1133](main.py:1133). This returns `(name, type, flags, readonly)` tuples for both sides of a potential link, used to match compatible slot types.
|
||
|
||
### Recompilation Safety
|
||
|
||
Every recompilation path follows the same discipline:
|
||
|
||
1. **Invalidate palette** (`palette.invalidate()`) — so `resolve_manifest_path()` picks up the new checksum pins
|
||
2. **Call `load_schema_kits_and_manifests()`** — which clears `active_schema_kits`, re-reads the DOM, and calls `compile_schema_registry()`
|
||
3. **Handle failure** — the caller decides the policy:
|
||
- File open: abort
|
||
- `apply_kit_selections()`: rollback the DOM and return the error
|
||
- Undo/redo (`reload_schema_registry()` at [undo.py:84](undo.py:84)): **swallow the error** — the state was valid once, so a failure means Sedona home changed underneath the session, and the next user action will report it
|
||
|
||
The `compile_schema_registry()` call itself clears `final_components_registry` before populating it ([schema.py:183](schema.py:183)), so there's never stale data mixed with fresh data — it's always a complete rebuild, not a merge.
|
||
|
||
### The Full Data Flow
|
||
|
||
```
|
||
<schema> block pending_schema_kits
|
||
↓ ↓
|
||
load_schema_kits_and_manifests()
|
||
↓ merge
|
||
kits_to_load (name → checksum)
|
||
↓ find_manifest_path() per kit
|
||
manifest_paths[]
|
||
↓
|
||
compile_schema_registry()
|
||
↓ ET.parse() each manifest
|
||
kit_schemas {"kit::Type" → <type> element}
|
||
↓ get_slots_with_inheritance() per type
|
||
↓ (recursive base-chain walk)
|
||
flat_slots[] [inherited..., own...]
|
||
↓ re-index with sequence slot_id
|
||
final_components_registry {"kit::Type" → [{slot_id, name, type, ...}]}
|
||
↓ ↓ ↓
|
||
get_slots_for_type() get_full_slots_for_type() get_all_slots_for_type()
|
||
↓ ↓ ↓
|
||
wiresheet boxes properties panel wire creation
|
||
```
|
||
|
||
The key architectural insight: the registry is a **compiled cache**, not a persistent store. It's always derivable from the schema block + pending list + Sedona home manifests, and it's rebuilt from scratch on every change. This makes it safe to clear and recompile freely — there's no migration, no partial update, no versioning. The cost is parsing a few dozen XML files, which is negligible compared to the UI work that follows.
|
||
|
||
---
|
||
|
||
## Trace 9 — File Save Pipeline
|
||
|
||
The save pipeline crosses **4 communities**: the main controller (`main.py`), the SAX file layer (`sax_file.py`), the kit reconciliation system (`sax_file.py` + `schema.py`), and the undo/dirty tracking layer (`undo.py` + `app_state.py`). There are three save entry points that converge on the same core sequence, plus a live-mode variant that bypasses most of it.
|
||
|
||
### Entry Points
|
||
|
||
**Ctrl+S** binds to `do_save()` ([main.py:430](main.py:430)), which forks immediately:
|
||
|
||
```python
|
||
def do_save(event=None):
|
||
if app_state.live_mode:
|
||
do_download_to_sax()
|
||
return "break"
|
||
sax_file.execute_direct_overwrite_save()
|
||
return "break"
|
||
```
|
||
|
||
In offline mode, it calls `execute_direct_overwrite_save()`. In live mode, it calls `do_download_to_sax()` — a simpler path that prompts for a filename (since there's no existing file) and writes without kit reconciliation.
|
||
|
||
**Save As** is wired directly from the menu to `sax_file.save_file_as()` ([sax_file.py:285](sax_file.py:285)). It optionally stamps the filename with a timestamp (controlled by `app_state.ADD_TIMESTAMP`), then runs the same core pipeline as the direct save.
|
||
|
||
**Live download** (`do_download_to_sax()` at [main.py:438](main.py:438)) writes the current device-sourced DOM to a user-chosen path via `write_sax_to_path()` — no kit reconciliation, no undo tracking, no dirty state. It's a snapshot export, not a document save.
|
||
|
||
### The Core Pipeline
|
||
|
||
Both `execute_direct_overwrite_save()` and `save_file_as()` follow the same 5-phase sequence:
|
||
|
||
#### Phase 1 — Metadata Sync
|
||
|
||
`sync_live_metadata_to_dom()` ([sax_file.py:202](sax_file.py:202)) reconciles the canvas state back into the XML DOM. This is necessary because component positions (the packed `meta` property) may have been updated only in the in-memory element references held by `canvas_comp_map`, not yet written back to the DOM tree that the file parser originally produced.
|
||
|
||
It does two things in one recursive walk (`sync_and_prune_tree`):
|
||
|
||
1. **Prunes dead components**: for every `<comp>` in the DOM, it checks whether the component is still "alive" in `active_components_list`. If a component was rendered on the canvas at some point (`is_previously_rendered`) but is no longer alive, it's removed from the DOM. This catches components that were deleted through undo/redo cycles where the DOM element was restored but the canvas state wasn't — a safety net, not the primary deletion mechanism.
|
||
|
||
2. **Syncs meta values**: for components that are alive, it copies the `meta` property value from the canvas-tracked element into the DOM element. The meta property packs the component's grid position (x, y) and group flags into a single 32-bit integer ([sax_dom.py:57](sax_dom.py:57)), so a component that was dragged on the wiresheet gets its new position written back here.
|
||
|
||
This sync is called at the **start** of every save path, including `write_sax_to_path()` for live downloads. It's the bridge between the "live editing state" (canvas positions, active component list) and the "document state" (the XML tree).
|
||
|
||
#### Phase 2 — Kit Reconciliation Gate
|
||
|
||
`reconcile_schema_kits_before_save()` ([sax_file.py:728](sax_file.py:728)) checks whether any components were placed from kits the schema doesn't declare. This was traced in detail in Trace 7, but its role in the save pipeline is as a **gate**: it returns `True` to proceed or `False` to abort.
|
||
|
||
```python
|
||
if not reconcile_schema_kits_before_save():
|
||
return # user cancelled — save is abandoned
|
||
```
|
||
|
||
If kits need adding, the user sees a confirmation dialog listing the new kits and the full post-save schema. If they cancel, the schema block is rolled back to its pre-save state and the save is abandoned — no file is written, no dirty state changes. This is the only point in the save pipeline where the user can back out.
|
||
|
||
#### Phase 3 — XML Serialization
|
||
|
||
The actual write is two lines:
|
||
|
||
```python
|
||
with open(path, "wb") as f:
|
||
f.write(b'<?xml version="1.0" encoding="UTF-8"?>\n')
|
||
ET.ElementTree(app_state.xml_root_element).write(
|
||
f, encoding="utf-8", xml_declaration=False
|
||
)
|
||
```
|
||
|
||
The XML declaration is written manually because Python's `ElementTree.write()` with `xml_declaration=True` produces `<?xml version='1.0' encoding='utf-8'?>` (single quotes, lowercase encoding) — a harmless difference to a parser but a noisy diff to a human comparing files. The manual line uses double quotes and uppercase `UTF-8`, matching the Sedona convention.
|
||
|
||
The file is opened in binary mode (`"wb"`) and `encoding="utf-8"` is passed to `ET.write()`, which handles the encoding internally. The `xml_declaration=False` suppresses the auto-generated one since the manual one already covers it.
|
||
|
||
There is **no temporary file or atomic rename**. The write goes directly to the target path. If the process crashes mid-write, the file is truncated. This is a conscious simplicity trade-off — `.sax` files are small (typically a few KB), the write is fast, and the editor's target environment (industrial automation workstations) values predictable paths over crash resilience.
|
||
|
||
#### Phase 4 — State Cleanup
|
||
|
||
After a successful write:
|
||
|
||
```python
|
||
app_state.mark_clean()
|
||
undo.mark_saved()
|
||
```
|
||
|
||
`mark_clean()` ([app_state.py:201](app_state.py:201)) clears `is_dirty`, tints the canvas background back to `CLEAN_TINT` (from the yellow-ish `DIRTY_TINT` that signals unsaved changes), and calls `refresh_dirty_display()` which updates the window title (removing any unsaved marker).
|
||
|
||
`mark_saved()` ([undo.py:70](undo.py:70)) pins the current undo stack depth:
|
||
|
||
```python
|
||
def mark_saved():
|
||
global _saved_depth
|
||
_saved_depth = len(_undo_stack)
|
||
```
|
||
|
||
This is how undo knows whether the document matches what's on disk. Every `_refresh()` call after an undo or redo compares `len(_undo_stack)` against `_saved_depth` — when they diverge, the document is dirty; when they match again (after enough undos to reach the saved state), the document transitions back to clean. This depth tracking is what makes "undo back to the saved state" automatically clear the dirty indicator without re-reading the file.
|
||
|
||
#### Phase 5 — UI Refresh
|
||
|
||
```python
|
||
refresh_treeview_from_dom(app_state.xml_root_element)
|
||
```
|
||
|
||
`refresh_treeview_from_dom()` ([sax_file.py:53](sax_file.py:53)) clears the navigation tree and rebuilds it from the DOM. This picks up any structural changes the kit reconciliation made (new `<kit>` entries in the schema). It also ensures the tree reflects the pruning that `sync_live_metadata_to_dom()` did — if dead components were removed from the DOM, they disappear from the tree here.
|
||
|
||
Finally, both save paths show a success dialog and post a status bar message.
|
||
|
||
### The Three Paths Compared
|
||
|
||
| Step | `execute_direct_overwrite_save` | `save_file_as` | `do_download_to_sax` |
|
||
|---|---|---|---|
|
||
| Metadata sync | yes | yes | yes (via `write_sax_to_path`) |
|
||
| Kit reconciliation | yes (gate) | yes (gate) | **no** |
|
||
| Path selection | uses `current_file_path` | prompts with optional timestamp | always prompts |
|
||
| XML write | to existing path | to chosen path | to chosen path |
|
||
| mark_clean | yes | yes | **no** (not a "document" save) |
|
||
| mark_saved (undo) | yes | yes | **no** |
|
||
| Tree refresh | yes | yes | **no** |
|
||
| Success dialog | yes | yes | status bar only |
|
||
|
||
### The Live Mode Variant
|
||
|
||
`write_sax_to_path()` ([sax_file.py:273](sax_file.py:273)) is the minimal path: sync metadata, write XML, done. No kit reconciliation because the device's schema is the authority in live mode — there's nothing to reconcile. No dirty/undo tracking because live mode doesn't use the undo system (hardware writes can't be reversed). It's a pure export: "write what the device looks like right now."
|
||
|
||
### Failure Modes
|
||
|
||
The pipeline has exactly **two abort points**:
|
||
|
||
1. **Kit reconciliation cancelled** — the user sees what kits would be added and says no. The schema block is rolled back, nothing is written, no state changes. The document remains dirty.
|
||
|
||
2. **Write exception** — the `try/except` around the whole pipeline catches any I/O error (permissions, full disk, locked file) and shows it in an error dialog. Because the dirty/clean state hasn't been changed yet (those lines come after the write), the document correctly remains marked as unsaved.
|
||
|
||
There is no abort for "the DOM is inconsistent" or "the metadata sync failed" — those operations manipulate in-memory data structures and are trusted to succeed. If they corrupt the DOM (a bug, not a runtime error), the corrupted XML is written to disk. The next open would either fail to parse or load with wrong positions, but the editor doesn't validate the DOM against any schema before writing.
|
||
|
||
### The Full Sequence
|
||
|
||
```
|
||
Ctrl+S or File → Save
|
||
↓
|
||
do_save() ──live?──→ do_download_to_sax()
|
||
↓ offline ↓
|
||
execute_direct_overwrite_save() write_sax_to_path(path)
|
||
↓ ↓
|
||
sync_live_metadata_to_dom() sync_live_metadata_to_dom()
|
||
↓ ↓
|
||
├── prune dead comps open(path, "wb")
|
||
└── copy canvas meta→DOM ↓
|
||
↓ ET.write() → done
|
||
reconcile_schema_kits_before_save()
|
||
↓ user cancels? → rollback, return
|
||
↓ user confirms
|
||
open(path, "wb")
|
||
↓
|
||
b'<?xml version="1.0"...?>\n' + ET.write()
|
||
↓
|
||
mark_clean() → canvas bg → CLEAN_TINT
|
||
mark_saved() → _saved_depth = len(_undo_stack)
|
||
↓
|
||
refresh_treeview_from_dom()
|
||
↓
|
||
status bar + success dialog
|
||
```
|
||
|
||
The key design insight: the save pipeline is **not just serialization**. It's the reconciliation point where deferred state (pending kits, canvas positions, pruned components) gets committed to the DOM before writing. The file on disk is always a complete, self-consistent snapshot — the in-flight editing state that lives in `canvas_comp_map`, `pending_schema_kits`, and `active_components_list` never leaks into the file partially.
|
||
|
||
---
|
||
|
||
## Trace 10 — File Open Pipeline
|
||
|
||
The file open pipeline crosses **5 communities**: the file layer (`sax_file.py`), the schema registry (`schema.py`), the palette (`palette.py`), the wiresheet renderer (`wiresheet.py`), and the main controller (`main.py`). It transforms a `.sax` file on disk into a fully interactive editing session in seven phases.
|
||
|
||
### Entry Point
|
||
|
||
`open_file()` ([sax_file.py:126](sax_file.py:126)) is the sole entry point, wired to the File menu. It gates on two preconditions before anything happens:
|
||
|
||
**Gate 1 — Sedona home** ([sax_file.py:127](sax_file.py:127)): if `SEDONA_HOME` isn't configured, a warning fires and the function returns. Without it, no manifests can be found, so no types can be resolved, and the file would be useless.
|
||
|
||
**Gate 2 — Live mode** ([sax_file.py:131](sax_file.py:131)): if a device connection is active, the user is asked whether to disconnect. This requires a deferred import of `do_device_disconnect` from `main.py` to break a circular dependency — `sax_file` imports `main` at call time, not at module load time.
|
||
|
||
### Phase 1 — Teardown
|
||
|
||
Before the new file loads, the old session is dismantled completely ([sax_file.py:154–170](sax_file.py:154)):
|
||
|
||
```python
|
||
app_state.current_file_path = file_path
|
||
app_state.active_components_list.clear()
|
||
app_state.pending_schema_kits.clear()
|
||
app_state.selected_canvas_tag = None
|
||
app_state.current_sheet_parent = None
|
||
undo.reset()
|
||
app_state.mark_clean()
|
||
|
||
app_state.tree_element_map.clear()
|
||
app_state.canvas_comp_map.clear()
|
||
app_state.canvas_tag_by_element.clear()
|
||
for item in tree.get_children():
|
||
tree.delete(item)
|
||
canvas.delete("all")
|
||
wiresheet.draw_grid()
|
||
```
|
||
|
||
This clears **seven categories of state**: selection state, component tracking maps, the undo history, the dirty flag, the treeview contents, and the canvas. `draw_grid()` puts the background grid lines back on the now-empty canvas so the user sees a blank sheet rather than a blank white rectangle.
|
||
|
||
The file path is set **before** parsing, not after. This means a failed parse leaves `current_file_path` pointing at the file that couldn't be opened — which `reset_to_startup_state()` cleans up in the exception handler.
|
||
|
||
### Phase 2 — XML Parsing
|
||
|
||
```python
|
||
xml_tree = ET.parse(file_path)
|
||
app_state.xml_root_element = xml_tree.getroot()
|
||
```
|
||
|
||
Two lines. `ElementTree.parse()` reads the file, validates well-formedness, and builds the in-memory DOM. The root element (`<sedonaApp>` typically) becomes the single source of truth for the entire session — every component, link, and property is a live reference into this tree. No copy is made; edits throughout the session mutate this same tree, and saving writes it back.
|
||
|
||
If the file is malformed XML, `ET.parse()` raises `ParseError`, caught by the outer handler which calls `reset_to_startup_state()`.
|
||
|
||
### Phase 3 — Schema Registry Compilation
|
||
|
||
```python
|
||
schema.load_schema_kits_and_manifests()
|
||
```
|
||
|
||
This is the critical step (traced in detail in Trace 8). It reads the `<schema>` block's `<kit>` entries, resolves each kit to a manifest file under `SEDONA_HOME/manifests/`, and compiles the full type registry into `final_components_registry`. Every `kit::Type` string in the file becomes a list of resolved slots — inherited, sorted, with facets.
|
||
|
||
**This is the one step that can reject a file after parsing succeeds.** If any declared kit has no manifest on this machine, `MissingManifestError` is raised. The handler at [sax_file.py:189](sax_file.py:189) shows all missing kit names in one dialog, calls `reset_to_startup_state()`, and the file is not opened. The error message points the user at the Sedona home folder — the fix is to install the missing manifests there, not to edit the file.
|
||
|
||
### Phase 4 — Navigation Tree Construction
|
||
|
||
```python
|
||
file_name = os.path.basename(file_path)
|
||
root_node = tree.insert("", "end", text=file_name, open=True)
|
||
|
||
app_element = app_state.xml_root_element.find("app")
|
||
if app_element is not None:
|
||
parse_xml_to_tree(tree, root_node, app_element)
|
||
else:
|
||
parse_xml_to_tree(tree, root_node, app_state.xml_root_element)
|
||
```
|
||
|
||
The tree is rooted at the filename, with the `<app>` element as the real starting point. `parse_xml_to_tree()` ([sax_file.py:34](sax_file.py:34)) recursively walks the DOM, inserting a Treeview row for every `<comp>` element. Each row is mapped in `tree_element_map[treeview_id] → xml_element`, which is how clicking a tree node later resolves to a DOM element.
|
||
|
||
The display text is `"name [id: N]"` when the component has an ID, or just `"name"` otherwise. Components that are folders (containing child components) automatically become expandable nodes because their children are recursively inserted.
|
||
|
||
### Phase 5 — Post-Tree Callback
|
||
|
||
```python
|
||
if _tree_filled is not None:
|
||
_tree_filled()
|
||
```
|
||
|
||
This calls `on_tree_filled()` ([main.py:1699](main.py:1699)), registered at startup via `set_tree_filled_callback()`. It's a decoupling seam — `sax_file.py` doesn't import `main.py` at module level (that would be circular), so it calls through a function pointer instead.
|
||
|
||
`on_tree_filled()` does five things:
|
||
|
||
1. **Closes stale editor panels** ([main.py:1710](main.py:1710)): if a properties panel or link panel is showing a component that no longer exists in the DOM, it's force-closed. This matters more for rebuilds after undo/delete than for fresh opens, but the same path runs in both cases.
|
||
|
||
2. **Refreshes dirty display** ([main.py:1715](main.py:1715)): applies or removes the red-name tag on the tree's root row. On a fresh open this clears it (the file was just loaded, so it's clean).
|
||
|
||
3. **Fits tree column** ([main.py:1716](main.py:1716)): auto-sizes the tree column width to fit the longest visible node text, so component names aren't truncated.
|
||
|
||
4. **Refreshes the palette** ([main.py:1717](main.py:1717)): `refresh_palette_kits()` ([main.py:2502](main.py:2502)) invalidates the palette cache, reads `active_schema_kits` (populated in Phase 3), and rebuilds the kit dropdown in the palette panel. Only kits declared in the file's schema appear — the palette shows what this app uses, not everything installed.
|
||
|
||
5. **Detects incomplete checksums** ([main.py:1719](main.py:1719)): if the file is newly opened (`current_file_path` changed) and any declared kit is missing a checksum, the Manage Kits dialog opens automatically via `show_kits()`. This nudges the user to pin versions rather than leaving the schema ambiguous. If checksums are incomplete, the palette is left empty (`palette_kit_names = []`) to prevent placing components from an ambiguous build.
|
||
|
||
### Phase 6 — First Wiresheet Render (Deferred)
|
||
|
||
The open pipeline itself does **not** render the wiresheet. The wiresheet is painted when the user clicks a tree node, which fires `on_tree_select()` ([main.py:621](main.py:621)). On a fresh open, this happens in one of two ways:
|
||
|
||
- The tree auto-selects the root node after insertion, generating a `<<TreeviewSelect>>` event
|
||
- The user clicks a component node in the tree
|
||
|
||
Either way, `on_tree_select()` reads the selected element from `tree_element_map`, sets `current_sheet_parent`, and calls `wiresheet.render_components()` ([wiresheet.py:941](wiresheet.py:941)) with the element's children.
|
||
|
||
`render_components()` does the actual painting:
|
||
|
||
1. **Clears the canvas** of component and link tags (the grid stays)
|
||
2. **Builds linked-slot index** — which slots have wires, so they're visible even if they'd normally be hidden
|
||
3. **Calculates meta scale** — components positioned beyond the normal 0–255 meta range get a coarser grid step
|
||
4. **For each component**: reads the `meta` property, unpacks the packed x/y coordinates, calls `measure_box()` to compute dimensions from slot count and text widths, then draws the box with header, type label, and slot rows
|
||
5. **Draws links** (`draw_links()`) — the wire routing system traced in Trace 5, connecting slot endpoints across boxes
|
||
|
||
### Phase 7 — View History Reset
|
||
|
||
Inside `on_tree_filled()`, the view history is reset ([main.py:1719–1721](main.py:1719)) when the file path changes:
|
||
|
||
```python
|
||
if app_state.current_file_path != _history_file_path:
|
||
_history_file_path = app_state.current_file_path
|
||
reset_view_history()
|
||
```
|
||
|
||
`reset_view_history()` ([main.py:1587](main.py:1587)) clears the back/forward stack and disables the navigation buttons. The old file's navigation trail is discarded — you can't go "back" to a sheet in a file that's no longer loaded.
|
||
|
||
### Error Recovery
|
||
|
||
The `try/except` wrapping Phases 1–5 catches two failure modes:
|
||
|
||
**`MissingManifestError`** ([sax_file.py:189](sax_file.py:189)): Phase 3 failed — the file parsed fine but needs kits this machine doesn't have. The error dialog names every missing kit and points at the Sedona home folder. `reset_to_startup_state()` unwinds everything back to a clean "no file open" state.
|
||
|
||
**General `Exception`** ([sax_file.py:197](sax_file.py:197)): anything else — malformed XML, I/O error, an unexpected crash in tree construction. Same recovery: show the error, reset to startup state.
|
||
|
||
`reset_to_startup_state()` ([sax_file.py:82](sax_file.py:82)) is thorough: it clears `current_file_path`, `xml_root_element`, every selection, every map, the undo stack, the dirty flag, the tree, and the canvas. The app returns to the exact state it was in before any file was opened — a blank slate.
|
||
|
||
### The Full Sequence
|
||
|
||
```
|
||
File → Open
|
||
↓
|
||
open_file()
|
||
↓
|
||
Gate: SEDONA_HOME set? ──no──→ warning, return
|
||
↓ yes
|
||
Gate: live mode? ──yes──→ ask disconnect ──no──→ return
|
||
↓ no ↓ yes
|
||
do_device_disconnect()
|
||
↓
|
||
filedialog.askopenfilename() ──cancelled──→ return
|
||
↓ path chosen
|
||
┌─ try ──────────────────────────────────────────────┐
|
||
│ │
|
||
│ Phase 1: Teardown │
|
||
│ clear all maps, selections, undo, dirty flag │
|
||
│ delete tree rows, clear canvas, redraw grid │
|
||
│ │
|
||
│ Phase 2: XML Parse │
|
||
│ ET.parse(path) → xml_root_element │
|
||
│ │
|
||
│ Phase 3: Schema Registry │
|
||
│ load_schema_kits_and_manifests() │
|
||
│ → read <kit> entries from <schema> │
|
||
│ → find_manifest_path() per kit │
|
||
│ → compile_schema_registry() │
|
||
│ → parse manifests, resolve inheritance │
|
||
│ → populate final_components_registry │
|
||
│ │
|
||
│ Phase 4: Navigation Tree │
|
||
│ insert root node (filename) │
|
||
│ parse_xml_to_tree() recursively │
|
||
│ → one row per <comp>, mapped in tree_element_map│
|
||
│ │
|
||
│ Phase 5: Post-Tree Callback │
|
||
│ on_tree_filled() │
|
||
│ → close stale panels │
|
||
│ → refresh dirty display (clean) │
|
||
│ → fit tree column widths │
|
||
│ → refresh palette (kit dropdown) │
|
||
│ → auto-open Manage Kits if checksums missing │
|
||
│ → reset view history │
|
||
│ │
|
||
└─────────────────────────────────────────────────────┘
|
||
↓ exception?
|
||
├── MissingManifestError → dialog naming missing kits
|
||
└── Exception → dialog with error message
|
||
↓ either
|
||
reset_to_startup_state() (back to blank)
|
||
|
||
↓ success (deferred)
|
||
<<TreeviewSelect>> event
|
||
↓
|
||
on_tree_select()
|
||
↓
|
||
Phase 6: Wiresheet Render
|
||
render_components()
|
||
→ measure boxes (from registry slots)
|
||
→ draw boxes at meta-decoded positions
|
||
→ route and draw wires (draw_links)
|
||
|
||
Phase 7: View History
|
||
reset_view_history() (new file, fresh trail)
|
||
```
|
||
|
||
### Key Design Insight
|
||
|
||
The open pipeline is **two-phase by design**: the synchronous part (Phases 1–5) builds the data structures, and the wiresheet render (Phase 6) is deferred to the tree selection event. This means the file is "open" — the tree is populated, the registry is compiled, the palette is loaded — before any boxes are drawn. A file with 200 components across 15 folders doesn't pay the cost of rendering all of them on open; it renders only the sheet the user clicks into.
|
||
|
||
The separation also means the wiresheet renderer never needs to know whether it's painting for a fresh open, an undo, a delete, or a navigation — it always gets a list of component elements and paints them. The caller decides which elements to hand it.
|
||
|
||
---
|
||
|
||
## Trace 11 — Live Device Connection Pipeline
|
||
|
||
The live connection pipeline crosses **6 communities**: the connect dialog (`connect_dialog.py`), the SOX protocol client (`sox.py`), the DASP transport layer (also `sox.py`), the device session bridge (`device_session.py`), the schema registry (`schema.py`), and the main controller (`main.py`). It replaces the file-backed DOM with one synthesized from a live Sedona device, then keeps it updated through COV polling.
|
||
|
||
### Entry Points
|
||
|
||
Two menu items funnel through the same connection callback:
|
||
|
||
**Device > Connect** calls `do_device_connect()` ([main.py:1999](main.py:1999)), which opens `ConnectDialog` — a manual host/port/user/password form.
|
||
|
||
**Device > Discover** calls `do_device_discover()` ([main.py:2020](main.py:2020)), which opens `DiscoverDialog` — a DASP broadcast scan that lists responding devices, then prompts for credentials when one is selected.
|
||
|
||
Both pass `_connect_to_device` as the callback.
|
||
|
||
### Phase 0 — Teardown of Offline Session
|
||
|
||
Before the dialog opens, both entry points handle the current file ([main.py:2001–2015](main.py:2001)):
|
||
|
||
```python
|
||
if app_state.current_file_path and not app_state.live_mode:
|
||
if app_state.is_dirty:
|
||
answer = messagebox.askyesnocancel(...)
|
||
if answer is None: return
|
||
if answer: sax_file.execute_direct_overwrite_save()
|
||
sax_file.reset_to_startup_state()
|
||
sax_file.refresh_treeview_from_dom(None)
|
||
wiresheet.render_components([])
|
||
```
|
||
|
||
If a file is open with unsaved changes, the user gets a Save/Don't Save/Cancel choice. Then the entire offline session is torn down — the same `reset_to_startup_state()` that file-open uses — before the dialog appears. The editor goes to its blank startup state.
|
||
|
||
### Phase 1 — Dialog and Background Thread
|
||
|
||
`ConnectDialog.__init__()` ([connect_dialog.py:24](connect_dialog.py:24)) builds a modal dialog with host, port, username, and password fields. It pre-fills from the last successful connection (persisted in `sedona_editor.ini` via `_load_last_connection()`).
|
||
|
||
When the user clicks Connect, `_do_connect()` ([connect_dialog.py:87](connect_dialog.py:87)) validates inputs and spawns a **daemon background thread**:
|
||
|
||
```python
|
||
self._thread = threading.Thread(
|
||
target=self._connect_worker,
|
||
args=(host, port_int, user, password),
|
||
daemon=True
|
||
)
|
||
self._thread.start()
|
||
self._poll_connect()
|
||
```
|
||
|
||
The connection must happen off the main thread because DASP handshaking involves UDP round-trips that can take seconds. The dialog polls every 100ms via `_poll_connect()` ([connect_dialog.py:129](connect_dialog.py:129)) using Tkinter's `.after()`, checking whether the thread finished.
|
||
|
||
`_connect_worker()` ([connect_dialog.py:118](connect_dialog.py:118)) calls the callback — `_connect_to_device()` — and stores the result as `("ok", result, ...)` or `("error", msg, ...)`. If the user cancels during the connection, `_do_cancel()` ([connect_dialog.py:146](connect_dialog.py:146)) sets `_cancelled = True` and spawns a cleanup thread that joins the worker and disconnects if a session was already established.
|
||
|
||
### Phase 2 — DASP + SOX Connection
|
||
|
||
`_connect_to_device()` ([main.py:2041](main.py:2041)) creates a `DeviceSession` and calls `ds.connect()` ([device_session.py:44](device_session.py:44)), which does five things in sequence on the background thread:
|
||
|
||
**Step 2a — DASP handshake**: `SoxClient.__init__()` stores credentials. `SoxClient.connect()` ([sox.py:375](sox.py:375)) creates a `DaspSocket`, calls `self._dasp_socket.connect(host, port, user, password)` which performs the DASP (Datagram Authenticated Session Protocol) challenge-response over UDP: HELLO → CHALLENGE → AUTHENTICATE → WELCOME. This establishes an encrypted authenticated session.
|
||
|
||
**Step 2b — Receiver thread**: `_SoxReceiver` ([sox.py:960](sox.py:960)) is started — a daemon thread that loops calling `DaspSession.receive()`, dispatching incoming datagrams as either responses (correlated to pending requests) or events (COV notifications). This thread runs for the lifetime of the connection.
|
||
|
||
**Step 2c — Read schema**: `self.sox_client.read_schema()` ([sox.py:408](sox.py:408)) sends the SOX `'v'` command and parses the `'V'` response into a list of `KitPart(name, checksum)` — the kits installed in the device's scode. This also calls `read_version()` to get the platform ID, scode flags, and kit version strings.
|
||
|
||
**Step 2d — Build slot registry**: `_load_slot_registry(parts)` ([device_session.py:87](device_session.py:87)) builds a **per-device slot registry** that maps `(kit_id, type_id)` to slot lists. This is separate from `final_components_registry` — it's the SOX-level slot map used for protocol operations (subscribe, read, write). For each kit part:
|
||
|
||
1. Find the manifest on disk via `schema.find_manifest(kit_name, checksum)`
|
||
2. If the kit is `sys`, extract `Component`'s base slots (meta, status, etc.)
|
||
3. Register each type's slots keyed by `(kit_id, type_id)`
|
||
4. Map `(kit_id, type_id) → "kit::TypeName"` in `type_name_map`
|
||
5. Update any already-cached components' `type_name`
|
||
|
||
After all kits, base slots from `sys::Component` are prepended to every type and slot IDs are renumbered to match the inherited-chain order. This renumbering is critical — SOX links use absolute slot IDs across the full inheritance chain, not the per-type local IDs the manifest declares.
|
||
|
||
**Step 2e — Load root component**: `self.sox_client.load(0)` ([sox.py:458](sox.py:458)) reads component 0 (the app root) from the device. This sends the SOX `'t'` (tree read) command for comp_id 0. The response is parsed by `_apply_to_cache()`, which creates a `SoxComponent` in the 1024-slot cache array with the component's name, kit_id, type_id, parent, children list, and subscription state.
|
||
|
||
Then `load_children(root_comp.id)` loads all direct children of the root, and `update(children, CONFIG | RUNTIME)` reads their config and runtime slot values. The device session returns `root_comp` to the caller.
|
||
|
||
### Phase 3 — Thread-to-UI Crossing
|
||
|
||
Back in `_connect_to_device()`:
|
||
|
||
```python
|
||
app_state.device_session = ds
|
||
app_state.live_mode = True
|
||
app_state.connection_status = "connected"
|
||
app_state.window.after(0, lambda: _on_connected(ds, root_comp))
|
||
```
|
||
|
||
The state flags are set (these are safe from a background thread — they're simple Python assignments). Then `window.after(0, ...)` schedules `_on_connected()` on the **main Tkinter thread**. This is the thread-crossing point — everything from here runs on the main thread where Tk operations are safe.
|
||
|
||
### Phase 4 — DOM Synthesis
|
||
|
||
`_on_connected()` ([main.py:2056](main.py:2056)) runs on the main thread:
|
||
|
||
**Step 4a — Window title**: switches to the live-mode title showing `host:port`.
|
||
|
||
**Step 4b — Menu state**: disables Connect/Discover, enables Disconnect and Download to SAX.
|
||
|
||
**Step 4c — Build DOM tree**: `ds.build_dom_tree(root_comp)` ([device_session.py:272](device_session.py:272)) synthesizes an `xml.etree.ElementTree` from the SOX cache — the same data structure a `.sax` file parse would produce:
|
||
|
||
```python
|
||
sax_root = ET.Element("sedonaApp")
|
||
schema_elem = ET.SubElement(sax_root, "schema")
|
||
# add <kit> entries from schema_parts
|
||
app_elem = ET.SubElement(sax_root, "app")
|
||
# recursively build <comp> elements from cache
|
||
```
|
||
|
||
`_build_comp_element()` ([device_session.py:298](device_session.py:298)) creates each `<comp>` with name, id, type attributes and `<prop>` sub-elements for every non-action slot that has a cached value. Float values are formatted to the precision declared in the slot's facets. Children are built recursively; uncached children get placeholder `<comp name="...">` elements so the tree shows an expand arrow.
|
||
|
||
**Step 4d — Build links**: `ds.build_links_elements(sax_root)` ([device_session.py:334](device_session.py:334)) scans every cached component's links and creates `<link from="..." to="...">` elements under a `<links>` container. Link endpoints are resolved from SOX `(comp_id, slot_id)` pairs to `"/path/to/comp.slotName"` strings — the same format `.sax` files use. Duplicate links are deduplicated via a `seen` set.
|
||
|
||
Slot name resolution (`_slot_name_by_id` at [device_session.py:369](device_session.py:369)) prefers `final_components_registry` (which has correctly resolved absolute slot IDs from inheritance) over the per-device `slot_registry` (which has local IDs that are wrong for inherited types). This two-tier fallback keeps things working even before the schema registry is fully compiled.
|
||
|
||
`app_state.xml_root_element = sax_root` — from this point on, the editor's DOM is the synthesized tree, indistinguishable to the rest of the codebase from one loaded from a file.
|
||
|
||
**Step 4e — Fake file path**: `app_state.current_file_path = f"device://{host}"` — a synthetic path so the window title and tree root display something meaningful, and so `current_file_path` being non-None signals "a session is active."
|
||
|
||
### Phase 5 — Schema Registry Compilation (Live Variant)
|
||
|
||
`_load_live_schema(ds)` ([main.py:2077](main.py:2077)) compiles the editor-side schema registry from the device's kit info. Unlike the file-open path that calls `load_schema_kits_and_manifests()`, this path:
|
||
|
||
1. Populates `active_schema_kits` directly from `schema_parts`, converting integer checksums to hex strings
|
||
2. Collects manifest paths itself, **tolerating missing kits** (status bar warning, not an error)
|
||
3. Calls `compile_schema_registry()` directly with whatever manifests were found
|
||
|
||
This tolerance is the key difference from file-open. A device might run kits the local machine doesn't have manifests for (a custom industrial kit, for example). File-open refuses — the editor can't describe types it can't resolve. Live mode proceeds gracefully — the wiresheet shows boxes for components whose types resolve, and any that don't resolve simply have no slots painted.
|
||
|
||
### Phase 6 — Tree and Wiresheet
|
||
|
||
```python
|
||
sax_file.refresh_treeview_from_dom(sax_root)
|
||
```
|
||
|
||
The standard tree rebuild from Trace 10's Phase 4. Since the synthesized DOM has the same structure as a parsed `.sax`, the same `parse_xml_to_tree()` recursion populates the navigation tree identically.
|
||
|
||
This triggers `on_tree_filled()`, which refreshes the palette, resets view history, and auto-opens Manage Kits if any kit lacks a checksum. The wiresheet renders when the user clicks a tree node, exactly as in the file-open path.
|
||
|
||
### Phase 7 — COV Polling Startup
|
||
|
||
```python
|
||
undo.reset()
|
||
app_state.mark_clean()
|
||
_start_cov_polling()
|
||
```
|
||
|
||
Undo is reset (live mode doesn't use undo — hardware writes can't be reversed) and the document is marked clean.
|
||
|
||
`_start_cov_polling()` ([main.py:2136](main.py:2136)) starts the 100ms Tkinter `.after()` loop traced in Trace 3:
|
||
|
||
```python
|
||
def poll():
|
||
ds = app_state.device_session
|
||
if ds is None or not ds.is_connected:
|
||
do_device_disconnect()
|
||
return
|
||
|
||
events = ds.poll_events()
|
||
if events:
|
||
for comp_id, what_char in events:
|
||
ds.update_element_from_cache(comp_id, app_state.xml_root_element)
|
||
wiresheet.render_current_sheet()
|
||
properties_panel.draw() # if open
|
||
|
||
_cov_poll_id = app_state.window.after(100, poll)
|
||
```
|
||
|
||
This is the heartbeat of live mode: every 100ms, drain the thread-safe event queue (fed by `_SoxReceiver`), copy updated slot values from the SOX cache into the synthesized DOM, and repaint. Connection loss is detected when `ds.is_connected` returns `False`, triggering automatic disconnect.
|
||
|
||
### Disconnect
|
||
|
||
`do_device_disconnect()` ([main.py:2106](main.py:2106)) is the inverse:
|
||
|
||
1. Cancel the COV poll timer
|
||
2. `ds.disconnect()` — unsubscribes all components, closes the DASP session, kills the receiver thread, clears the cache
|
||
3. Clear all live-mode state flags
|
||
4. Reset window title and menu states
|
||
5. `reset_to_startup_state()` — back to blank editor
|
||
|
||
### The Full Sequence
|
||
|
||
```
|
||
Device > Connect (or Discover)
|
||
↓
|
||
Phase 0: Teardown offline session
|
||
save if dirty → reset_to_startup_state()
|
||
↓
|
||
ConnectDialog / DiscoverDialog
|
||
↓ user clicks Connect
|
||
Phase 1: Background thread
|
||
↓
|
||
Phase 2: _connect_to_device() (background thread)
|
||
↓
|
||
2a. DaspSocket.connect() → UDP HELLO/CHALLENGE/AUTH/WELCOME
|
||
2b. _SoxReceiver.start() → daemon thread for incoming packets
|
||
2c. read_schema() → SOX 'v' → kit names + checksums
|
||
2d. _load_slot_registry() → parse manifests → slot_registry
|
||
2e. load(0) + load_children → SOX 't'/'c'/'r' → cache populated
|
||
↓
|
||
Phase 3: Thread crossing
|
||
app_state.live_mode = True
|
||
window.after(0, _on_connected)
|
||
↓
|
||
Phase 4: _on_connected() (main thread)
|
||
↓
|
||
4a. Window title → "Live Editor — host:port"
|
||
4b. Menu states → disable Connect, enable Disconnect
|
||
4c. build_dom_tree() → SoxCache → ET.Element tree
|
||
4d. build_links_elements() → link records → <link> elements
|
||
4e. xml_root_element = sax_root
|
||
↓
|
||
Phase 5: _load_live_schema()
|
||
active_schema_kits from schema_parts
|
||
compile_schema_registry() (tolerates missing kits)
|
||
↓
|
||
Phase 6: refresh_treeview_from_dom()
|
||
parse_xml_to_tree() → navigation tree
|
||
on_tree_filled() → palette, history, dirty display
|
||
↓
|
||
Phase 7: _start_cov_polling()
|
||
100ms .after() loop: queue → DOM → repaint
|
||
```
|
||
|
||
### The Two Registries
|
||
|
||
The live connection maintains **two parallel slot registries**, and understanding why is key to understanding the whole pipeline:
|
||
|
||
| | `slot_registry` | `final_components_registry` |
|
||
|---|---|---|
|
||
| **Location** | `sox_client.slot_registry` | `app_state.final_components_registry` |
|
||
| **Keyed by** | `(kit_id, type_id)` | `"kit::TypeName"` string |
|
||
| **Slot IDs** | Local per-type (from manifest) | Absolute across inheritance chain |
|
||
| **Built by** | `_load_slot_registry()` | `compile_schema_registry()` |
|
||
| **Used for** | SOX protocol operations (subscribe, write_prop, invoke) | UI rendering (wiresheet boxes, properties panel, wire endpoints) |
|
||
| **Inheritance** | Base slots prepended, IDs renumbered | Recursive `get_slots_with_inheritance()` |
|
||
|
||
The SOX protocol addresses slots by `(comp_id, slot_id)` where `slot_id` is the position in the flattened inheritance chain. The UI addresses slots by name within a `kit::Type`. Both registries resolve the same types from the same manifests, but structure them for their respective consumers. `_slot_name_by_id()` is the bridge — it translates protocol-level `(kit_id, type_id, slot_id)` into a slot name using whichever registry is available, preferring `final_components_registry` for its correct absolute IDs.
|
||
|
||
---
|
||
|
||
## Trace 12 — DASP Transport Layer
|
||
|
||
DASP (Datagram Authenticated Session Protocol) is the entire bottom of the stack — everything beneath SOX. It provides reliable, authenticated, full-duplex communication over UDP, ported directly from the Sedona Java sources. The implementation lives entirely in [dasp.py](dasp.py) (1034 lines) with three classes: `DaspMsg` (wire format), `DaspSession` (session state machine), and `DaspSocket` (multiplexer and I/O threads).
|
||
|
||
### Architecture Overview
|
||
|
||
```
|
||
┌──────────────────────────────────────────────────┐
|
||
│ SOX (sox.py) │
|
||
│ SoxClient.request() / .requests() │
|
||
│ _SoxReceiver thread │
|
||
├──────────────────────────────────────────────────┤
|
||
│ DASP Session (dasp.py : DaspSession) │
|
||
│ send() / receive() │
|
||
│ handshake, keepalive, reliable delivery │
|
||
│ SendWindow / ReceiveWindow │
|
||
├──────────────────────────────────────────────────┤
|
||
│ DASP Socket (dasp.py : DaspSocket) │
|
||
│ _receiver_loop thread (UDP recv → dispatch) │
|
||
│ _housekeeping_loop thread (100ms tick) │
|
||
│ _send_msg (encode → UDP send) │
|
||
├──────────────────────────────────────────────────┤
|
||
│ UDP (socket.SOCK_DGRAM) │
|
||
└──────────────────────────────────────────────────┘
|
||
```
|
||
|
||
There are **four threads** in a live connection:
|
||
1. **Main thread** — Tkinter UI + COV poll
|
||
2. **DASP Receiver** — `DaspSocket._receiver_loop` — UDP recv, decode, dispatch
|
||
3. **DASP Housekeeping** — `DaspSocket._housekeeping_loop` — 100ms tick for keepalive/retries/timeouts
|
||
4. **SOX Receiver** — `_SoxReceiver.run()` — pulls from `DaspSession._receive_queue`, dispatches events vs responses
|
||
|
||
### Layer 1 — Wire Format (`DaspMsg`)
|
||
|
||
`DaspMsg` ([dasp.py:123](dasp.py:123)) is the binary codec. Every DASP datagram has this structure:
|
||
|
||
```
|
||
Bytes 0-1: session_id (u16)
|
||
Bytes 2-3: seq_num (u16)
|
||
Byte 4: msg_type (high nibble) | num_fields (low nibble)
|
||
Bytes 5+: header fields, then payload
|
||
```
|
||
|
||
There are **8 message types**: DISCOVER (0), HELLO (1), CHALLENGE (2), AUTHENTICATE (3), WELCOME (4), KEEPALIVE (5), DATAGRAM (6), CLOSE (7). Only DATAGRAM carries application-layer (SOX) payload; the rest are control messages.
|
||
|
||
Header fields are self-describing TLV entries. Each field has a 1-byte ID whose low 2 bits encode the type: NIL (0), U2 (1, 2-byte unsigned), STR (2, null-terminated), BYTES (3, length-prefixed). The decoder walks fields sequentially, matching IDs to named attributes:
|
||
|
||
```python
|
||
if fid == HDR_VERSION: self.version = u2_val
|
||
elif fid == HDR_REMOTE_ID: self.remote_id = u2_val
|
||
elif fid == HDR_NONCE: self.nonce = bytes_val
|
||
elif fid == HDR_DIGEST: self.digest = bytes_val
|
||
...
|
||
```
|
||
|
||
The encoder backpatches two things: byte 4 (msg_type | num_fields, counted during encoding) and `HDR_MESSAGE_LEN` (the total datagram length, an Ontrol extension). The `include_message_len` flag controls whether this Ontrol extension is emitted — the Sedona spec doesn't have it, but Ontrol devices expect it.
|
||
|
||
### Layer 2 — The Handshake
|
||
|
||
`DaspSession.connect()` ([dasp.py:636](dasp.py:636)) runs the 3-step handshake:
|
||
|
||
**Step 1 — HELLO** ([dasp.py:657](dasp.py:657)):
|
||
|
||
```python
|
||
msg.msg_type = HELLO
|
||
msg.session_id = 0xFFFF # broadcast — no session yet
|
||
msg.remote_id = self.id # "this is my local session ID"
|
||
msg.version = 0x0100 # protocol version 1.0
|
||
msg.ideal_max = 512 # preferred max datagram size
|
||
msg.abs_max = 512 # absolute max datagram size
|
||
msg.receive_max = 31 # max outstanding unacked msgs
|
||
msg.receive_timeout = 30000 # ms before declaring timeout
|
||
```
|
||
|
||
Sent up to 3 times with `connect_timeout / 3` between attempts. If no response comes, `"No response from hello"` is raised. The response is either CHALLENGE (proceed to auth), WELCOME (no auth required), or CLOSE (rejected).
|
||
|
||
**Step 2 — AUTHENTICATE** ([dasp.py:679](dasp.py:679)):
|
||
|
||
The device sends a CHALLENGE containing a `nonce` (random bytes) and a `digest_algorithm` ("SHA-1" or "SHA-256"). Authentication is a two-layer hash:
|
||
|
||
```python
|
||
# Inner: always SHA-1, regardless of negotiated algorithm
|
||
cred = SHA1("user:password")
|
||
|
||
# Outer: uses negotiated algorithm
|
||
digest = algo(cred + nonce)
|
||
```
|
||
|
||
The inner hash is always SHA-1 — this is the Sedona credential format, not a negotiation result. The outer hash uses whatever the device requested. The AUTHENTICATE message carries the username and the digest bytes. Again sent up to 3 times, dropping duplicate CHALLENGEs (which arrive when multiple HELLOs crossed in flight).
|
||
|
||
**Step 3 — WELCOME** (or CLOSE):
|
||
|
||
A WELCOME response means the device accepted the credentials. `_tune()` ([dasp.py:714](dasp.py:714)) negotiates session parameters by taking the **minimum** of local and remote values for `ideal_max` and `abs_max`, the **maximum** for `receive_timeout`, and initializes the receive window from the WELCOME's seq_num. The send window size is set to `min(remote_receive_max, ack_more_byte_limit)`.
|
||
|
||
### Layer 3 — Reliable Delivery
|
||
|
||
DASP provides TCP-like reliability over UDP through two sliding window mechanisms:
|
||
|
||
#### SendWindow ([dasp.py:422](dasp.py:422))
|
||
|
||
A linked list of `_Packet` objects, each tracking:
|
||
- `seq_num` — 16-bit wrapping sequence number
|
||
- `payload` — the raw bytes
|
||
- `acked` — whether the remote has acknowledged this packet
|
||
- `sent_time`, `send_attempts` — for retry timing
|
||
- `enqueued_time` — for total-timeout detection
|
||
|
||
`send()` ([dasp.py:460](dasp.py:460)) assigns the next sequence number, enqueues the packet, and transmits immediately. If the window is full (`_size >= send_size`, typically 7–8 packets), `_enqueue()` **blocks** on a `Condition` until the remote acks something. This is the backpressure mechanism — a fast sender can't overwhelm a slow device.
|
||
|
||
`check_ack_headers()` ([dasp.py:505](dasp.py:505)) processes incoming ACKs. DASP uses **cumulative ACK + selective bitmask**:
|
||
- `msg.ack` — the highest sequence number received in order (cumulative)
|
||
- `msg.ack_more` — a bitmask of additional packets received out of order
|
||
|
||
The `_is_acked()` method ([dasp.py:546](dasp.py:546)) checks both: a packet is acked if its seq_num is below the cumulative ACK *or* its bit is set in `ack_more`. Acked packets are dequeued from the head of the linked list.
|
||
|
||
**Retransmission** has two triggers:
|
||
|
||
1. **Timer-based** (`send_retries()` at [dasp.py:565](dasp.py:565)): called every 100ms by housekeeping. Each unacked packet's retry interval uses **exponential backoff**: `send_retry * 2^(attempts-1)`, capped at `max_retry_backoff` (8 seconds). Default base retry is 1 second.
|
||
|
||
2. **Fast retransmit** (`_track_dup_acks()` at [dasp.py:533](dasp.py:533)): if the same ACK number arrives twice (2 duplicate ACKs), the head-of-line unacked packet is immediately retransmitted — classic TCP-style fast recovery without waiting for the timer.
|
||
|
||
#### ReceiveWindow ([dasp.py:318](dasp.py:318))
|
||
|
||
Manages incoming sequence numbers. `receive()` ([dasp.py:354](dasp.py:354)) accepts a seq_num and decides whether it's in the valid window of 31 packets:
|
||
|
||
- **Next in sequence** (`num == start + 1`): slides the window forward. If the next packet after that was already received out of order (bit set in `_seq_more`), the window keeps sliding until it hits a gap — this is how out-of-order arrivals are coalesced.
|
||
- **Within window but out of order**: sets the corresponding bit in `_seq_more`.
|
||
- **Outside window** (duplicate or too far ahead): rejected, returns `False`.
|
||
|
||
`set_ack_headers()` ([dasp.py:342](dasp.py:342)) piggybacks ACK information on outgoing messages: it sets `msg.ack` to the current window edge and `msg.ack_more` to the selective bitmask, only when there's room in the datagram (checked against `ideal_max`). This piggyback is why keepalives carry ack information too — they're an opportunity to acknowledge data even when no application response is pending.
|
||
|
||
### Layer 4 — DaspSocket (Multiplexer)
|
||
|
||
`DaspSocket` ([dasp.py:854](dasp.py:854)) owns the UDP socket and multiplexes sessions on it:
|
||
|
||
**Construction** ([dasp.py:860](dasp.py:860)): creates a UDP socket, binds to an ephemeral port, and starts two daemon threads:
|
||
|
||
**Receiver thread** (`_receiver_loop` at [dasp.py:940](dasp.py:940)): infinite loop of `recvfrom_into()` with a 1-second timeout. Each datagram is decoded into a `DaspMsg`, then `_dispatch()` routes it:
|
||
|
||
```python
|
||
def _dispatch(self, msg, addr, raw=None):
|
||
# DISCOVER responses go to the discovery list
|
||
if msg.msg_type == DISCOVER and self._discover_active:
|
||
self._discovered.append((addr[0], msg.platform_id))
|
||
return
|
||
|
||
session = self._sessions.get(msg.session_id)
|
||
if session is None: return # unknown session
|
||
if session.host != addr[0]: return # wrong source IP
|
||
if session.port != addr[1]: return # wrong source port
|
||
session.dispatch(msg)
|
||
```
|
||
|
||
The session ID in bytes 0–1 of the datagram is the routing key. The IP+port check is a basic security measure — packets from unexpected sources are silently dropped.
|
||
|
||
**Housekeeping thread** (`_housekeeping_loop` at [dasp.py:1015](dasp.py:1015)): 100ms tick that calls `session.housekeeping()` on every active session:
|
||
|
||
```python
|
||
def housekeeping(self):
|
||
# Receive timeout: no packet from remote in too long
|
||
if _ticks() - self.last_receive > self.receive_timeout:
|
||
self.close(ERR_TIMEOUT, "receive timeout")
|
||
return
|
||
|
||
# Send timeout: oldest unacked packet waited too long
|
||
oldest = self.send_window.send_retries()
|
||
if oldest > self.receive_timeout:
|
||
self.close(ERR_TIMEOUT, "unacked send timeout")
|
||
return
|
||
|
||
# Keepalive: unacked data to piggyback, or idle too long
|
||
if (self.receive_window.unacked() or
|
||
_ticks() - self.last_send > self.receive_timeout / 3):
|
||
self._keepalive()
|
||
```
|
||
|
||
This does three things: (1) checks for receive timeout (device went silent), (2) triggers send retries and checks for send timeout (device isn't acking), (3) sends keepalives — either to piggyback pending ACKs or to prevent idle timeout (every `receive_timeout / 3` seconds, so the remote never sees a gap longer than 1/3 of its timeout).
|
||
|
||
### Layer 5 — Session Dispatch
|
||
|
||
`DaspSession.dispatch()` ([dasp.py:758](dasp.py:758)) is called on the DASP receiver thread for every incoming datagram:
|
||
|
||
```python
|
||
def dispatch(self, msg):
|
||
self.num_received += 1
|
||
self.last_receive = _ticks() # resets receive timeout
|
||
|
||
if msg.ack >= 0:
|
||
self.last_ack = msg.ack # for SendWindow backpressure
|
||
|
||
self.send_window.check_ack_headers(msg) # process ACKs
|
||
|
||
if msg.msg_type == DATAGRAM:
|
||
if not self.receive_window.receive(msg.seq_num):
|
||
return # duplicate or out of window
|
||
self._enqueue(msg) # accepted → deliver to SOX
|
||
elif msg.msg_type in (CHALLENGE, WELCOME):
|
||
self._enqueue(msg) # handshake responses
|
||
elif msg.msg_type == CLOSE:
|
||
self._enqueue(msg)
|
||
self._shutdown(...) # remote closed
|
||
elif msg.msg_type == KEEPALIVE:
|
||
pass # ACK processing already done above
|
||
```
|
||
|
||
Every message updates `last_receive` and processes ACK headers. Only DATAGRAMs go through the receive window's sequence-number check. Accepted messages are enqueued into `_receive_queue` (a `queue.Queue(maxsize=2000)`).
|
||
|
||
### Layer 6 — SOX Receiver Thread
|
||
|
||
`_SoxReceiver` ([sox.py:960](sox.py:960)) bridges DASP to SOX. It loops calling `session.receive(timeout=1.0)`, which blocks on the `_receive_queue`:
|
||
|
||
```python
|
||
def run(self):
|
||
while self._alive and not client.is_closed:
|
||
dasp_msg = client._session.receive(timeout=1.0)
|
||
if dasp_msg is None: continue
|
||
payload = dasp_msg.payload
|
||
msg = SoxMsg(payload)
|
||
cmd = payload[0]
|
||
if cmd == ord('e'):
|
||
client._receive_event(msg) # COV event
|
||
else:
|
||
client._receive_response(msg) # request response
|
||
```
|
||
|
||
The first byte of the SOX payload is the command character. `'e'` is an event (COV notification from the device); everything else is a response to a pending request.
|
||
|
||
### Layer 7 — Request/Response Correlation
|
||
|
||
`SoxClient.requests()` ([sox.py:878](sox.py:878)) implements batched request/response correlation:
|
||
|
||
```python
|
||
def requests(self, reqs):
|
||
with self._lock: # one batch at a time
|
||
for i, req in enumerate(reqs):
|
||
req.reply_num = i # tag each request
|
||
|
||
self._responses = [None] * len(reqs) # preallocate response slots
|
||
self._requests = reqs
|
||
|
||
for req in reqs:
|
||
self._session.send(req.to_bytes()) # send all immediately
|
||
|
||
# wait for all responses, up to 30 seconds
|
||
while not all(r is not None for r in self._responses):
|
||
self._req_event.wait(timeout=0.5)
|
||
if self.is_closed:
|
||
raise SoxException("Session closed")
|
||
|
||
return list(self._responses)
|
||
```
|
||
|
||
Each SOX request carries a `reply_num` (0–254). When the device responds, `_receive_response()` ([sox.py:926](sox.py:926)) matches the response's `reply_num` to the corresponding request and fills that slot. The `_req_event` (a `threading.Event`) wakes the waiting thread every time a response arrives.
|
||
|
||
For batches larger than 255 requests, `_chunk_requests()` splits them into 255-request chunks and runs them sequentially.
|
||
|
||
### Discovery
|
||
|
||
`DaspSocket.discover()` ([dasp.py:957](dasp.py:957)) sends a DISCOVER broadcast and collects replies:
|
||
|
||
```python
|
||
msg.msg_type = DISCOVER
|
||
msg.session_id = 0xFFFF
|
||
msg.seq_num = 0xFFFF
|
||
|
||
self._sock.sendto(data, ("255.255.255.255", port)) # broadcast
|
||
self._sock.sendto(data, ("239.255.18.76", port)) # Sedona multicast
|
||
|
||
time.sleep(wait_time) # 3 seconds default
|
||
```
|
||
|
||
During the wait, the receiver thread collects DISCOVER responses via the `_discover_active` flag, accumulating `(ip_address, platform_id)` tuples. After the timeout, results are deduplicated by IP and returned.
|
||
|
||
### Session Close
|
||
|
||
`DaspSession.close()` ([dasp.py:814](dasp.py:814)):
|
||
|
||
```python
|
||
def close(self, error_code=-1, cause="close()"):
|
||
close_msg.msg_type = CLOSE
|
||
close_msg.session_id = self.remote_id
|
||
self._send_raw(close_msg)
|
||
if error_code < 0: # clean close (not error)
|
||
time.sleep(0.05)
|
||
self._send_raw(close_msg) # send twice for reliability
|
||
self._shutdown(cause)
|
||
```
|
||
|
||
A clean close sends the CLOSE message **twice** with a 50ms gap — since DASP is over UDP and there's no guarantee of delivery, the double-send improves the chance the device sees it. An error close sends only once (the session is already broken, no point retrying).
|
||
|
||
`_shutdown()` ([dasp.py:830](dasp.py:830)) kills the send window (waking any blocked senders), puts `None` into the receive queue (waking the SOX receiver), calls `on_close` callback, and frees the session from the socket's session map.
|
||
|
||
### The Full Thread Architecture
|
||
|
||
```
|
||
Main Thread DASP Receiver Thread
|
||
│ │
|
||
│ SoxClient.requests() │ DaspSocket._receiver_loop()
|
||
│ ├─ session.send(payload) │ ├─ recvfrom_into(buf)
|
||
│ │ └─ SendWindow.send() │ ├─ DaspMsg.decode(buf)
|
||
│ │ ├─ enqueue (blocks │ └─ session.dispatch(msg)
|
||
│ │ │ if window full) │ ├─ check_ack_headers
|
||
│ │ └─ _send_raw(msg) │ │ └─ dequeue acked
|
||
│ │ └─ UDP sendto │ │ notify senders
|
||
│ │ │ └─ _enqueue(msg)
|
||
│ └─ _req_event.wait() │ └─ receive_queue.put
|
||
│ (woken by response) │
|
||
│ │
|
||
SOX Receiver Thread DASP Housekeeping Thread
|
||
│ │
|
||
│ _SoxReceiver.run() │ DaspSocket._housekeeping_loop()
|
||
│ ├─ session.receive() │ └─ every 100ms:
|
||
│ │ └─ receive_queue.get │ session.housekeeping()
|
||
│ └─ dispatch: │ ├─ check receive timeout
|
||
│ ├─ 'e' → _receive_event │ ├─ send_retries() + backoff
|
||
│ │ └─ _apply_to_cache │ └─ keepalive if idle/unacked
|
||
│ │ └─ on_event(id,ch) │
|
||
│ └─ other → _receive_response │
|
||
│ └─ _responses[i] = msg │
|
||
│ _req_event.set() │
|
||
```
|
||
|
||
### Key Design Insights
|
||
|
||
**UDP over TCP**: DASP exists because Sedona runs on microcontrollers with 50KB of RAM. TCP's sliding window, congestion control, and connection state are too heavy. DASP implements the minimum viable subset — sequence numbers, ack bitmasks, retransmission, keepalive — in a protocol small enough to fit in an embedded device's memory. The tradeoff is that the desktop client carries the complexity of reimplementing reliable delivery.
|
||
|
||
**Piggyback ACKs**: every outgoing message (including keepalives) carries acknowledgement headers. This eliminates dedicated ACK packets — data responses ack data requests, keepalives ack data that arrived during an idle period. The `set_ack_headers()` call in `_to_msg()` and `_keepalive()` is the mechanism.
|
||
|
||
**Single-lock batching**: `SoxClient.requests()` holds `self._lock` for the entire batch. This means only one SOX operation (which may be many underlying requests) can be in flight at a time. The SOX receiver thread fills `_responses[]` and signals `_req_event`; the requesting thread wakes, checks completeness, and either waits again or returns. Events arriving during a request batch are dispatched separately by the SOX receiver and don't interfere with the response correlation.
|
||
|
||
**Graceful degradation**: the `ack_more_byte_limit` (7 by default) caps how far ahead of the ACK frontier the sender can get. If the device's receive window is small, the sender blocks. If datagrams are lost, exponential backoff retries up to 8 seconds. If nothing is heard for `receive_timeout` (30 seconds), the session closes and the editor's COV poll detects the disconnect.
|
||
|
||
---
|
||
|
||
## Trace 13: SOX Protocol Commands
|
||
|
||
The SOX (Sedona Object Exchange) protocol is the application-layer command set that rides atop DASP. Where Trace 12 showed how bytes are reliably delivered, this trace shows what those bytes *mean* — a cursor-based binary RPC system with 13 distinct command verbs, a 1024-slot component cache, request/response correlation, and a COV event pipeline.
|
||
|
||
### Layer 1 — The Binary Wire Format: `SoxMsg`
|
||
|
||
Every SOX interaction is a `SoxMsg` — a cursor-based big-endian binary buffer ([sox.py:46](sox.py:46)). It wraps a `bytearray` with a read cursor (`_pos`) and provides typed accessors: `u1()`, `u2()`, `i4()`, `str()` for reading, and `write_u1()`, `write_u2()`, `write_i4()`, `write_str()` for writing. The first two bytes of every message are always the **command byte** and the **reply number** — stored as `msg.command` and `msg.reply_num` during decode.
|
||
|
||
The reply number is the correlation key. When a client sends a batch of N requests, each gets `reply_num = 0..N-1` ([sox.py:887](sox.py:887)). The receiver thread matches each response back to its slot in the `_responses` array by `reply_num & 0xFF` ([sox.py:934](sox.py:934)). This gives a maximum batch size of 255; larger batches are chunked automatically ([sox.py:918](sox.py:918)).
|
||
|
||
### Layer 2 — The Value Codec
|
||
|
||
Nine Sedona types cross the wire, each with a numeric type ID ([sox.py:27](sox.py:27)):
|
||
|
||
| ID | Type | Wire size | Codec |
|
||
|----|------|-----------|-------|
|
||
| 0 | bool | 1 byte | `u1()`, truthy check |
|
||
| 1 | byte | 1 byte | `u1()` |
|
||
| 2 | short | 2 bytes | `u2()` signed |
|
||
| 3 | int | 4 bytes | `i4()` |
|
||
| 4 | long | 8 bytes | `struct.unpack('>q')` |
|
||
| 5 | float | 4 bytes | `struct.unpack('>f')` |
|
||
| 6 | double | 8 bytes | `struct.unpack('>d')` |
|
||
| 7 | buf | 2-byte length + N bytes | raw bytes |
|
||
| 8 | str | null-terminated | `str()` |
|
||
|
||
`decode_value()` ([sox.py:193](sox.py:193)) and `encode_value()` ([sox.py:222](sox.py:222)) are the universal codec pair. Every property read, write, and COV event goes through them. The encode path has a subtle detail: float values that arrive as Python `int` are cast to `float` before packing, because `struct.pack('>f', 0)` requires a float, not an int.
|
||
|
||
### Layer 3 — The Command Table
|
||
|
||
SOX uses single-character command codes. Request letters are lowercase; responses are uppercase (the correlation check at [sox.py:940](sox.py:940) verifies `cmd == (req_cmd & ~0x20)` — i.e., the response is the uppercase of the request, or `'!'` for error):
|
||
|
||
| Request | Response | Verb | Purpose |
|
||
|---------|----------|------|---------|
|
||
| `'v'` | `'V'` | readSchema | Read kit manifest (names + checksums) |
|
||
| `'y'` | `'Y'` | readVersion | Platform ID, kit versions, properties |
|
||
| `'c'` | `'C'` | update | Read component tree/config/runtime/links |
|
||
| `'r'` | `'R'` | readProp | Read a single property value |
|
||
| `'w'` | `'W'` | writeProp | Write a single property value |
|
||
| `'i'` | `'I'` | invoke | Invoke an action slot |
|
||
| `'a'` | `'A'` | add | Add a child component |
|
||
| `'d'` | `'D'` | delete | Delete a component |
|
||
| `'n'` | `'N'` | rename | Rename a component |
|
||
| `'o'` | `'O'` | reorder | Reorder children |
|
||
| `'l'` | `'L'` | link/unlink | Add or remove a wire link |
|
||
| `'s'` | `'S'` | subscribe | Subscribe to COV events |
|
||
| `'u'` | — | unsubscribe | Cancel a subscription |
|
||
| `'e'` | — | event | Unsolicited COV event (device → editor) |
|
||
|
||
### Layer 4 — Schema Discovery: `read_schema()` and `read_version()`
|
||
|
||
Connection begins with schema discovery. `read_schema()` ([sox.py:408](sox.py:408)) sends `'v'` and parses the `'V'` response: a count byte, then N `(name, checksum)` pairs as `KitPart` objects. This tells the editor which kits the device runs and at what build checksum. The result is cached in `self.schema_parts` — a second call returns instantly.
|
||
|
||
`read_version()` ([sox.py:423](sox.py:423)) goes further: it sends `'y'` to get the platform ID, scode flags, per-kit version strings, and key-value properties. The critical property is `soxVer` — the SOX protocol version. Its presence (checked via `self.sox_version` at [sox.py:446](sox.py:446)) gates whether the client uses batch subscribe/unsubscribe (SOX v2+) or the legacy per-component approach. This is the **version fork** that runs through every subscribe/unsubscribe call.
|
||
|
||
### Layer 5 — Component Loading: `load()` → `load_ids()` → `_apply_to_cache()`
|
||
|
||
Loading a component means requesting its tree record from the device. `load(comp_id)` ([sox.py:458](sox.py:458)) checks the cache first, then delegates to `load_ids()`. `load_ids()` ([sox.py:467](sox.py:467)) is the batch loader: it filters out already-cached IDs, sends `'c'` requests with `what='t'` (tree) for the rest, then feeds every `'C'` response through `_apply_to_cache()`.
|
||
|
||
`_apply_to_cache()` ([sox.py:767](sox.py:767)) is the **central cache mutation point** — every response and every event converges here. It reads `comp_id` and `what` from the message, then dispatches:
|
||
|
||
- **`what == 't'` (tree):** If uncached, `_apply_tree_new()` creates a fresh `SoxComponent` with kit_id, type_id, name, parent, permissions, and children list. If cached, `_apply_tree_cached()` updates the existing object. Both call `_apply_tree_body()` ([sox.py:808](sox.py:808)), which does something critical: it compares old vs. new children sets and **recursively removes** any children that disappeared (`cache_remove()` at [sox.py:757](sox.py:757) walks the subtree).
|
||
|
||
- **`what == 'c'` or `'C'` (config):** `_apply_props()` walks the type's slot registry, decoding each config property value from the message. Uppercase `'C'` means operator-only — a filtered view.
|
||
|
||
- **`what == 'r'` or `'R'` (runtime):** Same as config but for runtime slots. The `what_char == 'r'` check at [sox.py:838](sox.py:838) skips config slots; `'c'` skips runtime. This is how the same `_apply_props()` method serves both.
|
||
|
||
- **`what == 'l'` (links):** `_apply_links()` reads `Link` tuples (from_comp, from_slot, to_comp, to_slot) until the 0xFFFF sentinel, and replaces `comp.links`.
|
||
|
||
The slot-walking in `_apply_props()` ([sox.py:824](sox.py:824)) deserves attention. It calls `self._get_slots(kit_id, type_id)` to get the type's slot list from `slot_registry`, then iterates in declaration order, decoding one value per slot. This is **positional** — the protocol sends values in slot order with no slot-ID prefix. If the registry is wrong or out of sync, every subsequent value in the message decodes from the wrong offset. The `try/except` at [sox.py:843](sox.py:843) catches the resulting struct errors and breaks the loop, silently dropping remaining values rather than corrupting them.
|
||
|
||
### Layer 6 — Property Read/Write
|
||
|
||
**Read:** `read_prop()` ([sox.py:604](sox.py:604)) sends `'r'` with comp_id and slot_id. The response echoes both IDs plus the type ID, then the value. This is used for on-demand single-property reads.
|
||
|
||
**Write:** `write_prop()` ([sox.py:617](sox.py:617)) sends `'w'` with comp_id, slot_id, and the encoded value. The `'W'` response is a bare acknowledgment — no echo. This is the hot path for the Properties panel's live editing: user changes a field → `properties.py` calls `device_session.write_property()` → which calls `sox_client.write_prop()`.
|
||
|
||
### Layer 7 — Structural Mutations: add, delete, rename, reorder
|
||
|
||
**Add** ([sox.py:638](sox.py:638)) sends `'a'` with parent_id, kit_id, type_id, name, and optional config values. The response returns the new component's ID. The client then creates a local `SoxComponent`, resolves its type name via `_resolve_type_name()` ([sox.py:859](sox.py:859)), adds it to the cache, and appends it to the parent's children list. This is **optimistic local update** — the cache reflects the addition before any tree event arrives.
|
||
|
||
**Delete** ([sox.py:666](sox.py:666)) sends `'d'` with comp_id. After the `'D'` acknowledgment, the client unlinks the component from its parent's children list and calls `cache_remove()` — which recursively removes all descendants. Again optimistic: the cache is pruned immediately.
|
||
|
||
**Rename** ([sox.py:681](sox.py:681)) sends `'n'` with comp_id and new_name. After `'N'`, the cached component's name is updated locally.
|
||
|
||
**Reorder** ([sox.py:693](sox.py:693)) sends `'o'` with comp_id and the new children ID sequence. After `'O'`, `comp.children` is replaced with the new order. This is used when the user drags components in the tree to reorder them.
|
||
|
||
### Layer 8 — Wire Links: link, unlink, read_links
|
||
|
||
Link operations use the `'l'` command with a **sub-command byte** — `ord('a')` for add-link, `ord('d')` for delete-link ([sox.py:707](sox.py:707)). The `_link_cmd()` helper ([sox.py:713](sox.py:713)) packs: sub-command + from_comp_id + from_slot_id + to_comp_id + to_slot_id. The `'L'` response is a bare acknowledgment.
|
||
|
||
`read_links()` ([sox.py:725](sox.py:725)) takes a different approach — it sends a `'c'` (update) request with `what='l'` (links), not the `'l'` (link) command. The response is a `'C'` containing Link tuples terminated by `from_comp == 0xFFFF`. This overloading — `'c'` for reads, `'l'` for mutations — reflects the protocol's dual use of the `'c'` command as a general "read component data" verb.
|
||
|
||
### Layer 9 — Subscriptions and COV Events
|
||
|
||
Subscriptions tell the device to push change-of-value (COV) events to the editor. The subscription mask uses four bits: `TREE=0x01`, `CONFIG=0x02`, `RUNTIME=0x04`, `LINKS=0x08` ([sox.py:20](sox.py:20)).
|
||
|
||
**SOX v2+ batch subscribe** ([sox.py:524](sox.py:524)): sends `'s'` with mask + count + comp_id list in a single message. Efficient — one round-trip for many components.
|
||
|
||
**Legacy subscribe** ([sox.py:538](sox.py:538)): sends separate `'s'` requests per component per mask bit. Four components × 3 mask bits = 12 requests. The `requests()` batching still sends them in one round-trip, but the server processes them individually.
|
||
|
||
**Tree-wide subscription** ([sox.py:560](sox.py:560)): `subscribe_all_tree_events()` subscribes to ALL tree events across the entire component hierarchy. On SOX v2+, this sends mask=0xFF + count=0 (a special sentinel). On legacy, it subscribes comp 0 with mask `'a'` (all). After this, all components in the cache get their `TREE` subscription bit set.
|
||
|
||
**COV event dispatch:** When the `_SoxReceiver` thread ([sox.py:960](sox.py:960)) sees `cmd == 'e'`, it calls `_receive_event()` ([sox.py:947](sox.py:947)), which skips the command and reply bytes, then calls `_apply_to_cache(msg, is_event=True)`. The `is_event=True` flag triggers two additional behaviors: (1) auto-set the subscription mask bit via `_what_to_mask()` ([sox.py:1027](sox.py:1027)), and (2) invoke `self.on_event(comp_id, what_char)` — the callback that `device_session.py` wires to a queue, which `main.py` drains on its 100ms Tkinter poll loop.
|
||
|
||
### Layer 10 — Request/Response Correlation
|
||
|
||
The `requests()` method ([sox.py:878](sox.py:878)) is the single networking funnel. It:
|
||
|
||
1. **Guards** against a closed session (raises `SoxException`).
|
||
2. **Chunks** batches > 255 via `_chunk_requests()`.
|
||
3. **Assigns** `reply_num = 0..N-1` to each request under `self._lock`.
|
||
4. **Sends** all requests via `self._session.send()` (DASP reliable delivery).
|
||
5. **Waits** up to 30 seconds, polling every 500ms via `self._req_event` (a `threading.Event`).
|
||
6. **Correlates** responses: `_receive_response()` ([sox.py:926](sox.py:926)) matches each incoming message's `reply_num` to the response slot, verifying that the response command is the uppercase of the request command (or `'!'` for error).
|
||
7. **Returns** the completed response list, or raises `SoxException("Request timeout")`.
|
||
|
||
The `_SoxReceiver` daemon thread ([sox.py:960](sox.py:960)) runs the dispatch loop: receive a DASP message → wrap payload in `SoxMsg` → route `'e'` to `_receive_event`, `'k'`/`'z'` (file transfer) to discard, everything else to `_receive_response`. It catches `DaspException` (session closed) and generic `Exception` (malformed message) separately — the former breaks the loop, the latter continues.
|
||
|
||
### Layer 11 — Cache Architecture
|
||
|
||
The cache is a flat array of 1024 `SoxComponent` slots ([sox.py:353](sox.py:353)), indexed by component ID — O(1) lookup. `cache_add()` ([sox.py:751](sox.py:751)) grows the array by doubling when a component ID exceeds the current size. `cache_remove()` ([sox.py:757](sox.py:757)) recursively nullifies the component and all its descendants — a subtree wipe.
|
||
|
||
Each `SoxComponent` ([sox.py:284](sox.py:284)) holds: `id`, `kit_id`, `type_id`, `type_name`, `name`, `parent`, `children`, `permissions`, `values` (dict of slot_name → value), `links` (list of `Link`), and `subscription` (bitmask of active COV subscriptions). The `values` dict is the live truth — updated by `_apply_props()` on every response and event.
|
||
|
||
### Architectural Insight
|
||
|
||
SOX is a **positional binary protocol with optimistic local caching**. The critical design tension is between the positional value encoding (which demands a correct slot registry to decode) and the optimistic cache updates (which apply mutations locally before the server echoes them back as events). The slot registry acts as the Rosetta Stone: without it, `_apply_props()` cannot decode a single property value, because there are no field separators or type tags in the wire format — just raw bytes in slot-declaration order. This is why `device_session._load_slot_registry()` runs before any property read, and why a schema mismatch between editor and device corrupts silently rather than failing loudly.
|
||
|
||
---
|
||
|
||
## Trace 14: Invoke / Action Slot Flow
|
||
|
||
This trace follows a user interaction in live mode from two distinct entry points — the tree's root context menu and the Properties panel — through the `live_hooks` interception layer, down to the SOX `'i'` command on the wire.
|
||
|
||
### Entry Point A — App-Level Actions: The Tree Root Context Menu
|
||
|
||
Right-clicking the root node of the navigation tree while connected to a live device triggers `_on_tree_right_click()` at [main.py:910](main.py:910). When the clicked item has no parent (`parent_id == ""`), the code recognizes it as the tree root and populates a context menu with five app-level actions from `_APP_ACTIONS` ([main.py:487](main.py:487)): **save**, **hibernate**, **quit**, **restart**, **reboot**. Each menu entry is wired with a lambda that captures the action name and calls `_invoke_app_action(a)`.
|
||
|
||
`_invoke_app_action()` ([main.py:457](main.py:457)) is a self-contained pipeline:
|
||
|
||
1. **Guard:** Checks `device_session` is connected.
|
||
2. **Cache lookup:** Fetches component 0 (the Sedona `app` object) from `ds.sox_client.cache_get(0)`.
|
||
3. **Slot registry lookup:** Reads `slot_registry[(comp.kit_id, comp.type_id)]` — the device-side slot list for the app component's type.
|
||
4. **Slot search:** Finds the slot by name *and* `is_action` flag: `next((s for s in slots if s["name"] == slot_name and s.get("is_action")), None)`.
|
||
5. **Invoke:** Calls `ds.sox_client.invoke(0, slot["id"])` — comp_id is always 0 for the app component.
|
||
6. **Post-invoke side effects:** If the action was `"save"`, calls `app_state.mark_clean()` (the device persisted everything, so the editor's dirty flag clears). If the action was `"hibernate"`, `"quit"`, `"restart"`, or `"reboot"`, calls `do_device_disconnect()` — the device is about to go away, so the editor tears down the live session preemptively.
|
||
|
||
### Entry Point B — Component-Level Actions: `live_hooks.live_invoke()`
|
||
|
||
Any component's action slots can be invoked through `live_invoke()` at [live_hooks.py:176](live_hooks.py:176). This follows the same pattern as every other `live_hooks` function:
|
||
|
||
1. **Guard:** `is_live()` checks three conditions — `app_state.live_mode`, `device_session is not None`, and `device_session.is_connected` ([live_hooks.py:12](live_hooks.py:12)).
|
||
2. **Component ID resolution:** `_comp_id(element)` ([live_hooks.py:200](live_hooks.py:200)) maps the DOM element to a numeric comp_id. An `<app>` tag is always 0; everything else reads the `id` attribute.
|
||
3. **Slot resolution:** `_find_slot(ds, comp_id, slot_name)` ([live_hooks.py:212](live_hooks.py:212)) resolves the slot name to its numeric ID through a **two-tier lookup**: first the UI-level `final_components_registry` (keyed by type name), then the protocol-level `slot_registry` (keyed by `(kit_id, type_id)` tuple). The registry entry is returned as a dict with `name`, `id`, and `type_name`.
|
||
4. **Invoke:** `ds.sox_client.invoke(comp_id, slot_info["id"])`.
|
||
|
||
### The Two-Tier Slot Resolution
|
||
|
||
The `_find_slot()` helper at [live_hooks.py:212](live_hooks.py:212) is the critical bridge between the editor's world (slot names like `"reset"`) and the protocol's world (slot IDs like `5`). It first queries `final_components_registry` — the UI-level registry compiled from kit manifests by `schema.py`. If the component's type name is resolved via `type_name_map[(kit_id, type_id)]` and found in the registry, it returns an entry with `slot_id` from the manifest. Only if that fails does it fall back to `slot_registry[(kit_id, type_id)]` — the protocol-level registry built by `device_session._load_slot_registry()` during connection.
|
||
|
||
This preference order matters: the UI registry comes from the manifest files on disk (more complete metadata, including facets and type information), while the protocol registry is synthesized from the SOX schema read (just names and IDs). For action slots, the distinction is subtle — both know the slot's name and numeric ID — but the UI registry also carries the `is_action` flag, the `type` (always `"void"` for pure actions), and facets like `@hidden`.
|
||
|
||
### The Wire: `sox.invoke()`
|
||
|
||
`SoxClient.invoke()` ([sox.py:627](sox.py:627)) builds the SOX `'i'` command:
|
||
|
||
```
|
||
byte 0: 'i' (0x69) — command
|
||
byte 1: reply_num — correlation ID
|
||
byte 2-3: comp_id — big-endian u16
|
||
byte 4: slot_id — u8
|
||
[optional: encoded argument value]
|
||
```
|
||
|
||
The optional argument (`type_id` + `arg`) supports action slots that take a parameter — not common in Sedona, but the protocol allows it. `encode_value()` packs the argument in the slot's declared type format.
|
||
|
||
The response is a bare `'I'` acknowledgment — no payload beyond the command byte and reply number. `check_response('I')` verifies the uppercase echo; if the device returns `'!'` (error), `check_response` raises `SoxException`.
|
||
|
||
### The Properties Panel: Actions as Read-Only Display
|
||
|
||
Notably, the Properties panel does **not** render action slots as invoke buttons. `get_full_slots_for_type()` ([schema.py:292](schema.py:292)) returns all slots including actions, and `_build_slot_row()` ([properties.py:753](properties.py:753)) builds a row for each one. But action slots have type `"void"`, so `display_text()` ([properties.py:351](properties.py:351)) returns the string `"action"` — and since they are not editable (`is_editable()` checks for the `'c'` config flag, which actions lack), the Entry widget is set to `readonly` state. The user sees `"action"` in a greyed-out field with no way to interact.
|
||
|
||
The `@hidden` facet ([properties.py:247](properties.py:247)) further suppresses most action slots from the panel entirely — the manifest marks internal actions as hidden, and the panel's checkbox footer controls whether hidden rows appear.
|
||
|
||
### The `live_hooks` Interception Pattern
|
||
|
||
`live_hooks.py` ([live_hooks.py:1](live_hooks.py:1)) is the write-through interception layer for *all* live-mode mutations. It exposes seven functions, each following the same pattern:
|
||
|
||
| Function | SOX Command | Wire Code |
|
||
|----------|-------------|-----------|
|
||
| `live_set_prop_value()` | `write_prop()` | `'w'` → `'W'` |
|
||
| `live_add_component()` | `add()` | `'a'` → `'A'` |
|
||
| `live_delete_component()` | `delete()` | `'d'` → `'D'` |
|
||
| `live_rename_component()` | `rename()` | `'n'` → `'N'` |
|
||
| `live_add_link()` | `link()` | `'l'` → `'L'` |
|
||
| `live_delete_link()` | `unlink()` | `'l'` → `'L'` |
|
||
| `live_invoke()` | `invoke()` | `'i'` → `'I'` |
|
||
|
||
Each function: guards on `is_live()` → resolves comp_id from DOM element → resolves slot/type info from the two-tier registry → calls the `SoxClient` method → returns `True`/`False`. Callers use the boolean to decide what to show in the status bar.
|
||
|
||
The **property write path** ([live_hooks.py:20](live_hooks.py:20)) has an extra step: `_coerce_value()` ([live_hooks.py:248](live_hooks.py:248)) converts the string from the Properties panel's Entry widget into the Python type the SOX codec expects — `bool`, `int`, `float`, or `str` — based on the slot's type ID. Without this, `encode_value()` would try to `struct.pack('>f', "3.14")` and crash.
|
||
|
||
### The Save Button's Dual Path
|
||
|
||
The Properties panel's `save()` method ([properties.py:1102](properties.py:1102)) collects all edited rows, validates them via `parse_text()`, then writes each change to both the DOM (`set_prop_value()`) and the device (`live_hooks.live_set_prop_value()`). This is **write-through**: the DOM is the primary store, the device is a side-channel that may fail independently. If the device write fails, the DOM still has the value, and the status bar reports which properties failed on the device while succeeding locally.
|
||
|
||
### Architectural Insight
|
||
|
||
The invoke/action flow reveals a **deliberate asymmetry** in the editor's live mode architecture. Property writes are dual-tracked (DOM + device), with the DOM as the authoritative copy and the device as a write-through target that may fail. But action invocations are **device-only** — there is no DOM representation of "invoke was called." The `_invoke_app_action()` path doesn't touch the DOM at all; it sends the SOX command and reacts to the consequences (mark clean after save, disconnect after reboot). This is the correct model: an action is an imperative command to the device, not a state change to be persisted, and the editor's undo system has nothing to track for it.
|
||
|