Py_sax2sab/README.md

# PySedonac

Pure-Python Sedona app converter, both directions — `.sax` -> `.sab` and `.sab` -> `.sax`. No Java,
no JRE.

Ported from the Sedona 1.2 runtime library — `sedona/src/sedona/src/sedona/offline/OfflineApp.java`
and friends, **not** from `sedonac/` (that's the language compiler; app file conversion lives in the
runtime lib).

Verified byte-identical to `sedonac.exe` in both directions on a 189-component / 217-link app across
18 kits.

## Run it

    python run.py                    the window
    pythonw run.py                   the window with no console behind it
    python run.py app.sab [out]      straight to the command line converter

`run.py` is the single entry point: no arguments opens the GUI, any argument is handed to the
command line converter. Double-click it (or a shortcut to `pythonw run.py`) and you get the window.

## The window

    python run.py            (or: python gui.py)

Title bar carries the name and version (`SAB - SAX Sedona Files Converter 0.0.0.002`). One button
per direction — **SAB file to SAX file** and **SAX file to SAB file** — and the button you press
decides which extension the file chooser offers. The log shows the chosen file and its folder
*before* the conversion runs, then the output file and its folder when it finishes:

    sab -> sax
      INPUT FILE:
        app.sab
      INPUT PATH:
        C:\apps
      OUTPUT FILE:
        app-sab2sax-20260728-140609.sax
      OUTPUT PATH:
        C:\apps
      wrote 61294 bytes

The result is written next to the input under that timestamped name, so nothing is overwritten and
no save dialog gets in the way. Sedona home is shown at the top with a `Change...` button that
rewrites `system.properties`.

## The command line

    python run.py [in.sax|in.sab] [out]      (or: python sab2sax.py ...)

Like `sedonac`, the direction follows the input's extension: feed it a `.sax` and you get a `.sab`,
feed it a `.sab` and you get a `.sax`. Both arguments are optional — `sab2sax.py` on its own asks for
what it needs:

* no input → a file chooser opens, listing `.sax` and `.sab`;
* no output → a save dialog opens next to the input, pre-filled with a timestamped name:
  `app.sax` → `app-sab2sax-20260728-140609.sab`, so a run never overwrites the previous one.

`python sab.py ...` still works and does the same thing — it hands off to `sab2sax.py`.

Sedona home is *not* an argument — it comes from `system.properties` (below). As a library:

    import sab2sax
    sab2sax.convert("app.sax")                     # -> ("app-sab2sax-<stamp>.sab", nbytes)
    sab2sax.convert("app.sab", "out.sax")          # explicit output, no dialogs

    import sab, sax
    sab.encode("app.sax", home, "out.sab")         # one direction each, if you prefer
    sax.decode("app.sab", home, "out.sax")         # `home=None` reads system.properties

### What a `.sab` cannot give back

Only **config** props are stored in the binary, so a `.sab` -> `.sax` conversion cannot recover
runtime prop values — they come back as their manifest defaults. `sedonac` has exactly the same hole;
this is a property of the format, not of the port. (A prop equal to its default is not written to the
`.sax` at all, by either tool.)

## Configuration

Sedona home lives in `system.properties`, next to the scripts, under the same key sedonac uses:

    sedona.home=C:\path\to\sedona

It must be the directory holding `manifests/` — kit manifests are required, since slot ids and types
are resolved from them.

Plain Windows paths, no escaping. Java's `Properties` — what sedonac itself reads this file with —
treats `\` as an escape and so doubles them up; that form parses here too. The rule is `\\` collapses
to `\` and every other backslash is kept literally, so `C:\niagara` never turns into a newline.

If the key is missing, or the folder it names has no `manifests/`, a folder chooser opens on first
run and the choice is written back to `system.properties` (other lines and comments are preserved; a
duplicate `sedona.home` further down gets commented out). Dialogs need `tkinter` — on a headless box
set the key by hand instead.

`python config.py` prints the resolved home, prompting if it isn't set yet.

## Version

`config.NAME` / `config.VERSION` — currently `SAB - SAX Sedona Files Converter 0.0.0.002`. The window
title and every dialog title bar carry it; a chooser reads `SAB - SAX Sedona Files Converter
0.0.0.002 - Select the .sax or .sab application file to convert`, and the CLI prints the same line
when it starts, so it is obvious which build produced a file. Bump it in [config.py](config.py) and
all of them follow.

## Verify

    python verify.py [app.sax|app.sab] [sedona_home]

Point it at any app of yours and it round-trips it, direction by extension:

    app.sab  ->  sab -> sax -> sab      must come back byte-identical
    app.sax  ->  sax -> sab -> sax -> sab   the two sabs must be identical

The `.sax` case compares the two **binaries**, not the two texts, because going through a `.sab` is
lossy on purpose — runtime prop values and formatting are not in the binary. If the regenerated
`.sax` differs from the original, that is printed as a note rather than a failure. (For a `.sax`
sedonac itself wrote, they do match.)

With no file argument a chooser opens, listing `.sax` and `.sab`. Cancel it and the built-in
regression runs instead, against a `test_normal.sax` / `test_normal_sedonac.sab` pair in
`testprogram/`. That folder is **not** in the repo (gitignored) — drop the pair in yourself,
otherwise verify prints `SKIP` and exits 0:

    test_normal.sax -> sab            vs test_normal_sedonac.sab
    test_normal_sedonac.sab -> sax    vs test_normal.sax
    both round trips as above

`check_encode(sax)` / `check_decode(sab)` with no reference argument run `bin/sedonac.exe` on a copy
of the input and diff against that instead — a live comparison, if you have a JRE it will start
under.

Exit code is 0 on pass, 1 on failure. A malformed file fails with a one-line reason
(`comp 0 app: missing child 254`) rather than a traceback.

## Files

    run.py               entry point: window with no arguments, CLI with them
    gui.py               the window: a button per direction
    sab2sax.py           the CLI: dispatches on the input extension
    sab.py               sax -> sab encoder
    sax.py               sab -> sax decoder
    config.py            system.properties read/write, folder + file choosers
    verify.py            regression check against sedonac's output, both directions
    system.properties    sedona.home
    testprogram/         local only, gitignored: test apps to run verify.py against

## Format

Big-endian. **No padding, no alignment** — `Buf.bigEndian = true`, `checkAlignment = false`. The
`align()`/`pad()` helpers exist on `Buf` but the app encoder never calls them.

    "sapp" (i4 0x73617070) | version (i4 0x0003)
    schema:  u1 kitCount, then per kit: cstr name, i4 checksum
    u2 maxId
    components (ascending id, NOT tree order), each:
        u2 id | u1 kitId | u1 typeId
        cstr name | u2 parentId | u2 firstChildId | u2 nextSiblingId   (0xffff = none)
        config prop values in slot-id order, bare - no names, no ids
        u1 ';'
    u2 0xffff
    links: u2 fromComp | u1 fromSlot | u2 toComp | u1 toSlot
    u2 0xffff
    u1 '.'

Values: `bool` -> u1 (0/1, **2 = null**) | `byte` -> u1 | `short` -> u2 | `int` -> i4 | `long` -> i8 |
`float` -> f4 | `double` -> f8 | `Buf` -> u2 len + raw bytes | `asStr` -> u2 (len+1) + ASCII + NUL.

Slot flags (`SlotManifest.flagsToString`): `a` = action, `c` = config, `s` = asStr, `o` = operator.

## Three things that will bite you

**Kit order is normative.** `sys` at index 0, everything else alphabetical (`Schema.sortKits`). Kit id
is the position in that sorted list, and every component record references it. Document order of the
`<schema>` block in the SAX is irrelevant.

**Slot ids come from flattening.** `Type.resolveSlots` inherits base slots then appends declared ones;
the resulting index is the id written into links. An **action override** reuses the inherited id rather
than appending (`Type.addSlot`) — get that wrong and every subsequent slot id shifts.

**Missing prop is not null.** When a `<prop>` is absent the value is the manifest's `default`
attribute, or failing that `Value.defaultForType` -> **zero**. `null` (NaN, `0x7fc00000` for float)
only when the SAX literally says `val="null"`. This was the single bug between "same length" and
"byte identical".

## Going back: sab -> sax

The tree is rebuilt from the `firstChild`/`nextSibling` id chain, then written the way
`OfflineApp.encodeAppXml` does: two-space indent per level, a `<!-- /path -->` comment above every
`<comp>`, and a prop line only where the value differs from the slot default.

Two details worth knowing:

**Java float formatting.** `Value.encodeString` for a float is `java.lang.Float.toString` — shortest
digits that round-trip, always a decimal point, `E` notation outside `[1e-3, 1e7)`. `sax.java_float`
reproduces it (`2500.0`, `1.0E7`, `1.0E-4`).

**The default is a 32-bit float too.** Comparing the decoded value against a manifest default parsed
as a Python double makes `0.1` look different from `0.1` and writes out props that sedonac omits —
round the default to `f4` first.

## Not covered

Only what `testprogram/test_normal.sax` exercises is proven. Untested: action overrides, `Buf`-typed props
(base64 path is written but unexercised), non-ASCII strings (Sedona `Str` is ASCII-only and will
raise), and apps whose component ids exceed the 256-entry lookup table.

Also, `java_float` matches modern Java, not the old JDK's extra-digit quirk on subnormals — Java
prints `Float.MIN_VALUE` as `1.4E-45` where we print the equally round-trippable `1.0E-45`. Only
reachable with denormal floats in an app.

## Alternative

If a JRE is present, `sedona/bin/sedonac.exe <file.sax|.sab>` converts either direction off the file
extension, and additionally validates the app (schema resolution, RAM/FLASH sizing) in a way this
encoder does not.