Merge sab2sax 0.0.0.001: both-direction conversion, sedona home in system.properties

.gitignore

@@ -0,0 +1,5 @@
+__pycache__/
+*.pyc
+testprogram/
+*-sab2sax-*.sab
+*-sab2sax-*.sax

README.md

@@ -1,26 +1,98 @@
 # PySedonac
 
-Pure-Python `.sax` -> `.sab` encoder. No Java, no JRE.
+Pure-Python Sedona app converter, both directions — `.sax` -> `.sab` and `.sab` -> `.sax`. No Java,
+no JRE.
 
 Ported from the Sedona 1.2 runtime library — `sedona/src/sedona/src/sedona/offline/OfflineApp.java`
 and friends, **not** from `sedonac/` (that's the language compiler; app file conversion lives in the
 runtime lib).
 
-Verified byte-identical to `sedonac.exe` on a 189-component / 217-link app across 18 kits.
+Verified byte-identical to `sedonac.exe` in both directions on a 189-component / 217-link app across
+18 kits.
 
 ## Usage
 
-    python sab.py <app.sax> <sedona_home> <out.sab>
+    python sab2sax.py [in.sax|in.sab] [out]
 
-`sedona_home` is the directory holding `manifests/` — kit manifests are required, since slot ids and
+Like `sedonac`, the direction follows the input's extension: feed it a `.sax` and you get a `.sab`,
-types are resolved from them.
+feed it a `.sab` and you get a `.sax`. Both arguments are optional, so you can run it by
+double-click:
+
+* no input → a file chooser opens, listing `.sax` and `.sab`;
+* no output → a save dialog opens next to the input, pre-filled with a timestamped name:
+  `app.sax` → `app-sab2sax-20260728-140609.sab`, so a run never overwrites the previous one.
+
+`python sab.py ...` still works and does the same thing — it hands off to `sab2sax.py`.
+
+Sedona home is *not* an argument — it comes from `system.properties` (below). As a library:
+
+    import sab2sax
+    sab2sax.convert("app.sax")                     # -> ("app-sab2sax-<stamp>.sab", nbytes)
+    sab2sax.convert("app.sab", "out.sax")          # explicit output, no dialogs
+
+    import sab, sax
+    sab.encode("app.sax", home, "out.sab")         # one direction each, if you prefer
+    sax.decode("app.sab", home, "out.sax")         # `home=None` reads system.properties
+
+### What a `.sab` cannot give back
+
+Only **config** props are stored in the binary, so a `.sab` -> `.sax` conversion cannot recover
+runtime prop values — they come back as their manifest defaults. `sedonac` has exactly the same hole;
+this is a property of the format, not of the port. (A prop equal to its default is not written to the
+`.sax` at all, by either tool.)
+
+## Configuration
+
+Sedona home lives in `system.properties`, next to the scripts, under the same key sedonac uses:
+
+    sedona.home=C:\path\to\sedona
+
+It must be the directory holding `manifests/` — kit manifests are required, since slot ids and types
+are resolved from them.
+
+Plain Windows paths, no escaping. Java's `Properties` — what sedonac itself reads this file with —
+treats `\` as an escape and so doubles them up; that form parses here too. The rule is `\\` collapses
+to `\` and every other backslash is kept literally, so `C:\niagara` never turns into a newline.
+
+If the key is missing, or the folder it names has no `manifests/`, a folder chooser opens on first
+run and the choice is written back to `system.properties` (other lines and comments are preserved; a
+duplicate `sedona.home` further down gets commented out). Dialogs need `tkinter` — on a headless box
+set the key by hand instead.
+
+`python config.py` prints the resolved home, prompting if it isn't set yet.
+
+## Version
+
+`config.NAME` / `config.VERSION` — currently `sab2sax 0.0.0.001`. Every dialog title bar reads
+`sab2sax 0.0.0.001 - Select the .sax or .sab application file to convert`, and the CLI prints the
+same line when it starts, so it is obvious which build produced a file. Bump it in
+[config.py](config.py) and both follow.
 
 ## Verify
 
     python verify.py [sedona_home]
 
-Compares our output against the checked-in `test_normal_sedonac.sab`. Pass a sedona home containing
+Without an argument the home comes from `system.properties`. Three checks, run against a
-`bin/sedonac.exe` and call `check(sax)` with no reference to diff against a live sedonac run instead.
+`test_normal.sax` / `test_normal_sedonac.sab` pair in `testprogram/`. That folder is **not** in the
+repo (gitignored) — drop the pair in yourself, otherwise verify prints `SKIP` and exits 0:
+
+    test_normal.sax -> sab        vs test_normal_sedonac.sab
+    test_normal_sedonac.sab -> sax  vs test_normal.sax
+    test_normal_sedonac.sab -> sax -> sab   back to the same bytes
+
+`check_encode(sax)` / `check_decode(sab)` with no reference argument run `bin/sedonac.exe` on a copy
+of the input and diff against that instead — a live comparison, if you have a JRE it will start
+under.
+
+## Files
+
+    sab2sax.py           the CLI: dispatches on the input extension
+    sab.py               sax -> sab encoder
+    sax.py               sab -> sax decoder
+    config.py            system.properties read/write, folder + file choosers
+    verify.py            regression check against sedonac's output, both directions
+    system.properties    sedona.home
+    testprogram/         local only, gitignored: test apps to run verify.py against
 
 ## Format
 
@@ -60,12 +132,32 @@ attribute, or failing that `Value.defaultForType` -> **zero**. `null` (NaN, `0x7
 only when the SAX literally says `val="null"`. This was the single bug between "same length" and
 "byte identical".
 
+## Going back: sab -> sax
+
+The tree is rebuilt from the `firstChild`/`nextSibling` id chain, then written the way
+`OfflineApp.encodeAppXml` does: two-space indent per level, a `<!-- /path -->` comment above every
+`<comp>`, and a prop line only where the value differs from the slot default.
+
+Two details worth knowing:
+
+**Java float formatting.** `Value.encodeString` for a float is `java.lang.Float.toString` — shortest
+digits that round-trip, always a decimal point, `E` notation outside `[1e-3, 1e7)`. `sax.java_float`
+reproduces it (`2500.0`, `1.0E7`, `1.0E-4`).
+
+**The default is a 32-bit float too.** Comparing the decoded value against a manifest default parsed
+as a Python double makes `0.1` look different from `0.1` and writes out props that sedonac omits —
+round the default to `f4` first.
+
 ## Not covered
 
-Only what `test_normal.sax` exercises is proven. Untested: action overrides, `Buf`-typed props
+Only what `testprogram/test_normal.sax` exercises is proven. Untested: action overrides, `Buf`-typed props
 (base64 path is written but unexercised), non-ASCII strings (Sedona `Str` is ASCII-only and will
 raise), and apps whose component ids exceed the 256-entry lookup table.
 
+Also, `java_float` matches modern Java, not the old JDK's extra-digit quirk on subnormals — Java
+prints `Float.MIN_VALUE` as `1.4E-45` where we print the equally round-trippable `1.0E-45`. Only
+reachable with denormal floats in an app.
+
 ## Alternative
 
 If a JRE is present, `sedona/bin/sedonac.exe <file.sax|.sab>` converts either direction off the file

config.py

@@ -0,0 +1,188 @@
+"""Configuration: sedona home lives in `system.properties` next to this file.
+
+Same key sedonac itself uses, but written as a plain Windows path:
+
+    sedona.home=C:\\path\\to\\sedona
+
+We only *read* this file, so there is no reason to escape anything. Java's
+Properties (what sedonac uses) treats `\\` as an escape, so its own files double
+them up -- `\\\\path\\\\to`. Both parse here: `\\\\` collapses to `\\`, and every
+other backslash is kept literally, so `C:\\niagara` does not turn into a newline.
+
+If the key is missing, or the directory no longer holds `manifests/`, a folder
+chooser is shown and the answer is written back to `system.properties`.
+"""
+import os, sys
+
+NAME = "sab2sax"
+VERSION = "0.0.0.001"
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+PROPS = os.path.join(HERE, "system.properties")
+KEY = "sedona.home"
+
+
+def title(text):
+    """Every dialog carries the program name and version in its title bar."""
+    return "%s %s - %s" % (NAME, VERSION, text)
+
+
+# ---------- .properties ----------
+def _unescape(v):
+    """Collapse `\\\\` to `\\`; leave any other backslash alone.
+
+    Accepts both our plain paths and java-escaped ones, and never mistakes a
+    path segment for an escape (`C:\\niagara` stays `C:\\niagara`).
+    """
+    out, i = [], 0
+    while i < len(v):
+        if v[i] == "\\" and i + 1 < len(v) and v[i + 1] == "\\":
+            i += 1
+        out.append(v[i])
+        i += 1
+    return "".join(out)
+
+
+def read_props(path=PROPS):
+    props = {}
+    if not os.path.exists(path):
+        return props
+    with open(path, "r", encoding="utf-8", errors="replace") as f:
+        for line in f:
+            line = line.strip()
+            if not line or line[0] in "#!" or "=" not in line:
+                continue
+            k, v = line.split("=", 1)
+            props[k.strip()] = _unescape(v.strip())   # last wins
+    return props
+
+
+def write_prop(key, value, path=PROPS):
+    """Set `key`, keeping every other line of the file intact."""
+    lines = []
+    if os.path.exists(path):
+        with open(path, "r", encoding="utf-8", errors="replace") as f:
+            lines = f.read().splitlines()
+
+    new = "%s=%s" % (key, value)
+    done = False
+    for i, line in enumerate(lines):
+        s = line.strip()
+        if s and s[0] not in "#!" and "=" in s and s.split("=", 1)[0].strip() == key:
+            lines[i] = new if not done else "#" + line   # comment out later dupes
+            done = True
+    if not done:
+        lines.append(new)
+
+    with open(path, "w", encoding="utf-8") as f:
+        f.write("\n".join(lines) + "\n")
+
+
+# ---------- dialogs ----------
+def _tk():
+    """A hidden root window, or None when no GUI is available."""
+    try:
+        import tkinter as tk
+        root = tk.Tk()
+        root.withdraw()
+        return root
+    except Exception:
+        return None
+
+
+def ask_directory(text, initial=None):
+    root = _tk()
+    if root is None:
+        return None
+    try:
+        from tkinter import filedialog
+        return filedialog.askdirectory(title=title(text), initialdir=initial or HERE) or None
+    finally:
+        root.destroy()
+
+
+def ask_open_file(text, filetypes, initial=None):
+    root = _tk()
+    if root is None:
+        return None
+    try:
+        from tkinter import filedialog
+        return filedialog.askopenfilename(title=title(text), filetypes=filetypes,
+                                          initialdir=initial or HERE) or None
+    finally:
+        root.destroy()
+
+
+def ask_save_file(text, filetypes, defaultextension, initialfile=None, initial=None):
+    root = _tk()
+    if root is None:
+        return None
+    try:
+        from tkinter import filedialog
+        return filedialog.asksaveasfilename(title=title(text), filetypes=filetypes,
+                                            defaultextension=defaultextension,
+                                            initialfile=initialfile,
+                                            initialdir=initial or HERE) or None
+    finally:
+        root.destroy()
+
+
+# ---------- api ----------
+def is_sedona_home(path):
+    return bool(path) and os.path.isdir(os.path.join(path, "manifests"))
+
+
+def get_sedona_home(interactive=True):
+    """Path from system.properties, prompting (and saving) when unusable."""
+    home = read_props().get(KEY)
+    if is_sedona_home(home):
+        return home
+
+    if not interactive:
+        raise SystemExit("%s not set (or has no manifests/) in %s" % (KEY, PROPS))
+
+    why = "not set in" if not home else "invalid (%s), update" % home
+    print("sedona.home %s %s -- please choose the sedona installation folder" % (why, PROPS))
+    while True:
+        picked = ask_directory("Select Sedona home (the folder containing manifests/)",
+                               home if home and os.path.isdir(home) else None)
+        if not picked:
+            raise SystemExit("no sedona home selected")
+        picked = os.path.normpath(picked)
+        if is_sedona_home(picked):
+            write_prop(KEY, picked)
+            print("saved %s=%s to %s" % (KEY, picked, PROPS))
+            return picked
+        print("  %s has no manifests/ subfolder -- try again" % picked)
+
+
+SAX = ("Sedona app XML", "*.sax")
+SAB = ("Sedona app binary", "*.sab")
+ANY = ("All files", "*.*")
+
+
+def choose_input_file(initial=None):
+    """Pick the file to convert -- direction follows its extension."""
+    return ask_open_file("Select the .sax or .sab application file to convert",
+                         [("Sedona app (*.sax, *.sab)", "*.sax *.sab"), SAX, SAB, ANY],
+                         initial)
+
+
+def choose_sax_file(initial=None):
+    return ask_open_file("Select the .sax application file", [SAX, ANY], initial)
+
+
+def choose_sab_file(initialfile=None, initial=None):
+    """Save-as for the .sab produced from a .sax."""
+    return ask_save_file("Save the .sab application file", [SAB, ANY],
+                         ".sab", initialfile, initial)
+
+
+def choose_sax_output(initialfile=None, initial=None):
+    """Save-as for the .sax produced from a .sab."""
+    return ask_save_file("Save the .sax application file", [SAX, ANY],
+                         ".sax", initialfile, initial)
+
+
+if __name__ == "__main__":
+    print(get_sedona_home())

sab.py

@@ -1,6 +1,7 @@
 """Minimal pure-Python SAX -> SAB encoder, ported from sedona.offline.OfflineApp."""
-import os, struct, sys, base64
+import os, struct, sys, base64, datetime
 import xml.etree.ElementTree as ET
+import config
 
 class Buf:
     def __init__(self): self.b = bytearray()
@@ -13,14 +14,9 @@ class Buf:
     def str_(self, s): self.b += s.encode("ascii"); self.u1(0)
 
 # ---------- schema ----------
-def load_schema(sax_root, home):
+def load_kits(parts, home):
-    parts = []
+    """[(kitName, checksumHexOrNone)] in kit-id order -> (kits, types_by_qname)."""
-    for k in sax_root.find("schema").findall("kit"):
+    kits = []          # [(name, checksum, [typeElem, ...])]
-        parts.append((k.get("name"), k.get("checksum")))
-    # sortKits: sys first, rest alphabetical
-    parts.sort(key=lambda p: ("" if p[0] == "sys" else "\x01" + p[0]))
-
-    kits = []          # [(name, checksum, {typeName: typeElem})]
     types_by_q = {}
     for kit_id, (name, cks) in enumerate(parts):
         d = os.path.join(home, "manifests", name)
@@ -35,6 +31,13 @@ def load_schema(sax_root, home):
         kits.append((name, int(mroot.get("checksum"), 16), tlist))
     return kits, types_by_q
 
+
+def load_schema(sax_root, home):
+    parts = [(k.get("name"), k.get("checksum")) for k in sax_root.find("schema").findall("kit")]
+    # sortKits: sys first, rest alphabetical
+    parts.sort(key=lambda p: ("" if p[0] == "sys" else "\x01" + p[0]))
+    return load_kits(parts, home)
+
 def resolve_slots(qname, types_by_q, cache):
     """Flatten base chain -> [(name, type, flags, default)] indexed by slot id."""
     if qname in cache: return cache[qname]
@@ -81,7 +84,19 @@ def encode_value(out, vtype, flags, raw):
         raise Exception("unhandled slot type " + str(vtype))
 
 # ---------- main ----------
-def encode(sax_path, home, sab_path):
+def default_sab_name(sax_path):
+    """`app.sax` -> `app-sab2sax-20260728-140609.sab` -- never clobbers an existing .sab."""
+    stem = os.path.splitext(os.path.basename(sax_path))[0]
+    stamp = datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
+    return "%s-sab2sax-%s.sab" % (stem, stamp)
+
+
+def encode(sax_path, home=None, sab_path=None):
+    if home is None:
+        home = config.get_sedona_home()
+    if sab_path is None:
+        sab_path = os.path.join(os.path.dirname(os.path.abspath(sax_path)),
+                                default_sab_name(sax_path))
     root = ET.parse(sax_path).getroot()
     kits, types_by_q = load_schema(root, home)
     kit_id = {k[0]: i for i, k in enumerate(kits)}
@@ -162,5 +177,5 @@ def encode(sax_path, home, sab_path):
     return len(out.b)
 
 if __name__ == "__main__":
-    n = encode(sys.argv[1], sys.argv[2], sys.argv[3])
+    import sab2sax          # the CLI converts either direction; this is just one half
-    print("wrote", n, "bytes")
+    sab2sax.main(sys.argv)

sab2sax.py

@@ -0,0 +1,66 @@
+"""Entry point: converts either direction, the way `sedonac.exe <file>` does --
+the input's extension decides.
+
+    python sab2sax.py [in.sax|in.sab] [out]
+
+    .sax in -> .sab out    (sab.encode)
+    .sab in -> .sax out    (sax.decode)
+
+Omit the input and a file chooser opens; omit the output and a save dialog opens
+next to the input, pre-filled with a timestamped name. Sedona home comes from
+system.properties (see config.py).
+"""
+import os, sys
+import config, sab, sax
+
+
+def convert(in_path, out_path=None, home=None):
+    """Convert `in_path` by extension, no dialogs. Returns (out_path, bytes written).
+
+    With no `out_path`, writes a timestamped name next to the input."""
+    ext = os.path.splitext(in_path)[1].lower()
+    if ext not in (".sax", ".sab"):
+        raise Exception("don't know what to do with %s -- expected .sax or .sab" % in_path)
+    if out_path is None:
+        name = sab.default_sab_name(in_path) if ext == ".sax" else sax.default_sax_name(in_path)
+        out_path = os.path.join(os.path.dirname(os.path.abspath(in_path)), name)
+    if ext == ".sax":
+        return out_path, sab.encode(in_path, home, out_path)
+    return out_path, sax.decode(in_path, home, out_path)
+
+
+def main(argv):
+    print("%s %s" % (config.NAME, config.VERSION))
+    home = config.get_sedona_home()
+
+    in_path = argv[1] if len(argv) > 1 else config.choose_input_file()
+    if not in_path:
+        raise SystemExit("no input file selected")
+    in_path = os.path.abspath(in_path)
+    if not os.path.isfile(in_path):
+        raise SystemExit("no such file: " + in_path)
+
+    ext = os.path.splitext(in_path)[1].lower()
+    if ext not in (".sax", ".sab"):
+        raise SystemExit("don't know what to do with %s -- expected .sax or .sab" % in_path)
+
+    if len(argv) > 2:
+        out_path = argv[2]
+    elif ext == ".sax":
+        out_path = config.choose_sab_file(initialfile=sab.default_sab_name(in_path),
+                                          initial=os.path.dirname(in_path))
+    else:
+        out_path = config.choose_sax_output(initialfile=sax.default_sax_name(in_path),
+                                            initial=os.path.dirname(in_path))
+    if not out_path:
+        raise SystemExit("no output file selected")
+
+    if ext == ".sax":
+        n = sab.encode(in_path, home, out_path)
+    else:
+        n = sax.decode(in_path, home, out_path)
+    print("wrote %d bytes to %s" % (n, out_path))
+
+
+if __name__ == "__main__":
+    main(sys.argv)

sax.py

@@ -0,0 +1,347 @@
+"""The other direction: .sab -> .sax, ported from sedona.offline.OfflineApp
+(decodeAppBinary + encodeAppXml) and OfflineComponent/OfflineLink.
+
+Output is meant to match `sedonac.exe <file.sab>` byte for byte, down to the
+`<!-- /path -->` comments and Java's float formatting.
+
+Note what a .sab simply does not contain: only *config* props are stored, so
+runtime prop values cannot come back. sedonac has the same hole -- both write
+the manifest default for them, and a value equal to its default is not written
+at all.
+"""
+import os, struct, base64, math, datetime
+import sab, config
+
+
+class Reader:
+    def __init__(self, data):
+        self.b, self.p = data, 0
+
+    def take(self, n):
+        if self.p + n > len(self.b):
+            raise Exception("truncated sab at offset %d" % self.p)
+        v = self.b[self.p:self.p + n]
+        self.p += n
+        return v
+
+    def u1(self): return self.take(1)[0]
+    def u2(self): return struct.unpack(">H", self.take(2))[0]
+    def i4(self): return struct.unpack(">i", self.take(4))[0]
+    def i8(self): return struct.unpack(">q", self.take(8))[0]
+    def f4(self): return struct.unpack(">f", self.take(4))[0]
+    def f8(self): return struct.unpack(">d", self.take(8))[0]
+
+    def str_(self):
+        end = self.b.index(0, self.p)
+        s = self.b[self.p:end].decode("ascii")
+        self.p = end + 1
+        return s
+
+
+# ---------- values ----------
+def decode_value(r, vtype, flags):
+    if "s" in flags:                                   # asStr Buf: u2 len (incl NUL)
+        n = r.u2()
+        return r.take(n).split(b"\0")[0].decode("ascii")
+    if vtype == "bool":
+        v = r.u1()
+        return None if v == 2 else (v == 1)
+    if vtype == "byte":   return r.u1()
+    if vtype == "short":  return r.u2()
+    if vtype == "int":    return r.i4()
+    if vtype == "long":   return r.i8()
+    if vtype == "float":  return r.f4()
+    if vtype == "double": return r.f8()
+    if vtype in ("sys::Buf", "buf"):
+        return bytes(r.take(r.u2()))
+    raise Exception("unhandled slot type " + str(vtype))
+
+
+def default_value(vtype, flags, raw):
+    """The manifest `default`, or Value.defaultForType -> zero. Same rules as sab.py."""
+    if "s" in flags:
+        return raw or ""
+    if vtype == "bool":
+        return None if raw == "null" else raw in ("true", "1")
+    if vtype in ("byte", "short", "int", "long"):
+        return sab._int(raw)
+    if vtype == "float":
+        # the default is a 32-bit Float too, so round before comparing:
+        # 0.1 as a double is not the 0.1 that came out of the file
+        return to_f32(sab._flt(raw))
+    if vtype == "double":
+        return sab._flt(raw)
+    if vtype in ("sys::Buf", "buf"):
+        return base64.b64decode(raw) if raw else b""
+    raise Exception("unhandled slot type " + str(vtype))
+
+
+def same(a, b):
+    if isinstance(a, float) and isinstance(b, float):
+        return (math.isnan(a) and math.isnan(b)) or a == b
+    return type(a) is type(b) and a == b
+
+
+# ---------- java number formatting ----------
+F32_MAX = 3.4028234663852886e38          # largest finite float
+F32_INF_EDGE = 3.4028235677973366e38     # midpoint to 2**128: below rounds to F32_MAX
+
+
+def to_f32(v):
+    """Round a double to the nearest float. struct raises past F32_MAX instead
+    of rounding, so handle that edge ourselves."""
+    try:
+        return struct.unpack(">f", struct.pack(">f", v))[0]
+    except OverflowError:
+        if abs(v) < F32_INF_EDGE:
+            return math.copysign(F32_MAX, v)
+        return math.copysign(math.inf, v)
+
+
+def _shortest(a, single):
+    """Fewest digits that still round-trip -- what Java's Float/Double.toString picks."""
+    for prec in range(0, 17):
+        s = "%.*e" % (prec, a)
+        v = float(s)
+        if single:
+            v = to_f32(v)
+        if v == a:
+            break
+    mant, exp = s.split("e")
+    digits = mant.replace(".", "").rstrip("0") or "0"
+    return digits, int(exp)
+
+
+def java_float(v, single=True):
+    """Java Float.toString / Double.toString: always a decimal point, E-form
+    outside [1e-3, 1e7)."""
+    if math.isnan(v):
+        return "NaN"
+    if math.isinf(v):
+        return "-Infinity" if v < 0 else "Infinity"
+    neg = math.copysign(1.0, v) < 0
+    a = abs(v)
+    if a == 0.0:
+        return "-0.0" if neg else "0.0"
+
+    digits, exp = _shortest(a, single)
+    if -3 <= exp < 7:
+        if exp >= 0:
+            head = digits[:exp + 1].ljust(exp + 1, "0")
+            tail = digits[exp + 1:] or "0"
+        else:
+            head, tail = "0", "0" * (-exp - 1) + digits
+        out = head + "." + tail
+    else:
+        out = digits[0] + "." + (digits[1:] or "0") + "E" + str(exp)
+    return "-" + out if neg else out
+
+
+def encode_string(val, vtype, flags):
+    """Value.encodeString()."""
+    if "s" in flags:
+        return _literal(val)
+    if vtype == "bool":
+        return "null" if val is None else ("true" if val else "false")
+    if vtype == "float":
+        return "null" if math.isnan(val) else java_float(val, True)
+    if vtype == "double":
+        return "null" if math.isnan(val) else java_float(val, False)
+    if vtype in ("sys::Buf", "buf"):
+        return base64.b64encode(val).decode("ascii")
+    return str(val)
+
+
+def _literal(s):
+    """TextUtil.toLiteral."""
+    out = []
+    for c in s:
+        out.append({"\0": "\\0", "\n": "\\n", "\r": "\\r", "\t": "\\t",
+                    "\\": "\\\\", '"': '\\"', "$": "\\$"}.get(c, c))
+    return "".join(out)
+
+
+def _safe(s):
+    """XWriter.safe: escape markup, quotes, and anything outside printable ASCII."""
+    out = []
+    for c in s:
+        o = ord(c)
+        if o < 0x20 or o > 0x7e or c in "'\"":
+            out.append("&#x%x;" % o)
+        elif c == "<":
+            out.append("&lt;")
+        elif c == ">":
+            out.append("&gt;")
+        elif c == "&":
+            out.append("&amp;")
+        else:
+            out.append(c)
+    return "".join(out)
+
+
+# ---------- binary decode ----------
+def read_app(sab_path, home):
+    """-> (kits, comps by id, links). Each comp: id/type/name/parent/kids/props."""
+    r = Reader(open(sab_path, "rb").read())
+    if r.i4() != 0x73617070:
+        raise Exception("%s is not a .sab (bad magic)" % sab_path)
+    ver = r.i4()
+    if ver != 0x0003:
+        raise Exception("unsupported sab version 0x%x" % ver)
+
+    parts = []
+    for _ in range(r.u1()):
+        name = r.str_()
+        parts.append((name, "%08x" % (r.i4() & 0xFFFFFFFF)))
+    kits, types_by_q = sab.load_kits(parts, home)
+    r.u2()                                      # maxId, recomputed on the way out
+
+    qname_of = {}
+    for ki, (kname, _, tlist) in enumerate(kits):
+        for t in tlist:
+            qname_of[(ki, int(t.get("id")))] = "%s::%s" % (kname, t.get("name"))
+
+    cache = {}
+    by_id, order = {}, []
+    while True:
+        cid = r.u2()
+        if cid == 0xFFFF:
+            break
+        ki, ti = r.u1(), r.u1()
+        if (ki, ti) not in qname_of:
+            raise Exception("comp %d: no type %d in kit %d" % (cid, ti, ki))
+        qname = qname_of[(ki, ti)]
+        c = {"id": cid, "type": qname, "name": r.str_(), "kids": [], "props": []}
+        c["parentId"], c["childId"], c["siblingId"] = r.u2(), r.u2(), r.u2()
+        for name, vtype, flags, dflt in sab.resolve_slots(qname, types_by_q, cache):
+            if "a" in flags or "c" not in flags:
+                continue                        # config props only, in slot-id order
+            c["props"].append((name, vtype, flags, dflt, decode_value(r, vtype, flags)))
+        if r.u1() != ord(";"):
+            raise Exception("corrupted component %d %s" % (cid, c["name"]))
+        if cid in by_id:
+            raise Exception("duplicate id: %d" % cid)
+        by_id[cid] = c
+        order.append(cid)
+
+    # rebuild the tree from the child/sibling id chain
+    for c in by_id.values():
+        c["parent"] = by_id[c["parentId"]] if c["parentId"] != 0xFFFF else None
+        if c["childId"] != 0xFFFF:
+            kid = by_id[c["childId"]]
+            while True:
+                c["kids"].append(kid)
+                if kid["siblingId"] == 0xFFFF:
+                    break
+                kid = by_id[kid["siblingId"]]
+
+    links = []
+    while True:
+        fc = r.u2()
+        if fc == 0xFFFF:
+            break
+        fs, tc, ts = r.u1(), r.u2(), r.u1()
+        for cid in (fc, tc):
+            if cid not in by_id:
+                raise Exception("link references unknown comp id %d" % cid)
+        links.append((by_id[fc], fs, by_id[tc], ts))
+    if r.u1() != ord("."):
+        raise Exception("invalid app end marker")
+
+    return kits, by_id, order, links, types_by_q, cache
+
+
+# ---------- xml encode ----------
+def path_of(c):
+    parts = []
+    while c["parent"] is not None:
+        parts.append(c["name"])
+        c = c["parent"]
+    return "/" + "/".join(reversed(parts)) if parts else "/"
+
+
+def write_xml(out, kits, app, links, types_by_q, cache):
+    w = out.append
+    w("<?xml version='1.0'?>\n")
+    w("<sedonaApp>\n")
+    w("<schema>\n")
+    for name, cks, _ in kits:
+        w('  <kit name="%s" checksum="%08x" />\n' % (_safe(name), cks & 0xFFFFFFFF))
+    w("</schema>\n")
+
+    w("<app>\n")
+    write_props(w, app, 2)
+    for kid in app["kids"]:
+        write_comp(w, kid, 2)
+    w("</app>\n")
+
+    w("<links>\n")
+    for fc, fs, tc, ts in links:
+        w('  <link from="%s.%s" to="%s.%s"/>\n' % (
+            _safe(path_of(fc)), _safe(slot_name(fc, fs, types_by_q, cache)),
+            _safe(path_of(tc)), _safe(slot_name(tc, ts, types_by_q, cache))))
+    w("</links>\n")
+    w("</sedonaApp>\n")
+
+
+def slot_name(comp, slot_id, types_by_q, cache):
+    slots = sab.resolve_slots(comp["type"], types_by_q, cache)
+    if slot_id >= len(slots):
+        raise Exception("%s has no slot id %d" % (comp["type"], slot_id))
+    return slots[slot_id][0]
+
+
+def write_props(w, c, indent):
+    """Only props whose value differs from the slot default get written."""
+    n = 0
+    for name, vtype, flags, dflt, val in c["props"]:
+        if same(val, default_value(vtype, flags, dflt)):
+            continue
+        w('%s<prop name="%s" val="%s"/>\n' % (
+            " " * indent, _safe(name), _safe(encode_string(val, vtype, flags))))
+        n += 1
+    return n
+
+
+def write_comp(w, c, indent):
+    pad = " " * indent
+    w("%s<!-- %s -->\n" % (pad, path_of(c)))
+    w('%s<comp name="%s" id="%d" type="%s"' % (pad, _safe(c["name"]), c["id"], _safe(c["type"])))
+
+    body = []
+    n = write_props(body.append, c, indent + 2)
+    for kid in c["kids"]:
+        write_comp(body.append, kid, indent + 2)
+    if not n and not c["kids"]:
+        w("/>\n")
+    else:
+        w(">\n")
+        w("".join(body))
+        w("%s</comp>\n" % pad)
+
+
+# ---------- main ----------
+def default_sax_name(sab_path):
+    """`app.sab` -> `app-sab2sax-20260728-140609.sax`."""
+    stem = os.path.splitext(os.path.basename(sab_path))[0]
+    stamp = datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
+    return "%s-sab2sax-%s.sax" % (stem, stamp)
+
+
+def decode(sab_path, home=None, sax_path=None):
+    if home is None:
+        home = config.get_sedona_home()
+    if sax_path is None:
+        sax_path = os.path.join(os.path.dirname(os.path.abspath(sab_path)),
+                                default_sax_name(sab_path))
+
+    kits, by_id, order, links, types_by_q, cache = read_app(sab_path, home)
+    if 0 not in by_id:
+        raise Exception("no app component (id 0) in " + sab_path)
+
+    out = []
+    write_xml(out, kits, by_id[0], links, types_by_q, cache)
+    text = "".join(out)
+    with open(sax_path, "w", encoding="ascii", newline="\n") as f:
+        f.write(text)
+    return len(text)

system.properties

@@ -0,0 +1,4 @@
+# Directory of the Sedona installation (the folder holding manifests/).
+# Plain path, no escaping needed. If this is missing or wrong, sab.py asks for
+# it with a folder chooser and writes the answer back here.
+sedona.home=C:\Users\aa\OneDrive - Ontrol\Documents\_is\NiagaraAXSedona\sedona

verify.py

@@ -1,53 +1,111 @@
-"""Regression check: our SAB output must be byte-identical to sedonac.exe's.
+"""Regression check: our output must be byte-identical to sedonac.exe's, both ways.
+
+    sax -> sab   check_encode()
+    sab -> sax   check_decode()
+    sab -> sax -> sab  round trip
 
 Usage:  python verify.py [sedona_home]
+
+Without an argument the home comes from system.properties (see config.py), which
+pops a folder chooser the first time.
 """
 import os, sys, subprocess, tempfile, shutil
-import sab
+import sab, sax, config
 
 HERE = os.path.dirname(os.path.abspath(__file__))
-HOME = sys.argv[1] if len(sys.argv) > 1 else os.path.join(os.path.dirname(HERE), "sedona")
+HOME = sys.argv[1] if len(sys.argv) > 1 else config.get_sedona_home()
+TESTS = os.path.join(HERE, "testprogram")
 
 
-def check(sax, reference=None):
+def sedonac(src, tmp):
-    """Encode `sax` with sab.py; compare against `reference` or a fresh sedonac run."""
+    """Run sedonac on a copy of `src`; return the file it produced (needs a JRE)."""
-    tmp = tempfile.mkdtemp()
+    work = os.path.join(tmp, os.path.basename(src))
-    try:
+    shutil.copy(src, work)
-        if reference is None:
+    exe = os.path.join(HOME, "bin", "sedonac.exe")
-            work = os.path.join(tmp, os.path.basename(sax))
+    r = subprocess.run([exe, work], capture_output=True, text=True)
-            shutil.copy(sax, work)
+    if r.returncode != 0:
-            exe = os.path.join(HOME, "bin", "sedonac.exe")
+        print("  sedonac failed:\n" + r.stdout + r.stderr)
-            r = subprocess.run([exe, work], capture_output=True, text=True)
+        return None
-            if r.returncode != 0:
+    other = ".sax" if work.endswith(".sab") else ".sab"
-                print("  sedonac failed:\n" + r.stdout + r.stderr)
+    return os.path.splitext(work)[0] + other
-                return False
-            reference = os.path.splitext(work)[0] + ".sab"
 
-        mine = os.path.join(tmp, "mine.sab")
-        sab.encode(sax, HOME, mine)
 
-        a = open(reference, "rb").read()
+def compare(label, reference, mine, binary):
-        b = open(mine, "rb").read()
+    a = open(reference, "rb").read()
-        name = os.path.basename(sax)
+    b = open(mine, "rb").read()
-        if a == b:
+    if a == b:
-            print("  PASS  %-28s %d bytes identical" % (name, len(a)))
+        print("  PASS  %-28s %d bytes identical" % (label, len(a)))
-            return True
+        return True
 
-        diffs = [i for i in range(min(len(a), len(b))) if a[i] != b[i]]
+    diffs = [i for i in range(min(len(a), len(b))) if a[i] != b[i]]
-        print("  FAIL  %-28s sedonac=%d ours=%d, %d byte diffs" % (name, len(a), len(b), len(diffs)))
+    print("  FAIL  %-28s sedonac=%d ours=%d, %d byte diffs" % (label, len(a), len(b), len(diffs)))
-        if diffs:
+    if diffs:
-            i = diffs[0]
+        i = diffs[0]
+        if binary:
             lo, hi = max(0, i - 12), i + 12
             print("        first diff at offset %d" % i)
             print("        sedonac  %s" % a[lo:hi].hex())
             print("        ours     %s" % b[lo:hi].hex())
-        return False
+        else:
+            line = a[:i].count(b"\n") + 1
+            print("        first diff at line %d" % line)
+            print("        sedonac  %s" % a.splitlines()[line - 1])
+            print("        ours     %s" % b.splitlines()[line - 1])
+    return False
+
+
+def check_encode(sax_file, reference=None):
+    """sax -> sab, against `reference` or a fresh sedonac run."""
+    tmp = tempfile.mkdtemp()
+    try:
+        reference = reference or sedonac(sax_file, tmp)
+        if reference is None:
+            return False
+        mine = os.path.join(tmp, "mine.sab")
+        sab.encode(sax_file, HOME, mine)
+        return compare(os.path.basename(sax_file) + " -> sab", reference, mine, True)
+    finally:
+        shutil.rmtree(tmp, ignore_errors=True)
+
+
+def check_decode(sab_file, reference=None):
+    """sab -> sax, against `reference` or a fresh sedonac run."""
+    tmp = tempfile.mkdtemp()
+    try:
+        reference = reference or sedonac(sab_file, tmp)
+        if reference is None:
+            return False
+        mine = os.path.join(tmp, "mine.sax")
+        sax.decode(sab_file, HOME, mine)
+        return compare(os.path.basename(sab_file) + " -> sax", reference, mine, False)
+    finally:
+        shutil.rmtree(tmp, ignore_errors=True)
+
+
+def check_round_trip(sab_file):
+    """sab -> sax -> sab must come back to the same bytes."""
+    tmp = tempfile.mkdtemp()
+    try:
+        mid = os.path.join(tmp, "mid.sax")
+        back = os.path.join(tmp, "back.sab")
+        sax.decode(sab_file, HOME, mid)
+        sab.encode(mid, HOME, back)
+        return compare(os.path.basename(sab_file) + " round trip", sab_file, back, True)
     finally:
         shutil.rmtree(tmp, ignore_errors=True)
 
 
 if __name__ == "__main__":
     print("sedona home: %s" % HOME)
-    ok = check(os.path.join(HERE, "test_normal.sax"),
+    saxf = os.path.join(TESTS, "test_normal.sax")
-               os.path.join(HERE, "test_normal_sedonac.sab"))
+    sabf = os.path.join(TESTS, "test_normal_sedonac.sab")
+    missing = [f for f in (saxf, sabf) if not os.path.isfile(f)]
+    if missing:
+        # testprogram/ is deliberately not in the repo -- put the pair back to run this
+        print("  SKIP  no test files: %s" % ", ".join(os.path.basename(f) for f in missing))
+        sys.exit(0)
+
+    ok = check_encode(saxf, sabf)
+    ok &= check_decode(sabf, saxf)
+    ok &= check_round_trip(sabf)
     sys.exit(0 if ok else 1)