Development Status

Current Milestone

Milestone 3 — SDL Video Viewport, HUD, and Wayland Compatibility

Current Architecture Decisions

Language / UI: Python 3.9+ with PySDL2 (ctypes wrapper around system SDL2), with UI layout metrics now computed from the runtime window/display size instead of being fixed to 640x480
DLNA Discovery: Custom SSDP M-SEARCH implementation using asyncio datagrams + aiohttp for device description XML
Content Browsing: Direct SOAP/XML ContentDirectory client with DIDL-Lite parser (no dependency on async-upnp-client browsing at runtime — only aiohttp)
Playback: integrated GStreamer backend via PyGObject / GstPlayBin, decoding video into GstAppSink frames that are uploaded to SDL textures and rendered in the main SDL renderer
Playback viewport: SDL scales decoded video into a dedicated playback viewport in the same render pass as the HUD, with full-width video bounds and a black playback backdrop outside the viewport
Concurrency: Dedicated asyncio event loop in a daemon thread; thread-safe queues bridge it to the SDL2 main loop
Input: Keyboard mapping for desktop testing + SDL2 GameController for R36S D-pad/buttons
Font: Bundled package font preferred first; system font fallback kept only for development hosts
UI icons: prefer bundled monochrome glyphs or a bundled icon font subset instead of depending on OS emoji fonts
Playback HUD: SDL-rendered overlay is now simplified and compact, uses bundled playback icons, uses a smaller dedicated playback font with title ellipsis for 640x480 readability, supports auto-hide or fixed visibility, stays visible while playback is paused, and remains in the same SDL render pass as video
Wayland / DRM strategy: playback no longer depends on native overlay sinks or X11 window handles; R36S-class targets continue to prefer kmsdrm when no display server is present
Deploy packaging: a conda-forge-oriented environment.yml now defines a reproducible Miniforge/Miniconda environment for local development and release preparation
Python packaging: direct runtime dependency declarations now explicitly include aiohttp instead of relying on transitive installation through other packages
ArkOS deploy layout: on-device installs should place Miniforge and the git checkout under /home/ark to avoid the full /roms partition, while EmulationStation integration should stay lightweight under /roms/ports

Completed Tasks

Phase 1: Project bootstrap (pyproject.toml, requirements.txt, README.md, package layout under src/)
Phase 2: DLNA discovery (dlna/discovery.py — SSDP M-SEARCH, friendly-name fetch) and browsing (dlna/client.py — SOAP Browse, DIDL-Lite parser with relative-URL resolution + dlna/models.py domain models + dlna/browser_state.py navigation stack/cache)
Phase 3: SDL2 UI (ui/sdl_app.py — window, event loop, input dispatch; ui/screens.py — server list, browse list, playback, error screens; ui/theme.py — runtime-scaled layout helpers for 640x480 and 720x720-class displays)
Phase 4: Playback (player/backend.py abstract interface + player/gstreamer_backend.py integrated GStreamer backend)
Phase 5: Device integration (platform/controls.py — keyboard + gamecontroller mapping; platform/runtime.py — logging, R36S heuristic, SDL env hints)
Phase 7: Tests — 75 tests across 7 test files all passing (DIDL mapping, SOAP/XML parser, navigation state, playback backend, SDL redraw policy, input controls, runtime environment setup)
Desktop runtime verification completed: fixed SSDP discovery socket setup for IPv4 and removed pending-task shutdown noise from the async worker thread
Packaging hardening: bundled a local UI font asset and configured setuptools to ship it with the package
Real LAN regression fixed: Browse SOAP parser now handles Result elements both with and without a namespace, matching responses from the discovered MiniDLNA/Jellyfin servers
Playback backend pivoted to GStreamer because libmpv continued to create a separate native window on the desktop host instead of remaining embedded in the SDL UI
Milestone 2 is now implemented with GStreamer: playback uses GstPlayBin plus GstAppSink instead of an external player, libmpv, or native overlay sinks
SDL playback flow updated: decoded GStreamer frames are uploaded into SDL textures, playback end-of-stream returns automatically to the browser, and playback controls support pause/resume, relative seek, and volume
Milestone 3 implemented in code: SDL scales video into a dedicated viewport inside the SDL window, with reserved HUD margins instead of using the whole window area for video
Playback HUD expanded: progress bar, elapsed/duration, volume, buffer, resolution, and control legends are rendered around the video area and updated from GStreamer bus/pipeline queries
Playback-page flashing root cause addressed by removing native overlay composition entirely: video and HUD are now rendered together by SDL in one pass, with redraws driven by decoded frame availability and HUD state changes
Playback HUD simplified: the border around the video area was removed, playback control/status icons were added as bundled SVG+PNG assets, the title/timer top bar no longer overlaps, and playback now supports auto / fixed / hidden HUD modes through a dedicated command while staying visible when paused
UI scaling hardened for mixed small-display targets: list rows, HUD bands, icon sizes, viewport margins, and font sizes are now derived from the actual SDL window/display size so the app remains readable on both 640x480 and 720x720 screens
Deployment assets added: .gitignore, environment.yml, and a real LICENSE file so the project can be initialized and published as a clean git repository
Conda environment refreshed for current playback needs: runtime now includes GStreamer codec/plugin packages plus explicit Python build/test tooling, while editable install keeps the package code sourced from the repo checkout
Packaging fix: pyproject.toml now uses a valid TOML [project.urls] table so editable installs work with modern pip / tomllib
Copilot instructions and this status file
Device deployment reconnaissance completed on a real ArkOS-derived R36S over SSH: /roms is full, /home/ark has free space, required download tools are present, and /roms/ports plus gamelist.xml are the least invasive integration points for launchers

Tasks In Progress

Verify that the SDL-texture playback path is smooth enough on real host playback and on R36S hardware
Measure whether BGRA frame upload is acceptable on RK3326 or whether a future YUV texture path is needed
Device deployment on the physical R36S is now wired through ArkOS Ports -> MatHacks, with the heavy runtime under /home/ark and only a lightweight stub launcher under /roms/ports
Device env bootstrap on the physical R36S reaches a clean from r36s_dlna_browser.app import Application inside /home/ark/miniconda3/envs/r36s-dlna-browser
ArkOS launcher asset added at deploy/arkos/MatHacks.sh; current launcher uses the /home/ark/R36SHack checkout plus verified LD_LIBRARY_PATH, GST_PLUGIN_PATH, and LD_PRELOAD exports needed to load the system gstreamer1.0-libav plugins from the conda runtime
Rockchip MPP hardware decode now deployed: librockchip-mpp and gst-mpp compiled from source via Docker QEMU (arm64v8/ubuntu:focal), installed on device, mppvideodec confirmed visible to GStreamer. The gstreamer_backend.py probe auto-boosts mppvideodec rank if /dev/vpu_service is accessible.
Pre-built .so files bundled in deploy/arkos/mpp-libs/; setup_hw_decode.sh installs them automatically without network access.

NV12 Render Path Benchmark Log

All runs performed on the physical R36S (RK3326, 4× A35 @ 1.3 GHz, 1 GB RAM) over SSH. Stream: 1920×1080 H.264 MKV @ 24fps via MiniDLNA over LAN. Frame budget: 41.7 ms.

Commit	Copy strategy	Copy mean	Copy % budget	FPS	Dropped	A/V drift
`a201594`	`extract_dup` → bytes + `from_buffer_copy` → ctypes (2 copies, 6 MB/frame)	36,499 µs	87.6%	24.01	1	−42.8 ms
`da02e74`	`buffer.map(READ)` + `memmove` into reusable ctypes array (1 copy, 3 MB/frame)	33,551 µs	80.5%	23.98	0	−38.0 ms

Key observations (da02e74):

1 dropped frame eliminated (0 vs 1)
Jitter improved: stdev 2.6 ms vs 3.6 ms
A/V drift tighter: −38 ms vs −43 ms
Copy cost still 80.5% of frame budget — the 3.1 MB memmove on each frame is the remaining bottleneck
Further reduction requires DMA-buf zero-copy (kernel VPU→SDL import without CPU memcpy), which depends on device driver support not currently available through gst-mpp's appsink path

Blockers Or Open Questions

SDL2_ttf system library needed for text rendering (sudo dnf install SDL2_ttf on Fedora, sudo apt install libsdl2-ttf-2.0-0 on Debian/Ubuntu). The app handles its absence gracefully but will show no text.
Integrated playback requires system GStreamer plus Python GI bindings (for Fedora: python3-gobject gstreamer1 gstreamer1-plugins-base gstreamer1-plugins-good; add codec/plugin packages as needed for target media).
Root browse verified against two real DLNA servers on the LAN.
On-device testing on R36S hardware is pending.
The current SDL-texture path avoids window-manager dependencies but may still need optimization on low-end hardware if BGRA upload cost is too high.
The first Miniforge install attempt on the physical R36S failed because the downloaded installer was corrupt and crashed during extraction.
The physical R36S now has Miniconda installed at /home/ark/miniconda3; the dedicated app env exists at /home/ark/miniconda3/envs/r36s-dlna-browser, but package solves can hang on-device and are being handled incrementally.
The dedicated R36S conda env requires LD_LIBRARY_PATH=/home/ark/miniconda3/envs/r36s-dlna-browser/lib for GI and GStreamer shared libraries to resolve correctly.
GStreamer imports now succeed in the dedicated env (GLib, GObject, Gst, GstApp, GstVideo), and Application imports cleanly.
ArkOS menu launch works on the physical device, and DLNA browsing reaches real MiniDLNA content.
Real playback is currently blocked by missing decoder elements in the device env: direct probing of a MiniDLNA .mkv URL showed missing H.264 High Profile and MPEG-4 AAC decoders, while the user-facing "can't play a text file" message is a misleading fallback caused by an additional text stream in the container.
RESOLVED: gst-libav conda package on linux-aarch64 has an unfixable ABI mismatch: libavcodec.so links libdav1d.so.6 (from dav1d <1.3) but only dav1d 1.4.x (.so.7) is available, and via libxml2-16 it also pulls libicuuc.so.78 which is not packaged for linux-aarch64 on conda-forge. Solution: install system gstreamer1.0-libav (v1.16.1) via apt and use GST_PLUGIN_PATH + LD_PRELOAD to expose its plugins to the conda Python runtime.
On the physical R36S, avdec_h264 and avdec_aac now register and resolve when launched with: LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libgomp.so.1 GST_PLUGIN_PATH=/usr/lib/aarch64-linux-gnu/gstreamer-1.0 The LD_PRELOAD is required to avoid "cannot allocate memory in static TLS block" from the conda libgomp.so being loaded late by dlopen.
These variables are now persisted in /home/ark/miniconda3/envs/r36s-dlna-browser/etc/conda/activate.d/gst-env.sh and explicitly set in deploy/run.sh.

Next Recommended Actions

Consider profiling what the remaining 19.5% of frame budget (≈8 ms) consists of — likely SDL_UpdateNVTexture upload + render call overhead + Python GIL churn. If SDL upload is the bottleneck, try SDL_LockTexture for direct write instead.
Investigate DMA-buf / dmabuf fd import as a future zero-copy path: gst-mpp may expose DRM DMA-buf fds that SDL's KMSDRM backend can import directly, eliminating the CPU memmove entirely.
Run a visual playback smoke test on device directly via the app launcher (MatHacks.sh) to confirm HUD and video render correctly together under KMSDRM at the current 80.5% copy load.
If 80.5% copy cost causes visible stutter under load (UI overhead competing for A35 cycles), the next option is to reduce resolution at the appsink by inserting a videoscale element to 1280×720 before the appsink, cutting memmove to ~1.3 MB/frame (≈35% budget).
avdec_hevc is still missing (HEVC decoders not in system apt gstreamer1.0-libav 1.16.1); mppvideodec covers H.264/H.265/VP8/VP9 via HW so this is less critical now.

12 KiB Raw Permalink Blame History