Parent: #57 — Phase 1 Architecture
Summary
Decompose the monolithic InteractiveViewer (~80 instance variables, 6464 lines in engine.py) into composed subsystem objects. Each subsystem owns its own state and logic, replacing the current flat bag of instance variables with named, self-contained components that InteractiveViewer holds by reference.
Why not ECS?
The original proposal was a full Entity-Component-System architecture (World registry, entity IDs, generic query() dispatch). After research into Numba's extending API — including StructRef, @jitclass, @overload, and entry-point registration — we concluded ECS is the wrong pattern here:
-
Entity count is tiny. rtxpy scenes have ~10-20 conceptual entities (1 terrain, 1 camera, a handful of geometry layers, a few observers, 1 wind system, overlays). ECS shines with thousands-to-millions of entities iterated in cache-friendly SoA layout. With <20 entities you always know what exists and access it by name — entity-ID indirection and generic World.query() add overhead for no benefit.
-
The bottleneck is GPU, not CPU iteration. The hot path is OptiX ray tracing. CPU-side per-frame work (_tick()) is input processing, chunk loading decisions, and wind particle updates — all trivially fast relative to the ray trace. Numba-compiling ECS systems that iterate components would optimize something that doesn't matter.
-
Systems need Python interop that defeats @njit. Nearly every "system" calls into Python-only APIs — GLFW key callbacks, OptiX launch, ModernGL texture upload, zarr file I/O, PIL text rendering. You'd hit box/unbox boundaries at every system edge, negating any Numba benefit.
-
Numba extensions are for the compiler, not application architecture. The extending API (@jitclass, StructRef, @overload) is designed to teach Numba about new types — not to build application frameworks. The right use is targeted: @njit a specific hot inner loop when profiling proves it matters, not restructure the whole viewer around compiled types.
Plain composition gives the same decomposition benefit (camera code can't touch terrain state) without the entity-ID indirection, World registry, or Numba type-system complexity.
Current State
All scene state lives as instance variables on InteractiveViewer:
- Camera:
position, yaw, pitch, fov, move_speed, look_speed (engine.py:1290-1308)
- Terrain:
raster, _base_raster, terrain_shape, elev_min/max, subsample_factor, _terrain_mesh_cache (engine.py:1170-1521)
- Observers:
_observers dict[slot → Observer], _active_observer (engine.py:1358-1362)
- Overlays:
_overlay_layers, _active_overlay_data, _overlay_alpha, _terrain_layer_order (engine.py:1225-1239)
- Wind:
_wind_data, _wind_particles, _wind_ages, _wind_trails + GPU buffers (engine.py:1393-1416)
- Geometry layers:
_all_geometries, _geometry_layer_order, _layer_positions (engine.py:1249-1288)
- Rendering:
shadows, ambient, colormap, ao_enabled, dof_enabled, denoise_enabled, etc. (engine.py:1297-1340)
- GUI:
show_help, show_minimap, _help_text_rgba, _title_overlay_rgba (engine.py:1365-1387)
- Input:
_held_keys, _mouse_dragging, _mouse_last_x/y (engine.py:1430-1456)
Each subsystem manages its own state with no formal separation — camera, terrain, overlays, wind, and observers all mutate directly on the viewer instance.
Proposed Design
Subsystem classes (plain Python, no Numba)
class CameraState:
"""Camera transform and movement parameters."""
position: np.ndarray # (3,) float64
yaw: float
pitch: float
fov: float = 60.0
move_speed: float = 1.0
look_speed: float = 1.0
def forward_vector(self) -> np.ndarray: ...
def right_vector(self) -> np.ndarray: ...
def view_matrix(self) -> np.ndarray: ...
def apply_input(self, held_keys: set, dt: float): ...
class TerrainState:
"""Terrain raster data, mesh cache, and resolution tracking."""
raster: xr.DataArray
base_raster: xr.DataArray
elev_min: float
elev_max: float
subsample_factor: int
mesh_cache: dict
mesh_type: str # 'triangulated' | 'voxelated'
def rebuild_at_resolution(self, factor: int, rtx: RTX): ...
def sample_elevation(self, x: float, y: float) -> float: ...
class WindState:
"""Wind particle simulation arrays and GPU buffers."""
positions: np.ndarray
ages: np.ndarray
trails: np.ndarray
speed: float
# GPU buffer handles
def step(self, dt: float): ...
def reset_particles(self): ...
class OverlayManager:
"""Ordered overlay layers with compositing."""
layers: dict[str, np.ndarray]
layer_order: list[str]
active_key: Optional[str]
alpha: float = 0.6
def cycle_overlay(self, direction: int): ...
def composite(self) -> np.ndarray: ...
class ObserverManager:
"""Observer slots, drone mode, viewshed state."""
observers: dict[int, Observer]
active_slot: Optional[int]
def place_observer(self, slot: int, position: np.ndarray): ...
def cycle_drone_mode(self): ...
def update_viewshed(self, rtx: RTX): ...
class GeometryLayerManager:
"""Geometry layer ordering, visibility, and chunk loading."""
layers: dict[str, GeometryLayer]
layer_order: list[str]
active_key: Optional[str]
def cycle_layer(self, direction: int): ...
def load_chunks_near(self, camera_pos: np.ndarray): ...
def unload_distant_chunks(self, camera_pos: np.ndarray): ...
class RenderSettings:
"""Rendering parameters that don't change per-frame."""
shadows: bool = True
ambient: float = 0.3
colormap: str = 'terrain'
ao_enabled: bool = True
dof_enabled: bool = False
denoise_enabled: bool = False
vertical_exaggeration: float = 1.0
class InputState:
"""Raw input tracking — keys, mouse, modifiers."""
held_keys: set
mouse_dragging: bool = False
mouse_last_x: float = 0.0
mouse_last_y: float = 0.0
shift_held: bool = False
class HUDState:
"""Help overlay, minimap, FPS counter, title text."""
show_help: bool = False
show_minimap: bool = True
fps_history: deque
title_text: Optional[str] = None
Composed viewer
class InteractiveViewer:
def __init__(self, ...):
self.camera = CameraState(...)
self.terrain = TerrainState(...)
self.wind = WindState(...)
self.overlays = OverlayManager(...)
self.observers = ObserverManager(...)
self.geometry = GeometryLayerManager(...)
self.render_settings = RenderSettings(...)
self.input = InputState()
self.hud = HUDState()
self.rtx = RTX(...) # low-level OptiX backend (unchanged)
def _tick(self, dt: float):
self.camera.apply_input(self.input.held_keys, dt)
self.geometry.load_chunks_near(self.camera.position)
self.wind.step(dt)
self.observers.update_viewshed(self.rtx)
Selective Numba acceleration (only where profiling justifies)
The subsystem classes are plain Python. If profiling reveals a CPU-bound inner loop, apply @njit to that specific function — no architectural changes needed:
# Example: wind particle update is array-heavy and a natural @njit target
@njit
def _update_wind_particles(positions, velocities, ages, max_age, dt):
for i in range(len(ages)):
ages[i] += dt
if ages[i] > max_age:
ages[i] = 0.0
positions[i] = _random_spawn_position()
else:
positions[i] += velocities[i] * dt
# Example: bilinear terrain sampling called thousands of times for Z-snapping
@njit
def _bilinear_terrain_z(terrain_data, x, y, transform):
...
This is the pattern the Numba extending docs are designed for — targeted acceleration of hot functions — not architectural scaffolding.
Implementation Notes
- Start with CameraState — it has the clearest boundary (position, yaw, pitch, fov, speeds) and the
apply_input / view-matrix logic is self-contained
_tick() decomposes naturally (engine.py:3848-3966) into camera.apply_input() + geometry.load_chunks_near() + wind.step() calls
- Observer already has a class (engine.py:219-259) — wrap the dict-of-observers in
ObserverManager
- Keep
RTX as the low-level backend — subsystems call into it, but don't subclass it
- Extract incrementally — one subsystem at a time, running
explore() after each extraction to verify no regressions
_GASEntry in rtx.py (rtx.py:28-45) already has transform, visibility, handle — these map naturally to GeometryLayerManager internals
- Layer cycling (terrain, basemap, geometry, point cloud) becomes methods on the relevant manager classes rather than a generic "CycleSystem"
Acceptance Criteria
Parent: #57 — Phase 1 Architecture
Summary
Decompose the monolithic
InteractiveViewer(~80 instance variables, 6464 lines inengine.py) into composed subsystem objects. Each subsystem owns its own state and logic, replacing the current flat bag of instance variables with named, self-contained components thatInteractiveViewerholds by reference.Why not ECS?
The original proposal was a full Entity-Component-System architecture (World registry, entity IDs, generic
query()dispatch). After research into Numba's extending API — includingStructRef,@jitclass,@overload, and entry-point registration — we concluded ECS is the wrong pattern here:Entity count is tiny. rtxpy scenes have ~10-20 conceptual entities (1 terrain, 1 camera, a handful of geometry layers, a few observers, 1 wind system, overlays). ECS shines with thousands-to-millions of entities iterated in cache-friendly SoA layout. With <20 entities you always know what exists and access it by name — entity-ID indirection and generic
World.query()add overhead for no benefit.The bottleneck is GPU, not CPU iteration. The hot path is OptiX ray tracing. CPU-side per-frame work (
_tick()) is input processing, chunk loading decisions, and wind particle updates — all trivially fast relative to the ray trace. Numba-compiling ECS systems that iterate components would optimize something that doesn't matter.Systems need Python interop that defeats
@njit. Nearly every "system" calls into Python-only APIs — GLFW key callbacks, OptiX launch, ModernGL texture upload, zarr file I/O, PIL text rendering. You'd hit box/unbox boundaries at every system edge, negating any Numba benefit.Numba extensions are for the compiler, not application architecture. The extending API (
@jitclass,StructRef,@overload) is designed to teach Numba about new types — not to build application frameworks. The right use is targeted:@njita specific hot inner loop when profiling proves it matters, not restructure the whole viewer around compiled types.Plain composition gives the same decomposition benefit (camera code can't touch terrain state) without the entity-ID indirection, World registry, or Numba type-system complexity.
Current State
All scene state lives as instance variables on
InteractiveViewer:position,yaw,pitch,fov,move_speed,look_speed(engine.py:1290-1308)raster,_base_raster,terrain_shape,elev_min/max,subsample_factor,_terrain_mesh_cache(engine.py:1170-1521)_observersdict[slot → Observer],_active_observer(engine.py:1358-1362)_overlay_layers,_active_overlay_data,_overlay_alpha,_terrain_layer_order(engine.py:1225-1239)_wind_data,_wind_particles,_wind_ages,_wind_trails+ GPU buffers (engine.py:1393-1416)_all_geometries,_geometry_layer_order,_layer_positions(engine.py:1249-1288)shadows,ambient,colormap,ao_enabled,dof_enabled,denoise_enabled, etc. (engine.py:1297-1340)show_help,show_minimap,_help_text_rgba,_title_overlay_rgba(engine.py:1365-1387)_held_keys,_mouse_dragging,_mouse_last_x/y(engine.py:1430-1456)Each subsystem manages its own state with no formal separation — camera, terrain, overlays, wind, and observers all mutate directly on the viewer instance.
Proposed Design
Subsystem classes (plain Python, no Numba)
Composed viewer
Selective Numba acceleration (only where profiling justifies)
The subsystem classes are plain Python. If profiling reveals a CPU-bound inner loop, apply
@njitto that specific function — no architectural changes needed:This is the pattern the Numba extending docs are designed for — targeted acceleration of hot functions — not architectural scaffolding.
Implementation Notes
apply_input/ view-matrix logic is self-contained_tick()decomposes naturally (engine.py:3848-3966) intocamera.apply_input()+geometry.load_chunks_near()+wind.step()callsObserverManagerRTXas the low-level backend — subsystems call into it, but don't subclass itexplore()after each extraction to verify no regressions_GASEntryin rtx.py (rtx.py:28-45) already has transform, visibility, handle — these map naturally toGeometryLayerManagerinternalsAcceptance Criteria
CameraStateclass extracted with position, rotation, fov, speeds, andapply_input()/view_matrix()methodsTerrainStateclass extracted with raster data, mesh cache, resolution managementRenderSettingsextracted as a standalone config objectInteractiveViewer.__init__creates composed subsystem objects instead of flat instance variables_tick()delegates to subsystem methodsexplore()functionality preserved — no user-visible behavior change@njit-accelerated inner loop identified by profiling (stretch goal)