vrlizate 1.11.0
vrlizate: ^1.11.0 copied to clipboard
VR 3D engine for Flutter. Scene graph, mesh rendering, lighting, stereoscopic projection, head tracking, interaction, animation, and effects.
Changelog #
1.11.0 — 2026-09-05 #
This release reconciles the package manifest (previously 1.8.0) with the
repository's 1.9.0 and 1.10.0 development notes. Historical headings describe
repository milestones; they do not imply that every version was published to
pub.flutter-io.cn.
Added — Movable 3D Home, Demo Lighting & Hybrid Ray Tracing #
- Added
VrlizateScenequality profiles with hardware-aware sphere tessellation and bounded object-space ray-query budgets. - Added
HybridRayTracerand render-pass integration for direct-light visibility shadows shared by both stereo eyes. - Rebuilt the example Home as movable world-space screens with a 3D frame, floor, lights, Home bar, touch look/recenter, and spatial back navigation.
- Improved every example scene with correct custom-node rendering, lighting, fog, PBR geometry, corrected cinema screen orientation, and lower-frequency radar sampling.
Added — Parent/Child Smartphone Controller MVP #
- Versioned controller profile and sequenced pose protocol with normalized control layouts, quaternion orientation, angular velocity, acceleration, touch/button state, and optional range source/confidence.
- Authenticated local TCP reference transport with pooled-event snapshots, connection state, profile/pose streams, latency measurement, and conditional web stub.
- Child-side
VrRemoteImuControllerwith screen-axis normalization, gyro 3DoF integration, recentering, rate limiting, and latest-frame backpressure. - Parent-side
VrRemoteControllerSessionandVrControllerAvatarNodefor a head-relative visual phone, highlighted controls, stale-frame rejection, and canonical input arbitration. - Added
VrControllerArmModelfor a bounded, smoothed visual hand position derived from orientation, without claiming measured positional tracking. - Added
VrRemoteTouchpadDriverandVrRemotePosePredictorfor normalized touch input and bounded angular prediction using measured transport latency. - Added
SpatialNavigationArrow, a mesh-based upper-left back affordance with gaze/tap interaction, and allocation-conscious transform-copy helpers.
Fixed #
GazePointer.update(dwellEnabled: false)now discards dwell progress while preserving hover and manual input. Re-enabling starts a full dwell interval even when the target has not changed.- Engine tap/dwell picking skips non-interactive geometry and dispatches dwell
activation to the target's
Pointable. - Spatial panels, text, grid floors, radar, and other opt-in custom nodes participate in render sorting and drawing. Specular lighting uses the camera position and treats directional and ambient lights appropriately.
- Replaced the arbiter's out-of-range timestamp sentinel with a JavaScript-safe exact integer so the public exports continue compiling for web/WASM.
Scope and compatibility #
- The core renderer remains Dart/Canvas based. The optional
vrlizate_sceneFlutter GPU adapter is a separate package, not this release's renderer. - Hybrid tracing approximates object-level direct-light visibility against AABBs; it is not per-pixel path tracing or ray-traced reflections.
- The reference native transport uses authenticated, unencrypted JSON over local TCP. BLE/Wi-Fi Direct are extension points, not implemented transports.
- Pooled input events avoid event-object churn; JSON, snapshots, scene updates, and rendering are not an end-to-end Zero-GC pipeline.
1.10.0 — 2026-09-03 #
Added — Zero-GC Multimodal Input Arbiter, Reference Gamepad Driver & Open-Source Spec #
- Zero-GC Multimodal Input Arbiter (
VrInputArbiter):- Preallocated fixed ring buffer pool (64 slots) with synchronous borrow and release semantics ensuring zero garbage collection pressure during 60–120 Hz update loops.
- Strict priority arbitration:
touch/remotePhone(maximum) >gamepad/sonar/cameraHand(medium) >gaze(base). - Configurable 400ms hysteresis cooldown suppressing base gaze selection while external active inputs are manipulated.
- Debug-mode leak detection (
assertNoLeaks). - Extensible
VrInputDriverandVrInputSinkabstractions for decoupled community hardware drivers.
- Reference Gamepad Driver (
VrGamepadDriver):- Standardized hardware-agnostic adapter mapping A/B/X/Y buttons, L2/R2 analog triggers, and analog thumbsticks to canonical
VrInputTypeevents. - Configurable radial deadzones for analog sticks (
stickDeadzone = 0.15) and triggers (triggerDeadzone = 0.05). - Reusable payload maps avoiding per-frame map allocations.
- Standardized hardware-agnostic adapter mapping A/B/X/Y buttons, L2/R2 analog triggers, and analog thumbsticks to canonical
- Reference 3DoF Laser Pointer Driver (
VrLaserPointerDriver):- Standardized spatial pointing ray driver for 2nd smartphone remotes and 3DoF controllers.
- Computes forward pointing vector
orientation.rotate(Vector3(0, 0, -1))and emitsVrInputType.pointerMove. - Maps trigger press/release to canonical
VrInputType.trigger. - Dynamic
recenter()establishing the forward yaw reference pose without hardware drift. - Priority overrides via
VrInputPrioritizedDriverwithmediumarbitration.
- Progressive Smartphone VR Open-Source Architecture:
- Bilingual
ARCHITECTURE.md,CONTRIBUTING.md, and updatedREADME.mddefining Progressive Hardware Tiers (Lite, Standard, Pro) for mobile VR democratization.
- Bilingual
Fixed #
- Optics & Tracking Stability:
- Fixed
CameraRigidentity matrix reset and IPD bounding. - Corrected bias drift in
HeadTracker.
- Fixed
1.9.0 — 2026-08-27 #
Added — VR Ergonomics, Horizon Stabilization & Sensor Fusion #
- Zero-Roll Horizon Stabilization (
CameraRig): Orientation is now calculated as decoupled $Q_{\text{Yaw}}(\text{World } Y) \times Q_{\text{Pitch}}(\text{Local } X)$ enforcing $\text{Roll} = 0^\circ$. Prevents sideways floor tilting and eliminates cumulative gyroscope roll drift on mobile viewers. - Calibrated Landscape Gyroscope Fusion (
HeadTracker): Exact sensor axis mapping for horizontal VR viewers (DeviceOrientation.landscapeLeft). Yaw (left/right) and pitch (up/down) track physical head movements 1:1 with accelerometer gravity anchoring $\text{pitch} = \text{atan2}(\text{accelZ}, \text{accelX})$. - Ergonomic Pitch Gain (
HeadTracker.pitchGain = 1.25): Gentle $\pm 40^\circ$ neck tilts smoothly reach the full vertical field (ground/feet at $-80^\circ$ and ceiling/zenith at $+80^\circ$) without cervical strain. - Sensor Jitter Damping (
HeadTracker.jitterDamping): Deadband threshold ($<0.00015\text{ rad}$) and fast exponential low-pass filter ($\alpha = 0.85$) eliminate IMU thermal noise and hand micro-tremors without adding latency. - Hands-Free & One-Tap Recenter (
recenter()): Addedrecenter()toRotationTarget,CameraRig,HeadTracker, andDesktopInputDriver(bound to key 'R') to realign the forward heading ($0^\circ$) instantly. - Micro-Haptic Feedback (
GazePointer.enableHaptics): Subtle tactile vibration feedback on gaze hover enter (selectionClick), dwell selection (lightImpact), and manual tap/recenter (mediumImpact). - VAC (Vergence-Accommodation Conflict) Comfort Constants (
CameraRig): StandardizedcomfortDistanceMin($1.2\text{ m}$),comfortDistanceDefault($1.8\text{ m}$), andcomfortDistanceMax($3.0\text{ m}$) for spatial UI depth. - IPD Calibration & Bounds (
CameraRig): StandardizedminIpd($50\text{ mm}$),defaultIpd($64\text{ mm}$), andmaxIpd($80\text{ mm}$) with automatic clamping. - Desktop Snap Turning (
DesktopInputDriver): Discrete $30^\circ$ snap turns using keyboard arrow keys for anti-motion-sickness navigation.
1.8.0 — 2026-08-14 #
Added — Joystick-Free Input Stack (5 channels, all optional) #
- Adaptive Gaze Dwell (
GazePointer): repeated selections of the same target accelerate dwell (configurabledwellAcceleration, clamped atminDwellDuration);gazeGracePeriodtolerates brief gaze slips without losing progress;onDwellProgresscallback; richer reticle with white→accent progress ring, quarter tick marks, hover halo, and selection flash;resetAdaptation()for scene changes. - Walk-in-Place Locomotion (
WalkInPlaceDetector+WalkInPlaceLocomotion): VR-Step-style step detection from the accelerometer — runtime gravity estimation (orientation-agnostic), high-pass vertical bob signal, peak detection with refractory period, cadence tracking (steps/min) over a sliding window. Locomotion maps cadence to head-relative ground-plane velocity with smooth decay. Move through virtual spaces inside a closed viewer with no joystick. - Camera Hand-Tracking Pipeline (
CameraHandTrackingDriver): connects real camera ML frames (HandLandmarkFramefrom MediaPipe Hand Landmarker / ML Kit, fed by the host app) to the existing 26-joint OpenXRHandState— handedness routing, normalized→metric reprojection into a configurable tracking volume, per-landmark One Euro filtering (jitter-free when still, lag-free when fast), confidence gating, and tracking-loss timeout. - Hand Gesture Recognition (
HandGestureRecognizer): debounced semantic gestures fromHandStateposes — pinch start/end edges (the VR "click"), fist (back), flat hand (stop), point (aim ray), thumbs up (confirm), victory — with configurableminHoldDurationto suppress single-frame ML false positives. - Ultrasonic Gesture Channel (
UltrasonicGestureChannel): FingerIO/LLAP/AudioGest-family active sonar — pure-Dart radix-2 FFT with Hann windowing, sideband Doppler imbalance analysis around a 17–20 kHz carrier, adaptive noise floor, presence (hover enter/exit) and motion (push/pull) gestures with cooldown. Works in the dark with the headset cover closed. Host app plays the tone fromgenerateTone()and feeds microphone PCM. - Wi-Fi RTT Room-Scale Positioning (
WifiRttPositioning): IEEE 802.11mc trilateration via weighted Gauss-Newton least squares — anchor registry, freshest-measurement-per-anchor windowing, outlier rejection with re-solve, stdDev-based weighting, and smoothed position stream. Uses the public Android 9+ RTT API (host app feeds measurements); a portable alternative to CSI-based sensing, which requires rooted firmware. InputFusion: unifies all channels into oneVrInputActionstream (SelectAction,BackAction,AimAction,MoveAction,ConfirmAction) with priority resolution — hand pinch supersedes in-progress gaze dwell, ultrasonic push maps to select/confirm, pull to back, walk-in-place cadence to continuous movement — plus cross-channel select cooldown.
Compatibility #
- All new drivers are transport-agnostic pure Dart: no new plugin dependencies; camera, microphone, speaker, and Wi-Fi RTT remain host-app responsibilities, keeping the package WASM-ready and platform 6/6.
1.7.0 — 2026-08-14 #
Added #
- Desktop Input Driver (
DesktopInputDriver): First-class desktop support for macOS/Windows/Linux targets — mouse-look (drag, same sign convention as touch fallback, optionalinvertY), WASD + Q/E locomotion with camera-relative movement and Shift sprint multiplier, rebindable keys, and focus-loss key release. - Mouse 3D Picking:
screenPointToRay()unprojects any screen point through the inverse view-projection into a world ray, andpick()raycasts the scene — the desktop equivalent ofGazePointer. DesktopInputRegionwidget: drop-in wrapper that forwards pointer and keyboard events to the driver, with click-to-pick and hover-ray callbacks.VREngine.enableDesktopInput(): one-call integration; locomotion updates run automatically in the engine tick.
Fixed #
- WASM Compatibility:
dart:isolateis now behind a conditional import (BackgroundIsolate— real implementation on VM/native, no-op stub on web/WASM). The head-tracking sensor fusion and WiFi CSI processing isolate entry points moved to the platform-specific library. The package is now fully WASM-ready, achieving platform support 6/6 on pub.flutter-io.cn. - Group Node AABB Culling Bug (Raycast Silently Broken):
Node.worldAabbfor group nodes was a phantom ±0.5m box at the node's position that did NOT include children. TheRaycaster's broad-phase pruned entire subtrees when the ray missed that box — any node farther than 0.5m from its parent's origin could never be raycast (gaze selection, mouse picking).worldAabbis now a cached union of the node's own bounds and all descendants (invalidated upward on any transform/hierarchy change), while self-hits, physics, and frustum culling use the newownWorldAabb(own geometry only, preserving previous semantics).
1.6.0 — 2026-08-06 #
Fixed #
Transform3D.lookAtVertical Axis Flip (Black Screen Bug): Removed an erroneous rotation-matrix transpose —makeViewMatrixalready stores basis vectors as columns, so transposing flipped the Y component of the camera forward vector. Cameras looking steeply down (e.g. board-game and top-down scenes) ended up looking up, causing the entire scene to be frustum-culled and rendering a black screen. Added a regression test asserting the full forward vector for look-down cameras.- Head Tracking Scale (33× Attenuation): Gyroscope deltas were already integrated radians, but the default
sensitivity = 0.03scaled them down to 3% of real head motion — tracking appeared dead and only touch-drag worked. Default sensitivity is now1.0(true 1:1 tracking) inHeadTracker,HeadTracker.forCamera, andVREngine.enableHeadTracking.
Changed #
- Yaw Channel Now Gyro-Only (Drift-Free Turns): The yaw (device-X in landscape) channel no longer anchors to
atan2(accelY, accelZ), which is undefined noise when the phone is held level in a viewer and pulled the view back after head turns. Yaw now integrates the gyroscope directly (bias handled by the existing anti-drift auto-calibration); pitch keeps the gravity-anchored complementary filter. Applies to both the background-Isolate and main-thread fallback paths. - Removed ×1.8 Vertical Amplification: With correct 1:1 scaling, amplified pitch is no longer needed and was disorienting; pitch is now also 1:1.
Migration Notes #
- If your app passed an explicit
sensitivityvalue tuned for the old broken scale (e.g.0.03), remove it or retune around1.0= 1:1.
1.5.0 — 2026-07-21 #
Added #
- Real UV Texture Mapping (
ImageShader): ConnectedVRTexturemap to vertex UV coordinates withImageShaderandBlendMode.modulate, rendering textured 3D meshes on CPU/Canvas buffers. - PBR Specular Reflection Highlights: Integrated Blinn-Phong & Cook-Torrance specular highlights using half-vector specular terms modulated by material
metallicandroughness. - Google Cardboard Protobuf Base64 QR Decoder: Implemented a pure-Dart Varint reader in
DeviceParams.fromCardboardQrUridecoding official Google Cardboard QR configuration URIs (?p=...). - FaceTrackerDriver (Looking-Glass Holographic 3D Window): Added live 3D face position tracking driver feeding head offsets to off-axis asymmetric projection matrices in
CameraRig. - MediaPipe 21-Landmark Hand Converter (
MediaPipeHandDriver): Added 21-landmark MediaPipe tracking converter mapping camera hand inputs into 26 OpenXR joint structures. - Anaglyph Red/Cyan 3D Stereoscopic Painter (
VRAnaglyphPainter): Added stereoscopic 3D rendering mode using red/cyan channels for 3D viewing without Cardboard viewers. - Stable Shader Uniform Index Mapping: Fixed Flutter custom shader uniform index resolution to maintain sequential positional mappings instead of hash collisions.
- GPU Instanced Mesh Rendering Signatures (
GPURenderer): AddeddrawInstancedandbindInstancedTransformBuffersignatures for hardware-accelerated instanced render passes.
Fixed #
- Dynamic Viewport Scanline Projections: Updated
HologramMeshNodescanline rendering to receive dynamic canvas viewport dimensions.
1.4.0 — 2026-07-02 #
Added #
- Asynchronous Time Warp (ATW): Implemented GPU-based frame prediction and rotation warp cache. When frame rendering times drop below 60fps (>18ms), the engine automatically reuses the previous frame's eye buffers and skews them using the latest head-tracking delta rotation matrix on the GPU, preventing VR motion sickness.
- Chromatic Aberration Correction: Added software-based radial dispersion correction inside the distortion mesh. Splits the image render into Red (1.008x scale), Green (1.0x), and Blue (0.992x) channels with
BlendMode.plusto neutralize cheap plastic lens aberration. - FSR (FidelityFX Super Resolution) Mobile Scaling: Added dynamic resolution scaling (
fsrScalefield inRenderPass). Viewports render to a lower density offline frame-buffer and scale back to screen resolution with bilinear interpolation and sharpening, saving 40%+ GPU/battery draw. - Infinite Resolution Vector Font Rendering (SDF Simulation): Upgraded
SpatialTextto rasterize glyphs at a high-res base size (120px) and scale down the Canvas transform, generating ultra-dense vector contours that stay perfectly sharp under magnifying lenses. - Fixed Foveated Rendering (FFR): Enabled viewport frustum zoning to skip heavy lighting and mesh calculations for objects situated in the peripheral field of view.
- Cardboard Trigger Tap & Pointer Integration: Integrated tap events inside
VREngine.handleTap()andGazePointer. Allows physical screen clicks to trigger press/release and hover actions on interactivePointablenodes (e.g.SpatialButton) at the cursor gaze position. - Anti-Drift Calibration Filter: Added continuous low-pass drift adjustment to head-tracking gyroscope sensor fusion. Automatically recalibrates sensor bias when rotational speeds fall below
0.015 rad/s.
Fixed #
- glTF Benchmark Tolerance: Adjusted timing thresholds in GLTF parsing stress tests to prevent false negatives under high CPU loads.
1.3.0 — 2026-05-27 #
Added #
- OpenXR 26-Joint Skeletal Alignment & Gestures: Standardized input layer to the Khronos OpenXR specification, supporting physical bone linkages (25 bone joints), 3D orientations, and advanced mathematical gesture calculations (
isFlatHand,isThumbsUp,isVictory). - Projected Radial Lens Distortion: Implemented polynomial barrel distortion directly in the Normalized Device Coordinates (NDC) projection of the vertex shader pipeline, enabling cardboard VR distortion correction with zero CPU/GPU rendering overhead.
- Isolate-Based Background Sensor Fusion: Decoupled complementary sensor filters (IMU gyroscope and accelerometer math) and predictive head tracking calculations from the UI thread to a dedicated background Isolate thread, avoiding frame rate drops.
- Volumetric Hologram Mesh Node (
HologramMeshNode): Added a simulated volumetric shader employing a triple-pass shell technique (Core, body with flickering and glitch offsets, and glowing envelope) along with vertical scanline effects. - Isolate-Based Background WiFi Sensing Engine (
WifiSensingSystem): Added background Isolate support to process OFDM Channel State Information (CSI) subcarriers, computing multipath disturbance standard deviation to track physical trajectories and vitals (respiration rate) without optical cameras. - 3D Skeletal Bone Cylinder Rendering: Renders full 3D skeletons between parent/child joints dynamically using quaternion orientation rotations.
1.2.0 — 2026-05 #
Changed #
- Refactored
VRParticleandVRRingclasses to makecolora non-final mutable field to enable real-time dynamic modulation for audio-reactive and visual animation effects.
1.1.0 — 2026-05 #
Added #
- IMU Complementary Sensor Fusion: Stabilized landscape VR tracking by combining high-frequency gyroscope integration with low-frequency absolute gravity accelerometer vectors ($\alpha = 0.98$) to eliminate head tracking drift.
- Google Cardboard Neck Model: Implemented realistic head-to-neck physical pivot rotations for camera projection, delivering comfortable stereoscopic parallax and dramatically reducing VR motion sickness.
- Turbo-Canvas Render Engine Optimization: Shifted flat-shaded face rendering to use Flutter's ultra-high-performance
canvas.drawVerticeswithui.Verticesbatched triangulation, replacing slow iterative path drawing with direct GPU-friendly vertex buffers. - Injectable Sensor Streams: Refactored
HeadTrackerconstructor to accept optional mock streams (gyroscopeStreamOverride,accelerometerStreamOverride), enabling complete deterministic offline simulation and testing of IMU inputs. - Expanded Test Suite: Added comprehensive unit tests validating complementary filter math stability, noisy drift calibration offsets, symmetric eye offsets, and physical neck model boundaries (170 tests passing at 100%).
- OSI Apache-2.0 License compliance: Replaced generic license templates with the official Apache 2.0 license file to recover full pub.flutter-io.cn score points under file conventions.
Fixed #
- Static Analysis warnings: Resolved all 14 lint warnings and info items (e.g. deprecated
.redusage, unused imports/variables, non-const declarations, and package dependency rules) achieving 0 issues across the engine and example. - Dependency boundaries updated: Raised
sensors_plusconstraint to^7.0.0and optimizedvector_mathto^2.2.0to support modern platforms while avoiding SDK version solver pins.
Future Roadmap & Surrounding Integrations #
- Unified Locomotion Driver: Incorporate touch joysticks from neighboring 3D engines (e.g.
openworlddart) to combine with the built-in accelerometer physical step-locomotion peak detector. - OpenXR Joint Mapping Alignment: Standardize the 26-joint tracking framework inside
HandStateto match Khronos OpenXR specifications for future FFI driver compatibility. - Cardboard QR Profile Loader: Implement pure-Dart base64 protocol buffer profile decoders to parse QR codes for custom lens radial distortion models dynamically.
- Aesthetic "WOW" Factor Expansion: Build togglable PBR Physics Playgrounds (interacting with metallic reflectives and grabbing objects with dynamic impulse responses) and Deep Space Flight Simulators directly within the interactive sample app.
1.0.0 — 2026-04 #
Added #
- VR Celestial Compass example: wireframe globe, cardinal markers, SpatialText labels, zone detection, step locomotion
analysis_options.yamlwith Flutter lint rulesexample/app scaffold (Android/iOS/macOS/Web/Linux/Windows) with pub.flutter-io.cn example.dartissue_trackerandplatformsfields in pubspec.yaml- Cover image in README and pub.flutter-io.cn screenshots
- Zone detection system based on Meta VR research (30°/55°/135° thresholds)
- Step detection via accelerometer peak detection
Fixed #
- Head tracking: correct landscape gyroscope axis mapping (X→yaw, Y→pitch)
- Head tracking: inverted horizontal direction fixed
- Head tracking: vertical sensitivity amplified ×1.8 for comfortable up/down range
- LitMeshNode:
lightslist now mutable (wasconst [], caused runtime crash) - Deprecated
Color.red/green/blue/alphaAPI replaced withColor.r/g/b/a - Removed unnecessary
dart:typed_dataimport in texture.dart - Fixed dangling library doc comment in webxr_session.dart
- All 36 analyzer warnings and hints resolved (0 issues)
- Dart format applied to all 59 source files
Changed #
- Version 1.0.0 — first stable release
- README Quick Start uses pub.flutter-io.cn dependency
0.1.0 — 2026-04 #
Added #
- Initial VR engine: 59 Dart source files
- Scene graph with hierarchical nodes and transforms
- Mesh rendering with Phong + PBR materials
- Stereoscopic side-by-side projection
- Head tracking via gyroscope (sensors_plus)
- Interaction: gaze pointer, raycast, grabbable, pointable
- Locomotion: fly, teleport, walk
- Animation: clips, mixer, keyframes, skeleton, skin deformer
- Effects: bloom, fog, SSAO, vignette, lens distortion
- Primitive geometries: cube, sphere, plane, cylinder
- Basic glTF 2.0 parser
- Spatial UI: billboard, panel, button, text
- WebXR session bridge
- Debug overlay and grid floor
- Physics: rigid body, AABB collision, gravity
- 163 tests with benchmarks
- 5 demo examples: VR Gallery, Treasure Hunt, Open World, Spatial UI, Hand Tracking
