vrlizate 1.11.0 copy "vrlizate: ^1.11.0" to clipboard
vrlizate: ^1.11.0 copied to clipboard

VR 3D engine for Flutter. Scene graph, mesh rendering, lighting, stereoscopic projection, head tracking, interaction, animation, and effects.

Changelog #

1.11.0 — 2026-09-05 #

This release reconciles the package manifest (previously 1.8.0) with the repository's 1.9.0 and 1.10.0 development notes. Historical headings describe repository milestones; they do not imply that every version was published to pub.flutter-io.cn.

Added — Movable 3D Home, Demo Lighting & Hybrid Ray Tracing #

  • Added VrlizateScene quality profiles with hardware-aware sphere tessellation and bounded object-space ray-query budgets.
  • Added HybridRayTracer and render-pass integration for direct-light visibility shadows shared by both stereo eyes.
  • Rebuilt the example Home as movable world-space screens with a 3D frame, floor, lights, Home bar, touch look/recenter, and spatial back navigation.
  • Improved every example scene with correct custom-node rendering, lighting, fog, PBR geometry, corrected cinema screen orientation, and lower-frequency radar sampling.

Added — Parent/Child Smartphone Controller MVP #

  • Versioned controller profile and sequenced pose protocol with normalized control layouts, quaternion orientation, angular velocity, acceleration, touch/button state, and optional range source/confidence.
  • Authenticated local TCP reference transport with pooled-event snapshots, connection state, profile/pose streams, latency measurement, and conditional web stub.
  • Child-side VrRemoteImuController with screen-axis normalization, gyro 3DoF integration, recentering, rate limiting, and latest-frame backpressure.
  • Parent-side VrRemoteControllerSession and VrControllerAvatarNode for a head-relative visual phone, highlighted controls, stale-frame rejection, and canonical input arbitration.
  • Added VrControllerArmModel for a bounded, smoothed visual hand position derived from orientation, without claiming measured positional tracking.
  • Added VrRemoteTouchpadDriver and VrRemotePosePredictor for normalized touch input and bounded angular prediction using measured transport latency.
  • Added SpatialNavigationArrow, a mesh-based upper-left back affordance with gaze/tap interaction, and allocation-conscious transform-copy helpers.

Fixed #

  • GazePointer.update(dwellEnabled: false) now discards dwell progress while preserving hover and manual input. Re-enabling starts a full dwell interval even when the target has not changed.
  • Engine tap/dwell picking skips non-interactive geometry and dispatches dwell activation to the target's Pointable.
  • Spatial panels, text, grid floors, radar, and other opt-in custom nodes participate in render sorting and drawing. Specular lighting uses the camera position and treats directional and ambient lights appropriately.
  • Replaced the arbiter's out-of-range timestamp sentinel with a JavaScript-safe exact integer so the public exports continue compiling for web/WASM.

Scope and compatibility #

  • The core renderer remains Dart/Canvas based. The optional vrlizate_scene Flutter GPU adapter is a separate package, not this release's renderer.
  • Hybrid tracing approximates object-level direct-light visibility against AABBs; it is not per-pixel path tracing or ray-traced reflections.
  • The reference native transport uses authenticated, unencrypted JSON over local TCP. BLE/Wi-Fi Direct are extension points, not implemented transports.
  • Pooled input events avoid event-object churn; JSON, snapshots, scene updates, and rendering are not an end-to-end Zero-GC pipeline.

1.10.0 — 2026-09-03 #

Added — Zero-GC Multimodal Input Arbiter, Reference Gamepad Driver & Open-Source Spec #

  • Zero-GC Multimodal Input Arbiter (VrInputArbiter):
    • Preallocated fixed ring buffer pool (64 slots) with synchronous borrow and release semantics ensuring zero garbage collection pressure during 60–120 Hz update loops.
    • Strict priority arbitration: touch / remotePhone (maximum) > gamepad / sonar / cameraHand (medium) > gaze (base).
    • Configurable 400ms hysteresis cooldown suppressing base gaze selection while external active inputs are manipulated.
    • Debug-mode leak detection (assertNoLeaks).
    • Extensible VrInputDriver and VrInputSink abstractions for decoupled community hardware drivers.
  • Reference Gamepad Driver (VrGamepadDriver):
    • Standardized hardware-agnostic adapter mapping A/B/X/Y buttons, L2/R2 analog triggers, and analog thumbsticks to canonical VrInputType events.
    • Configurable radial deadzones for analog sticks (stickDeadzone = 0.15) and triggers (triggerDeadzone = 0.05).
    • Reusable payload maps avoiding per-frame map allocations.
  • Reference 3DoF Laser Pointer Driver (VrLaserPointerDriver):
    • Standardized spatial pointing ray driver for 2nd smartphone remotes and 3DoF controllers.
    • Computes forward pointing vector orientation.rotate(Vector3(0, 0, -1)) and emits VrInputType.pointerMove.
    • Maps trigger press/release to canonical VrInputType.trigger.
    • Dynamic recenter() establishing the forward yaw reference pose without hardware drift.
    • Priority overrides via VrInputPrioritizedDriver with medium arbitration.
  • Progressive Smartphone VR Open-Source Architecture:
    • Bilingual ARCHITECTURE.md, CONTRIBUTING.md, and updated README.md defining Progressive Hardware Tiers (Lite, Standard, Pro) for mobile VR democratization.

Fixed #

  • Optics & Tracking Stability:
    • Fixed CameraRig identity matrix reset and IPD bounding.
    • Corrected bias drift in HeadTracker.

1.9.0 — 2026-08-27 #

Added — VR Ergonomics, Horizon Stabilization & Sensor Fusion #

  • Zero-Roll Horizon Stabilization (CameraRig): Orientation is now calculated as decoupled $Q_{\text{Yaw}}(\text{World } Y) \times Q_{\text{Pitch}}(\text{Local } X)$ enforcing $\text{Roll} = 0^\circ$. Prevents sideways floor tilting and eliminates cumulative gyroscope roll drift on mobile viewers.
  • Calibrated Landscape Gyroscope Fusion (HeadTracker): Exact sensor axis mapping for horizontal VR viewers (DeviceOrientation.landscapeLeft). Yaw (left/right) and pitch (up/down) track physical head movements 1:1 with accelerometer gravity anchoring $\text{pitch} = \text{atan2}(\text{accelZ}, \text{accelX})$.
  • Ergonomic Pitch Gain (HeadTracker.pitchGain = 1.25): Gentle $\pm 40^\circ$ neck tilts smoothly reach the full vertical field (ground/feet at $-80^\circ$ and ceiling/zenith at $+80^\circ$) without cervical strain.
  • Sensor Jitter Damping (HeadTracker.jitterDamping): Deadband threshold ($<0.00015\text{ rad}$) and fast exponential low-pass filter ($\alpha = 0.85$) eliminate IMU thermal noise and hand micro-tremors without adding latency.
  • Hands-Free & One-Tap Recenter (recenter()): Added recenter() to RotationTarget, CameraRig, HeadTracker, and DesktopInputDriver (bound to key 'R') to realign the forward heading ($0^\circ$) instantly.
  • Micro-Haptic Feedback (GazePointer.enableHaptics): Subtle tactile vibration feedback on gaze hover enter (selectionClick), dwell selection (lightImpact), and manual tap/recenter (mediumImpact).
  • VAC (Vergence-Accommodation Conflict) Comfort Constants (CameraRig): Standardized comfortDistanceMin ($1.2\text{ m}$), comfortDistanceDefault ($1.8\text{ m}$), and comfortDistanceMax ($3.0\text{ m}$) for spatial UI depth.
  • IPD Calibration & Bounds (CameraRig): Standardized minIpd ($50\text{ mm}$), defaultIpd ($64\text{ mm}$), and maxIpd ($80\text{ mm}$) with automatic clamping.
  • Desktop Snap Turning (DesktopInputDriver): Discrete $30^\circ$ snap turns using keyboard arrow keys for anti-motion-sickness navigation.

1.8.0 — 2026-08-14 #

Added — Joystick-Free Input Stack (5 channels, all optional) #

  • Adaptive Gaze Dwell (GazePointer): repeated selections of the same target accelerate dwell (configurable dwellAcceleration, clamped at minDwellDuration); gazeGracePeriod tolerates brief gaze slips without losing progress; onDwellProgress callback; richer reticle with white→accent progress ring, quarter tick marks, hover halo, and selection flash; resetAdaptation() for scene changes.
  • Walk-in-Place Locomotion (WalkInPlaceDetector + WalkInPlaceLocomotion): VR-Step-style step detection from the accelerometer — runtime gravity estimation (orientation-agnostic), high-pass vertical bob signal, peak detection with refractory period, cadence tracking (steps/min) over a sliding window. Locomotion maps cadence to head-relative ground-plane velocity with smooth decay. Move through virtual spaces inside a closed viewer with no joystick.
  • Camera Hand-Tracking Pipeline (CameraHandTrackingDriver): connects real camera ML frames (HandLandmarkFrame from MediaPipe Hand Landmarker / ML Kit, fed by the host app) to the existing 26-joint OpenXR HandState — handedness routing, normalized→metric reprojection into a configurable tracking volume, per-landmark One Euro filtering (jitter-free when still, lag-free when fast), confidence gating, and tracking-loss timeout.
  • Hand Gesture Recognition (HandGestureRecognizer): debounced semantic gestures from HandState poses — pinch start/end edges (the VR "click"), fist (back), flat hand (stop), point (aim ray), thumbs up (confirm), victory — with configurable minHoldDuration to suppress single-frame ML false positives.
  • Ultrasonic Gesture Channel (UltrasonicGestureChannel): FingerIO/LLAP/AudioGest-family active sonar — pure-Dart radix-2 FFT with Hann windowing, sideband Doppler imbalance analysis around a 17–20 kHz carrier, adaptive noise floor, presence (hover enter/exit) and motion (push/pull) gestures with cooldown. Works in the dark with the headset cover closed. Host app plays the tone from generateTone() and feeds microphone PCM.
  • Wi-Fi RTT Room-Scale Positioning (WifiRttPositioning): IEEE 802.11mc trilateration via weighted Gauss-Newton least squares — anchor registry, freshest-measurement-per-anchor windowing, outlier rejection with re-solve, stdDev-based weighting, and smoothed position stream. Uses the public Android 9+ RTT API (host app feeds measurements); a portable alternative to CSI-based sensing, which requires rooted firmware.
  • InputFusion: unifies all channels into one VrInputAction stream (SelectAction, BackAction, AimAction, MoveAction, ConfirmAction) with priority resolution — hand pinch supersedes in-progress gaze dwell, ultrasonic push maps to select/confirm, pull to back, walk-in-place cadence to continuous movement — plus cross-channel select cooldown.

Compatibility #

  • All new drivers are transport-agnostic pure Dart: no new plugin dependencies; camera, microphone, speaker, and Wi-Fi RTT remain host-app responsibilities, keeping the package WASM-ready and platform 6/6.

1.7.0 — 2026-08-14 #

Added #

  • Desktop Input Driver (DesktopInputDriver): First-class desktop support for macOS/Windows/Linux targets — mouse-look (drag, same sign convention as touch fallback, optional invertY), WASD + Q/E locomotion with camera-relative movement and Shift sprint multiplier, rebindable keys, and focus-loss key release.
  • Mouse 3D Picking: screenPointToRay() unprojects any screen point through the inverse view-projection into a world ray, and pick() raycasts the scene — the desktop equivalent of GazePointer.
  • DesktopInputRegion widget: drop-in wrapper that forwards pointer and keyboard events to the driver, with click-to-pick and hover-ray callbacks.
  • VREngine.enableDesktopInput(): one-call integration; locomotion updates run automatically in the engine tick.

Fixed #

  • WASM Compatibility: dart:isolate is now behind a conditional import (BackgroundIsolate — real implementation on VM/native, no-op stub on web/WASM). The head-tracking sensor fusion and WiFi CSI processing isolate entry points moved to the platform-specific library. The package is now fully WASM-ready, achieving platform support 6/6 on pub.flutter-io.cn.
  • Group Node AABB Culling Bug (Raycast Silently Broken): Node.worldAabb for group nodes was a phantom ±0.5m box at the node's position that did NOT include children. The Raycaster's broad-phase pruned entire subtrees when the ray missed that box — any node farther than 0.5m from its parent's origin could never be raycast (gaze selection, mouse picking). worldAabb is now a cached union of the node's own bounds and all descendants (invalidated upward on any transform/hierarchy change), while self-hits, physics, and frustum culling use the new ownWorldAabb (own geometry only, preserving previous semantics).

1.6.0 — 2026-08-06 #

Fixed #

  • Transform3D.lookAt Vertical Axis Flip (Black Screen Bug): Removed an erroneous rotation-matrix transpose — makeViewMatrix already stores basis vectors as columns, so transposing flipped the Y component of the camera forward vector. Cameras looking steeply down (e.g. board-game and top-down scenes) ended up looking up, causing the entire scene to be frustum-culled and rendering a black screen. Added a regression test asserting the full forward vector for look-down cameras.
  • Head Tracking Scale (33× Attenuation): Gyroscope deltas were already integrated radians, but the default sensitivity = 0.03 scaled them down to 3% of real head motion — tracking appeared dead and only touch-drag worked. Default sensitivity is now 1.0 (true 1:1 tracking) in HeadTracker, HeadTracker.forCamera, and VREngine.enableHeadTracking.

Changed #

  • Yaw Channel Now Gyro-Only (Drift-Free Turns): The yaw (device-X in landscape) channel no longer anchors to atan2(accelY, accelZ), which is undefined noise when the phone is held level in a viewer and pulled the view back after head turns. Yaw now integrates the gyroscope directly (bias handled by the existing anti-drift auto-calibration); pitch keeps the gravity-anchored complementary filter. Applies to both the background-Isolate and main-thread fallback paths.
  • Removed ×1.8 Vertical Amplification: With correct 1:1 scaling, amplified pitch is no longer needed and was disorienting; pitch is now also 1:1.

Migration Notes #

  • If your app passed an explicit sensitivity value tuned for the old broken scale (e.g. 0.03), remove it or retune around 1.0 = 1:1.

1.5.0 — 2026-07-21 #

Added #

  • Real UV Texture Mapping (ImageShader): Connected VRTexture map to vertex UV coordinates with ImageShader and BlendMode.modulate, rendering textured 3D meshes on CPU/Canvas buffers.
  • PBR Specular Reflection Highlights: Integrated Blinn-Phong & Cook-Torrance specular highlights using half-vector specular terms modulated by material metallic and roughness.
  • Google Cardboard Protobuf Base64 QR Decoder: Implemented a pure-Dart Varint reader in DeviceParams.fromCardboardQrUri decoding official Google Cardboard QR configuration URIs (?p=...).
  • FaceTrackerDriver (Looking-Glass Holographic 3D Window): Added live 3D face position tracking driver feeding head offsets to off-axis asymmetric projection matrices in CameraRig.
  • MediaPipe 21-Landmark Hand Converter (MediaPipeHandDriver): Added 21-landmark MediaPipe tracking converter mapping camera hand inputs into 26 OpenXR joint structures.
  • Anaglyph Red/Cyan 3D Stereoscopic Painter (VRAnaglyphPainter): Added stereoscopic 3D rendering mode using red/cyan channels for 3D viewing without Cardboard viewers.
  • Stable Shader Uniform Index Mapping: Fixed Flutter custom shader uniform index resolution to maintain sequential positional mappings instead of hash collisions.
  • GPU Instanced Mesh Rendering Signatures (GPURenderer): Added drawInstanced and bindInstancedTransformBuffer signatures for hardware-accelerated instanced render passes.

Fixed #

  • Dynamic Viewport Scanline Projections: Updated HologramMeshNode scanline rendering to receive dynamic canvas viewport dimensions.

1.4.0 — 2026-07-02 #

Added #

  • Asynchronous Time Warp (ATW): Implemented GPU-based frame prediction and rotation warp cache. When frame rendering times drop below 60fps (>18ms), the engine automatically reuses the previous frame's eye buffers and skews them using the latest head-tracking delta rotation matrix on the GPU, preventing VR motion sickness.
  • Chromatic Aberration Correction: Added software-based radial dispersion correction inside the distortion mesh. Splits the image render into Red (1.008x scale), Green (1.0x), and Blue (0.992x) channels with BlendMode.plus to neutralize cheap plastic lens aberration.
  • FSR (FidelityFX Super Resolution) Mobile Scaling: Added dynamic resolution scaling (fsrScale field in RenderPass). Viewports render to a lower density offline frame-buffer and scale back to screen resolution with bilinear interpolation and sharpening, saving 40%+ GPU/battery draw.
  • Infinite Resolution Vector Font Rendering (SDF Simulation): Upgraded SpatialText to rasterize glyphs at a high-res base size (120px) and scale down the Canvas transform, generating ultra-dense vector contours that stay perfectly sharp under magnifying lenses.
  • Fixed Foveated Rendering (FFR): Enabled viewport frustum zoning to skip heavy lighting and mesh calculations for objects situated in the peripheral field of view.
  • Cardboard Trigger Tap & Pointer Integration: Integrated tap events inside VREngine.handleTap() and GazePointer. Allows physical screen clicks to trigger press/release and hover actions on interactive Pointable nodes (e.g. SpatialButton) at the cursor gaze position.
  • Anti-Drift Calibration Filter: Added continuous low-pass drift adjustment to head-tracking gyroscope sensor fusion. Automatically recalibrates sensor bias when rotational speeds fall below 0.015 rad/s.

Fixed #

  • glTF Benchmark Tolerance: Adjusted timing thresholds in GLTF parsing stress tests to prevent false negatives under high CPU loads.

1.3.0 — 2026-05-27 #

Added #

  • OpenXR 26-Joint Skeletal Alignment & Gestures: Standardized input layer to the Khronos OpenXR specification, supporting physical bone linkages (25 bone joints), 3D orientations, and advanced mathematical gesture calculations (isFlatHand, isThumbsUp, isVictory).
  • Projected Radial Lens Distortion: Implemented polynomial barrel distortion directly in the Normalized Device Coordinates (NDC) projection of the vertex shader pipeline, enabling cardboard VR distortion correction with zero CPU/GPU rendering overhead.
  • Isolate-Based Background Sensor Fusion: Decoupled complementary sensor filters (IMU gyroscope and accelerometer math) and predictive head tracking calculations from the UI thread to a dedicated background Isolate thread, avoiding frame rate drops.
  • Volumetric Hologram Mesh Node (HologramMeshNode): Added a simulated volumetric shader employing a triple-pass shell technique (Core, body with flickering and glitch offsets, and glowing envelope) along with vertical scanline effects.
  • Isolate-Based Background WiFi Sensing Engine (WifiSensingSystem): Added background Isolate support to process OFDM Channel State Information (CSI) subcarriers, computing multipath disturbance standard deviation to track physical trajectories and vitals (respiration rate) without optical cameras.
  • 3D Skeletal Bone Cylinder Rendering: Renders full 3D skeletons between parent/child joints dynamically using quaternion orientation rotations.

1.2.0 — 2026-05 #

Changed #

  • Refactored VRParticle and VRRing classes to make color a non-final mutable field to enable real-time dynamic modulation for audio-reactive and visual animation effects.

1.1.0 — 2026-05 #

Added #

  • IMU Complementary Sensor Fusion: Stabilized landscape VR tracking by combining high-frequency gyroscope integration with low-frequency absolute gravity accelerometer vectors ($\alpha = 0.98$) to eliminate head tracking drift.
  • Google Cardboard Neck Model: Implemented realistic head-to-neck physical pivot rotations for camera projection, delivering comfortable stereoscopic parallax and dramatically reducing VR motion sickness.
  • Turbo-Canvas Render Engine Optimization: Shifted flat-shaded face rendering to use Flutter's ultra-high-performance canvas.drawVertices with ui.Vertices batched triangulation, replacing slow iterative path drawing with direct GPU-friendly vertex buffers.
  • Injectable Sensor Streams: Refactored HeadTracker constructor to accept optional mock streams (gyroscopeStreamOverride, accelerometerStreamOverride), enabling complete deterministic offline simulation and testing of IMU inputs.
  • Expanded Test Suite: Added comprehensive unit tests validating complementary filter math stability, noisy drift calibration offsets, symmetric eye offsets, and physical neck model boundaries (170 tests passing at 100%).
  • OSI Apache-2.0 License compliance: Replaced generic license templates with the official Apache 2.0 license file to recover full pub.flutter-io.cn score points under file conventions.

Fixed #

  • Static Analysis warnings: Resolved all 14 lint warnings and info items (e.g. deprecated .red usage, unused imports/variables, non-const declarations, and package dependency rules) achieving 0 issues across the engine and example.
  • Dependency boundaries updated: Raised sensors_plus constraint to ^7.0.0 and optimized vector_math to ^2.2.0 to support modern platforms while avoiding SDK version solver pins.

Future Roadmap & Surrounding Integrations #

  • Unified Locomotion Driver: Incorporate touch joysticks from neighboring 3D engines (e.g. openworlddart) to combine with the built-in accelerometer physical step-locomotion peak detector.
  • OpenXR Joint Mapping Alignment: Standardize the 26-joint tracking framework inside HandState to match Khronos OpenXR specifications for future FFI driver compatibility.
  • Cardboard QR Profile Loader: Implement pure-Dart base64 protocol buffer profile decoders to parse QR codes for custom lens radial distortion models dynamically.
  • Aesthetic "WOW" Factor Expansion: Build togglable PBR Physics Playgrounds (interacting with metallic reflectives and grabbing objects with dynamic impulse responses) and Deep Space Flight Simulators directly within the interactive sample app.

1.0.0 — 2026-04 #

Added #

  • VR Celestial Compass example: wireframe globe, cardinal markers, SpatialText labels, zone detection, step locomotion
  • analysis_options.yaml with Flutter lint rules
  • example/ app scaffold (Android/iOS/macOS/Web/Linux/Windows) with pub.flutter-io.cn example.dart
  • issue_tracker and platforms fields in pubspec.yaml
  • Cover image in README and pub.flutter-io.cn screenshots
  • Zone detection system based on Meta VR research (30°/55°/135° thresholds)
  • Step detection via accelerometer peak detection

Fixed #

  • Head tracking: correct landscape gyroscope axis mapping (X→yaw, Y→pitch)
  • Head tracking: inverted horizontal direction fixed
  • Head tracking: vertical sensitivity amplified ×1.8 for comfortable up/down range
  • LitMeshNode: lights list now mutable (was const [], caused runtime crash)
  • Deprecated Color.red/green/blue/alpha API replaced with Color.r/g/b/a
  • Removed unnecessary dart:typed_data import in texture.dart
  • Fixed dangling library doc comment in webxr_session.dart
  • All 36 analyzer warnings and hints resolved (0 issues)
  • Dart format applied to all 59 source files

Changed #

  • Version 1.0.0 — first stable release
  • README Quick Start uses pub.flutter-io.cn dependency

0.1.0 — 2026-04 #

Added #

  • Initial VR engine: 59 Dart source files
  • Scene graph with hierarchical nodes and transforms
  • Mesh rendering with Phong + PBR materials
  • Stereoscopic side-by-side projection
  • Head tracking via gyroscope (sensors_plus)
  • Interaction: gaze pointer, raycast, grabbable, pointable
  • Locomotion: fly, teleport, walk
  • Animation: clips, mixer, keyframes, skeleton, skin deformer
  • Effects: bloom, fog, SSAO, vignette, lens distortion
  • Primitive geometries: cube, sphere, plane, cylinder
  • Basic glTF 2.0 parser
  • Spatial UI: billboard, panel, button, text
  • WebXR session bridge
  • Debug overlay and grid floor
  • Physics: rigid body, AABB collision, gravity
  • 163 tests with benchmarks
  • 5 demo examples: VR Gallery, Treasure Hunt, Open World, Spatial UI, Hand Tracking
4
likes
160
points
141
downloads
screenshot

Documentation

API reference

Publisher

verified publisheropenneom.dev

Weekly Downloads

VR 3D engine for Flutter. Scene graph, mesh rendering, lighting, stereoscopic projection, head tracking, interaction, animation, and effects.

Repository (GitHub)
View/report issues
Contributing

Topics

#vr #virtual-reality #three-d #game-engine #flutter

License

Apache-2.0 (license)

Dependencies

flutter, sensors_plus, vector_math

More

Packages that depend on vrlizate