Task #236
Updated by Sanghoon Lee 12 days ago
Adds additional features for target analysis to the Xela Allegro Hand web UI interface. Adds tools to the Allegro Hand Xela web UI, similar to the 2F WebUI, to monitor whether robot control based on sensor input performs the desired action at the appropriate time. These features are FollowCam and Analysis. * FollowCam * Analysis !{width:800px}clipboard-202603231324-ja2nh.png! !clipboard-202603231324-ja2nh.png! --------------- h1. xela_taxel_sidecar_ah - FollowCam / Analysis h2. 1. Overview and design objective The AH sidecar design addresses higher taxel density and model-side mapping complexity compared to 2F. Design intent: * keep a stable sidecar service/topic contract while adding AH mapping/pattern semantics * decouple bridge mode switching from optional viz-node mode service availability * preserve compatibility with @std_xela_taxel_viz_ahv4@ namespace-local TF conventions This enables incremental AH web visualization improvements without forcing changes in tactile producer or std-viz packages. h2. 2. Bridge design h3. 2.1 Mapping model * mapping file provides flat index and frame-name contract * pattern file provides 2D grid projection * hand-side auto resolution switches prefix conversion and default pattern h3. 2.2 Signal model * force-first rendering with taxel fallback * baseline/deadband for force and taxel paths * optional group/module filtering (@show_thumb@, @show_module_XX@) h3. 2.3 URDF point model * enabled by @emit_urdf_points@ * TF lookup with cache (@tf_cache_ttl_sec@) * optional frozen coordinates (@freeze_urdf_positions@) h2. 3. Mode manager design * service endpoint: @/xela_viz_mode_manager/set_mode@ * always updates bridge params * viz service call controlled by: ** @use_viz_set_mode@ ** @require_viz_set_mode_service@ This avoids hard failure when target viz node has no @set_mode@. h2. 4. Launch design @xela_taxel_sidecar_cpp.launch.py@: * resolves hand/mapping/pattern at launch time (@OpaqueFunction@) * configures transport + mode manager options * supports namespace-local TF remap @xela_taxel_sidecar.launch.py@: thin wrapper to cpp launch. h2. 5. Analysis feature design h3. 5.1 Purpose Analysis provides real-time event detection and a scrolling timeline visualization for tactile interactions on the Allegro Hand. It surfaces contact onset/release, slip warnings, and explicit grasp-phase markers in a compact sparkline panel without blocking the primary visualization modes. h3. 5.2 Default state Analysis is *enabled by default* on page load (@setAnalysisVisible(true)@). The panel can be toggled at any time via the *Analysis* toolbar button. h3. 5.3 State model |_.Object|_.Key fields|_.Description| |@analysisState@|@visible: true@|Panel visibility flag| |@analysisEventState@|@contactThreshold: 0.14@, @lastContactActive@, @lastSlipActive@, @currentPhase@|Event tracking across frames| |@timelineState@|@samplePeriodMs: 100@, @windowMs: 30000@, @points[]@, @events[]@|Rolling telemetry ring buffer| h3. 5.4 Telemetry computation @computeAnalysisTelemetry@ is called every frame on the incoming bridge payload. Derives per-frame statistics from @payload.urdf.points@ (falling back to @payload.grid.points@): * @normMean@ — mean force magnitude across all sensor points * @fzSum@ — sum of vertical (Z) forces * @activeCount@ — count of points exceeding activation threshold 0.12 * @tip03Norm@ — peak force on module 03 (thumb tip) * @tip13Norm@ — peak force on module 13 (index tip) h3. 5.5 Event classification Three event source paths are merged into a unified event list. *Implicit contact events* (inferred from tip force levels): * @contact_detected@ — both @tip03Norm@ and @tip13Norm@ ≥ @contactThreshold@ (0.14), fires only during grasp/pinch phases * @contact_released@ — transition out of active contact state *Implicit slip events* (from @payload.meta.slip@): * @slip_warning@ — slip reflex output active (@reflex_output_active@) * @slip_active@ — slip condition sustained; tracked by trigger count changes *Explicit grasp events* (from @payload.meta.grasp_event@): * Grasp/pinch phase markers (approach, grasp, retreat, home) * Recovery, abort, success, and impedance/filter change indicators * Explicit events suppress nearby implicit events within a ±450 ms suppression window Each event record carries: @label@, @kind@, @tag@ (single char), @priority@, @explicit@, and @timestampMs@. h3. 5.6 Timeline UI * *Sparkline canvas* (@id="sparkline"@, 500 × 34 px) — plots @normMean@ with event markers overlaid as coloured vertical ticks * *Window controls* — 10 s / 30 s (default) / 60 s rolling window * *Pause/Resume* — freezes scrolling view for inspection * *Event tooltip* (@id="eventTooltip"@) — shown on hover; displays label, phase, source, and elapsed time since event h3. 5.7 Visibility control flow <pre> setAnalysisVisible(visible) → statsRow.style.display = visible ? "" : "none" → analysisBtn.classList.toggle("active", visible) → if (!visible) setEventTooltipVisible(false) </pre> h3. 5.8 Payload metadata dependencies |_.Feature|_.Required bridge field|_.Enable condition| |Slip events|@payload.meta.slip@|@include_slip_meta_@ = true in bridge node| |Grasp events|@payload.meta.grasp_event@|@include_grasp_event_meta_@ = true in bridge node| |Contact inference|@payload.urdf.points@ or @payload.grid.points@|Always available| h2. 6. FollowCam feature design h3. 6.1 Purpose FollowCam provides an automatic camera that continuously tracks the *thumb tip (module 03)* and *index tip (module 13)* of the Allegro Hand in RobotModel (URDF) mode. It eliminates the need for manual camera repositioning during dexterous manipulation tasks. h3. 6.2 Availability constraints * Only active when @state.uiMode === "robot"@ (RobotModel mode) * Button is rendered in the toolbar at all times but is @disabled@ when not in RobotModel mode * Automatically deactivated and camera reset to auto-frame when switching away from RobotModel mode h3. 6.3 State model |_.Field|_.Default|_.Description| |@enabled@|@false@|Master on/off flag| |@manualOrbitActive@|@false@|True while user is orbiting; pauses automatic update| |@captureViewOnNextUpdate@|@false@|Requests user offset capture on next update cycle| |@tipModule03Regex@|@/_[01]_03_/@|TF frame selector for thumb tip| |@tipModule13Regex@|@/_[01]_13_/@|TF frame selector for index tip| |@tipModule51/52/53Regex@|—|TF frame selectors for palm reference sensors| |@forwardDistance@|@-0.20@|Camera standoff along forward axis (m)| |@heightOffset@|@0.10@|Camera height offset above target base (m)| |@lookAhead@|@0.028@|Look-ahead offset applied to camera target point (m)| |@positionLerp@|@0.24@|Smoothing factor for camera position| |@targetLerp@|@0.26@|Smoothing factor for camera look-at point| |@forwardLerp@|@0.22@|Smoothing factor for forward direction vector| |@maxCamStep@|@0.030@|Maximum camera position movement per update (m)| |@maxTargetStep@|@0.024@|Maximum look-at target movement per update (m)| |@jumpRejectDist@|@0.16@|Distance threshold for jump-rejection filter (m)| |@jumpRejectLimit@|@2@|Consecutive outlier frames before accepting new position| URL query parameters can override @forwardDistance@, @heightOffset@, @lookAhead@, and @forwardSign@ at page load (@runtime_config.js@). h3. 6.4 Camera update algorithm @updateFollowCamera@ is called every render frame when @followCamState.enabled === true@. <pre> 1. Guard: URDF renderer must be ready 2. Frame resolution - resolve tip03 center ← module 03 taxel frames via regex - resolve tip13 center ← module 13 taxel frames via regex - resolve palm centers ← modules 51, 52, 53 (optional) 3. Target base = midpoint(tip03, tip13) 4. Jump rejection - if |targetBase − filteredTargetBase| > jumpRejectDist increment jumpRejectCount; skip update if count < jumpRejectLimit - else reset jumpRejectCount 5. Forward direction - preferred: palm-centroid → target base vector - fallback: cross(tip03 − tip13, world-up) - apply forwardLerp smoothing; enforce minForwardDeg deadband 6. Local basis: right = cross(forward, world-up); up = cross(right, forward) 7. User orbit capture - on manualOrbitActive → captureViewOnNextUpdate = true - on captureViewOnNextUpdate: decompose camera into local basis offsets 8. Camera goal computation - if user offsets captured: apply offsets in current local basis - else: cameraGoal = targetBase + forward * forwardDistance + up * heightOffset targetGoal = targetBase + forward * lookAhead 9. Smooth interpolation with per-step clamping - lerp camera position; clamp to maxCamStep - lerp look-at target; clamp to maxTargetStep 10. Apply to Three.js OrbitControls </pre> h3. 6.5 Manual orbit interaction When the user drags/scrolls the 3D view while FollowCam is active: * @manualOrbitActive@ is set to @true@ * On the next FollowCam update, the current camera offset relative to the tracked target is captured into @userCameraOffsetLocal@ / @userTargetOffsetLocal@ * Subsequent updates apply these offsets in the continuously updated local frame, so the user's preferred viewpoint is maintained while still tracking finger movement h3. 6.6 Enable/disable flow <pre> setFollowCamEnabled(true) → followCamState.enabled = true → reset manualOrbitActive, jumpRejectCount, filteredTargetBase, filteredForward → hint: "FollowCam enabled: thumb/index fingertip tracking." → applyOrbitProfileForMode() → updateModeButtons() setFollowCamEnabled(false) → followCamState.enabled = false → hint: "FollowCam disabled: RobotModel fixed orbit mode." → autoFrameUrdfMeshCamera(true) → applyOrbitProfileForMode() → updateModeButtons() </pre> h3. 6.7 URL configuration parameters |_.Parameter|_.Default|_.Description| |@follow_cam_forward_distance@|@-0.20@|Camera standoff from target along forward axis (m)| |@follow_cam_height_offset@|@0.10@|Camera height above target base (m)| |@follow_cam_look_ahead@|@0.028@|Look-ahead shift of camera target point (m)| |@follow_cam_forward_sign@|@-1.0@|Sign of forward vector; flip for different hand orientations| h2. 7. Risks * wrong mapping/pattern combination for selected model * optional viz service disabled while user expects viz-side mode switch * TF frame mismatch in URDF mode * Analysis slip/grasp events silent if bridge metadata subscriptions not enabled (@include_slip_meta_@, @include_grasp_event_meta_@) * FollowCam tip-frame resolution fails silently if URDF not fully loaded or TF frames missing; camera freezes at last valid position * FollowCam jump-rejection may delay tracking recovery after rapid large motions h2. 8. Validation * launch left/right models * verify output topic and web updates * verify mode toggling with @use_viz_set_mode@ off/on * verify URDF points in bridge payload * Analysis: confirm panel visible on page load; verify contact/slip events annotated on sparkline during grasp sequence * Analysis: toggle panel off/on; confirm tooltip appears on hover over event markers * Analysis: enable @include_slip_meta_@ and @include_grasp_event_meta_@; verify explicit grasp events suppress implicit contact events within 450 ms window * FollowCam: enter RobotModel mode; verify button enabled; click to activate; confirm camera tracks thumb/index tips * FollowCam: manual orbit while FollowCam active; confirm user offset captured and maintained across subsequent frames * FollowCam: switch away from RobotModel; confirm FollowCam deactivates and camera resets to auto-frame * FollowCam: pass @follow_cam_forward_distance@ URL param; verify camera standoff matches supplied value