WebXR controls that work on Quest and Vision Pro

I already had a browser orbit viewer for the house. The headset page at /creations/joes-house/xr is a second route: same model, different input. Quest-style controllers and Vision Pro hands both work. That is not two control systems.

Both devices show up as WebXR input sources. @react-three/xr turns those into React Three Fiber pointers (laser, pinch, gaze). Buttons are meshes with onClick. Grabs go through @react-three/handle. Same hit targets, both headsets.

Live path: joemore.com/creations/joes-house/xr. Source lives under components/creations/joes-house/xr/.

One pointer path

A SpatialButton is a RoundedBox plus a canvas label. Trigger select and pinch select both land on onClick. Stop the event so the orbit / grab layer does not steal it.

onPointerDown={(event) => { event.stopPropagation(); if (!disabled) setPressed(true); }}
onClick={(event) => { event.stopPropagation(); if (!disabled) onSelect(); }}

That lives in SpatialUI.tsx. Reuse it as-is on the next scene.

Own route, no SSR

WebXR is browser-only. The house keeps a dedicated page (pages/creations/joes-house/xr.tsx) that loads XRExperience with next/dynamic and { ssr: false }. Do not hang navigator.xr off the desktop viewer.

Check first, enter on the click

useXRCapabilities.ts refuses insecure pages, then asks navigator.xr.isSessionSupported('immersive-ar') and 'immersive-vr'. Hide the buttons that cannot start.

The session itself must start from the tap. No async preflight before requestSession — browsers treat that as not a user gesture. Keep a native session ref in case setSession fails, otherwise you can leak an open immersive session with nothing on screen.

const session = await navigator.xr!.requestSession(mode, runtime.sessionInit);
nativeSession.current = session;
manager.setReferenceSpaceType("local");
await manager.setSession(session);

See enter() in XRExperience.tsx.

The store

session.ts is the reusable config. Hands on, built-in teleport pointers off (you teleport with your own meshes), transient pointer and gaze on, controller/hand GLBs served from your site rather than a CDN.

const sessionInit: XRSessionInit = {
	requiredFeatures: [],
	optionalFeatures: ["local-floor", "hand-tracking", "hit-test"],
};

const store = createXRStore({
	emulate: false,
	offerSession: false,
	enterGrantedSession: false,
	handTracking: true,
	hand: { teleportPointer: false },
	controller: { teleportPointer: false },
	transientPointer: true,
	gaze: true,
	baseAssetPath: new URL(
		"/creations/joes-house/xr-inputs/",
		window.location.href,
	).href,
	defaultControllerProfileId: "generic-trigger",
	defaultXRHandProfileId: "generic-hand",
	customSessionInit: sessionInit,
});

Point baseAssetPath at a copy of public/creations/joes-house/xr-inputs/ under the new creation. Wrap the Canvas with <XR store={store}>, put an XROrigin in the scene, start with local space and upgrade to local-floor when the runtime has it (XRScene.tsx). AR: transparent clear. VR: your own backdrop.

Grab the model, not the camera

TabletopGizmo.tsx attaches @react-three/handle Handles to the model group, not the camera.

• Move: RGB axis widgets. Clamp a single gesture to 3 m so a tracking jump cannot fling the house into the void.
• Rotate: gold torus, yaw only, plus a fatter invisible hit torus so pinch / laser is not a pixel hunt.
• Cancel the drag on inputsourceschange, visibility loss, or session end.

<Handle targetRef={targetRef} translate="as-rotate" rotate="y" scale={false} multitouch={false}>
  {/* gold ring + generous hit torus */}
</Handle>
<Handle targetRef={targetRef} translate rotate={false} scale={false} multitouch={false}>
  {/* RGB move handles */}
</Handle>

UI that follows the head — until you aim at it

FollowingPanel in FloatingMenu.tsx sits in front of the visitor with a dead zone so small head motion stays quiet. pauseOnHover freezes interpolation while a pointer is on a button, otherwise the panel slides out from under a pinch. Tabletop: Menu / Exit sit on the ring. Life-size: the dock follows the head.

Look down when you lose the house

RecoveryControl.tsx does not parent to the table. Pitch down about 20° and Reset table appears. Copy this even if you skip the gold ring — it is the difference between “I lost the model” and “I can recover without tearing the session down”.

Copy this, not the house bits

Reuse: the XR route, capability hook, createXRStore options, click-path requestSession + setSession, <XR> + XROrigin, SpatialButton, Handle gizmo, FollowingPanel, RecoveryControl, local xr-inputs GLBs.

Leave behind: TABLE_SCALE, garden teleport stations, AR hit-test reticle, life-size toggle, __HOUSE_XR_METRICS__. Those are this model, not the control layer.

Porting checklist

1. New /xr route, dynamic(..., { ssr: false }).
2. Copy session.ts + useXRCapabilities.ts + the enter() gesture.
3. Wrap your Canvas in <XR store={store}>.
4. Swap the house group for your model; keep the spatial widgets.
5. Ship xr-inputs/ under that creation’s public prefix.
6. Real headsets need HTTPS (localhost is fine on the same machine).

That is the whole trick: one pointer UI, two kinds of hands.