Put a virtual ball on a real table and the first few seconds can look convincing. Move your hand between the camera and the ball, and the illusion becomes much more demanding. The application now needs to know which surface is in front.
The early WebXR Depth API explainer proposes access to depth information for experiences that need a better relationship with their surroundings. It discusses collision and pathfinding, with occlusion as a more quality-sensitive use. At this stage it is a proposal, not a promise that every browser and phone provides equivalent depth.
A depth image is a set of distance estimates associated with image locations. To hide the correct part of a virtual object, the renderer compares the virtual depth with the estimated real depth. Where the real surface is nearer, the virtual surface should disappear behind it.
The comparison only makes sense when the measurements agree about coordinates. A depth sensor may have a different resolution or field of view from the color camera. It may also report values in a different representation. Sampling depth using the wrong coordinates can produce an occlusion mask that follows the hand but sits several pixels beside it.
A conceptual depth comparison. The real and virtual measurements must share aligned coordinates and compatible distance conventions.
Time is another coordinate. If the depth image describes an earlier frame, a moving hand leaves a visible mismatch even when the spatial transform is correct. Filtering can reduce noise, but filtering over time can also make the boundary slow to react. The application has to decide which error is less objectionable.
I would start with interactions that tolerate imperfect measurements. A rough understanding of the floor can keep a character from walking through it. A broad obstacle can influence a path. A tiny virtual object slipping accurately between two fingers asks for much finer information and more consistent updates.
Missing depth is not a rare exceptional state to be ignored. Reflective, transparent or poorly observed surfaces can be difficult, depending on the sensing method. The application should distinguish an absent measurement from a measured empty area. Otherwise it may confidently place an object inside something it never understood.
There is an interesting design choice in how much of this uncertainty the user sees. A placement indicator can become tentative on an unreliable surface. An object can wait for a stable location before settling. These behaviors are often more understandable than letting the renderer continuously correct a visibly unstable estimate.
The API's value would be to make such reasoning possible on the web, alongside the rest of an experience. Access alone does not solve the experience. Developers still need alignment, timing, confidence handling and a sensible fallback when depth is unavailable.
I am interested in that modest progression: from drawing over the camera image to letting the surrounding space influence what the application does. The convincing cases will be the ones whose interactions match the quality of information the device can actually provide.