Skip to content

Single-canvas inline, drop XRPresentationContext #656

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jun 13, 2019
Merged

Single-canvas inline, drop XRPresentationContext #656

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
b7a1a8f
Single-canvas inline, drop XRPresentationContext
toji May 17, 2019
771b48a
Updated the explainer
toji May 17, 2019
2dbf9f1
Addressing some feedback
toji May 22, 2019
bed660f
Move direct output flag to render state
toji May 23, 2019
9b65d29
Comment about the WebGLRenderingContext's attributes taking precedenc…
toji May 23, 2019
458034b
Changed option to compositionDisabled
toji May 28, 2019
41c03e6
Fixed up explainer
toji May 29, 2019
006f5a1
Fixed a broken link to an outdated product
toji May 31, 2019
dfbc4bb
Change composition disabled to be implicit based on session type
toji Jun 12, 2019
a0a5310
Fixing a few loose ends
toji Jun 13, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 11 additions & 61 deletions explainer.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -365,77 +365,35 @@ function checkARSupport() {

The UA may choose to present the immersive AR session's content via any type of display, including dedicated XR hardware (for devices like HoloLens or Magic Leap) or 2D screens (for APIs like [ARKit](https://developer.apple.com/arkit/) and [ARCore](https://developers.google.com/ar/)). In all cases the session takes exclusive control of the display, hiding the rest of the page if necessary. On a phone screen, for example, this would mean that the session's content should be displayed in a mode that is distinct from standard page viewing, similar to the transition that happens when invoking the `requestFullscreen` API. The UA must also provide a way of exiting that mode and returning to the normal view of the page, at which point the immersive AR session must end.

## Rendering to the Page

There are a couple of scenarios in which developers may want to present content rendered with the WebXR Device API on the page instead of (or in addition to) a headset: Mirroring and inline rendering. Both methods display WebXR content on the page via a Canvas element with an `XRPresentationContext`. Like a `WebGLRenderingContext`, developers acquire an `XRPresentationContext` by calling the `HTMLCanvasElement` or `OffscreenCanvas` `getContext()` method with the context id of "xrpresent". The returned `XRPresentationContext` is permanently bound to the canvas.

An `XRPresentationContext` can only be supplied imagery by an `XRSession`, though the exact behavior depends on the scenario in which it's being used. The context is associated with a session by setting the `XRRenderState`'s `outputContext` to the desired `XRPresentationContext` object. An `XRPresentationContext` cannot be used with multiple `XRSession`s simultaneously, so when an `XRPresentationContext` is set as the `outputContext` for a session's `XRRenderState`, any session it was previously associated with will have it's `renderState.outputContext` set to `null`.

### Mirroring

On desktop devices, or any device which has an external display connected to it, it's frequently desirable to show what the user in the headset is seeing on the external display. This is usually referred to as mirroring.

In order to mirror WebXR content to the page, the session's `renderState.outputContext` must be set to a `XRPresentationContext`. Once a valid `outputContext` has been set any content displayed on the headset will then be mirrored into the canvas associated with the `outputContext`.

When mirroring only one eye's content will be shown, and it should be shown without any distortion to correct for headset optics. The UA may choose to crop the image shown, display it at a lower resolution than originally rendered, and the mirror may be multiple frames behind the image shown in the headset. The mirror may include or exclude elements added by the underlying XR system (such as visualizations of room boundaries) at the UA's discretion. Pages should not rely on a particular timing or presentation of mirrored content, it's really just for the benefit of bystanders or demo operators.

The UA may also choose to ignore the `outputContext` on systems where mirroring is inappropriate, such as devices without an external display like mobile or all-in-one systems.

```js
function beginXRSession() {
let mirrorCanvas = document.createElement('canvas');
let mirrorCtx = mirrorCanvas.getContext('xrpresent');
document.body.appendChild(mirrorCanvas);

navigator.xr.requestSession('immersive-vr')
.then((session) => {
// A mirror context isn't required to render, so it's not necessary to
// wait for the updateRenderState promise to resolve before continuing.
// It may mean that a frame is rendered which is not mirrored.
session.updateRenderState({ outputContext: mirrorCtx });
onSessionStarted(session);
})
.catch((reason) => { console.log("requestSession failed: " + reason); });
}
```

### Inline sessions
## Inline sessions

There are several scenarios where it's beneficial to render a scene whose view is controlled by device tracking within a 2D page. For example:

- Using phone rotation to view panoramic content.
- Taking advantage of 6DoF tracking on devices (like [Tango](https://get.google.com/tango/) phones) with no associated headset.
- Taking advantage of 6DoF tracking on devices with no associated headset, like [ARCore](https://developers.google.com/ar/) or [ARKit](https://developer.apple.com/arkit/) enabled phones.
- Making use of head-tracking features for devices like [zSpace](http://zspace.com/) systems.

These scenarios can make use of inline sessions to render tracked content to the page. Using an inline session also enables content to use a single rendering path for both inline and immersive presentation modes. It also makes switching between inline content and immersive presentation of that content easier.

The [`RelativeOrientationSensor`](https://w3c.github.io/orientation-sensor/#relativeorientationsensor) and [`AbsoluteOrientationSensor`](https://w3c.github.io/orientation-sensor/#absoluteorientationsensor) interfaces (see [Motion Sensors Explainer](https://w3c.github.io/motion-sensors/)) can be used to polyfill the first case.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not this PR, but boy does this sentence seem strange now that we're committed to inline.


Similar to mirroring, to make use of this mode the `XRRenderState`'s `outputContext` must be set. At that point content rendered to the `XRRenderState`'s `baseLayer` will be rendered to the canvas associated with the `outputContext`. The UA is also allowed to composite in additional content if desired. (In the future, if multiple layers are used their composited result will be what is displayed in the `outputContext`.)

Immersive and inline sessions can use the same render loop, but there are some differences in behavior to be aware of. Most importantly, inline sessions will not pump their render loop if they do not have a valid `outputContext`. Instead the session acts as though it has been [suspended](#handling-suspended-sessions) until a valid `outputContext` has been assigned.
`XRWebGLLayer` created with an `inline` session will not allocate a new WebGL framebuffer but instead set the `framebuffer` attribute to `null`. That way when `framebuffer` is bound all WebGL commands will naturally execute against the WebGL context's default framebuffer and display on the page like any other WebGL content. When that layer is set as the `XRRenderState`'s `baseLayer` the inline session is able to render it's output to the page.

Immersive and inline sessions may run their render loops at at different rates. During immersive sessions the UA runs the rendering loop at the XR device's native refresh rate. During inline sessions the UA runs the rendering loop at the refresh rate of page (aligned with `window.requestAnimationFrame`.) The method of computation of `XRView` projection and view matrices also differs between immersive and inline sessions, with inline sessions taking into account the output canvas dimensions and possibly the position of the users head in relation to the canvas if that can be determined.

Most instances of inline sessions will only provide a single `XRView` to be rendered, but UA may request multiple views be rendered if, for example, it's detected that that output medium of the page supports stereo rendering. As a result pages should always draw every `XRView` provided by the `XRFrame` regardless of what type of session has been requested.

UAs may have different restrictions on inline sessions that don't apply to immersive sessions. For instance, the UA does not have to guarantee the availability of tracking data to inline sessions, and even when it does a different set of `XRReferenceSpace` types may be available to inline sessions versus immersive sessions.

```js
let inlineCanvas = document.createElement('canvas');
let inlineCtx = inlineCanvas.getContext('xrpresent');
document.body.appendChild(inlineCanvas);

function beginInlineXRSession() {
// Request an inline session in order to render to the page.
navigator.xr.requestSession('inline')
.then((session) => {
// Inline sessions must have an output context prior to rendering, so
// it's a good idea to wait until the outputContext is confirmed to have
// taken effect before rendering.
session.updateRenderState({ outputContext: inlineCtx }).then(() => {
onSessionStarted(session);
});
// Inline sessions must have an appropriately constructed WebGL layer
// set as the baseLayer prior to rendering. (This code assumes the WebGL
// context has already been made XR compatible.)
let glLayer = new XRWebGLLayer(session, gl, { compositionDisabled: true });
session.updateRenderState({ baseLayer: glLayer });
onSessionStarted(session);
})
.catch((reason) => { console.log("requestSession failed: " + reason); });
}
Expand Down Expand Up @@ -543,7 +501,7 @@ function drawScene() {

Whenever possible the matrices given by `XRView`'s `projectionMatrix` attribute should make use of physical properties, such as the headset optics or camera lens, to determine the field of view to use. Most inline content, however, won't have any physically based values from which to infer a field of view. In order to provide a unified render pipeline for inline content an arbitrary field of view must be selected.

By default a vertical field of view of 0.5 radians (90 degrees) is used for inline sessions. The horizontal field of view can be computed from the vertical field of view based on the width/height ratio of the `outputContext`'s canvas.
By default a vertical field of view of 0.5 radians (90 degrees) is used for inline sessions. The horizontal field of view can be computed from the vertical field of view based on the width/height ratio of the `XRWebGLLayer`'s associated canvas.

If a different default field of view is desired, it can be specified by passing a new `inlineVerticalFieldOfView` value, in radians, to the `updateRenderState` method:

Expand Down Expand Up @@ -647,15 +605,13 @@ dictionary XRRenderStateInit {
double depthFar;
double inlineVerticalFieldOfView;
XRWebGLLayer? baseLayer;
XRPresentationContext? outputContext
};

[SecureContext, Exposed=Window] interface XRRenderState {
readonly attribute double depthNear;
readonly attribute double depthFar;
readonly attribute double? inlineVerticalFieldOfView;
readonly attribute XRWebGLLayer? baseLayer;
readonly attribute XRPresentationContext? outputContext;
};

//
Expand Down Expand Up @@ -696,6 +652,7 @@ enum XREye {
//

dictionary XRWebGLLayerInit {
boolean compositionDisabled = false;
boolean antialias = true;
boolean depth = true;
boolean stencil = false;
Expand Down Expand Up @@ -748,11 +705,4 @@ partial dictionary WebGLContextAttributes {
partial interface WebGLRenderingContextBase {
Promise<void> makeXRCompatible();
};

//
// RenderingContext
//
[SecureContext, Exposed=Window] interface XRPresentationContext {
readonly attribute HTMLCanvasElement canvas;
};
```
Loading