WebGPURenderer: Geometry transformations are broken at large scales

[gkjohnson]

Description

When rendering objects at a large distance from the origin in WebGPURenderer the transformations are choppy and imprecise. There are no issues with WebGLRenderer. My assumption is that the model * view transformation is being performed on the GPU vs CPU, as is done with WebGLRenderer.

Related to NASA-AMMOS/3DTilesRendererJS#1079

Reproduction steps

1. Render objects with transformations that are far from the origin.
2. Notice incorrect, jittery transformations.
3. Notice that animations look correct with WebGLRenderer.

Code

// positions the whole scene very far from the origin
scene.position
  .set( 1, 1, 1 )
  .multiplyScalar( 1e8 );

Live example

https://jsfiddle.net/1z9wjtk0/2/

Replace WebGPURenderer with WebGLRenderer in the above demo to see the difference.

Screenshots

No response

Version

175

Device

No response

Browser

No response

OS

No response

[sunag]

It is necessary to replace the default nodes to compute modelViewMatrix via CPU, as mentioned in the PR, this was replaced for performance reasons.

Example with the fix
https://jsfiddle.net/rpg7hjoy/

import { highpModelViewMatrix, highpModelNormalViewMatrix } from 'three/tsl';

renderer.nodes.modelViewMatrix = highpModelViewMatrix;
renderer.nodes.modelNormalViewMatrix = highpModelNormalViewMatrix;

We can consider a renderer property for this like renderer.modelViewCPU = true 🤔

[cmhhelgeson]

Seems like @gkjohnson predicted this in the initial PR: #29299 (comment)

I think something like renderer.useHighPrecision would be a better name over renderer.mobileViewCPU. If a beginner user is scaling their coordinates to 1e8 like the fiddle listed above, I think it's more intuitive that they would set a flag for more precise data rather than a flag that only indicates that the modelView matrix is being calculated on the CPU. Basically, the connection between large coordinate spaces and higher precision data formats may be more obvious than the connection between those large spaces and calculating on the CPU.

[sunag]

renderer.useHighPrecision looks good to me 👍

[gkjohnson]

I think this is a pretty big step back in terms of usability especially for new users. As mentioned in the above-referenced comment, users already struggle with rendering precision in cases that aren't currently handled by WebGLRenderer. I understand the inclination to chase every bit of performance gain but there are other ways to improve perf for users that are reaching matrix operation bottle necks that aren't even available in WebGPURenderer yet (like batching) so removing a feature like this seems a bit premature imo. If this kind of performance gain was needed then we should have removed it from WebGLRenderer, no?

Here are some other options for gaining render performance without removing the default high precision behavior for users:

• Use batching to limit operations needed per-mesh.
• Add a flag that defaults to true to enable these types of operations.
• Automatically toggle this kind of high precision transformation to on when the camera reaches a certain distance from the origin.
• Pre-transform the matrixWorld position to be relative to the camera, as suggested in the other issue (though this will have other implications).

[sunag]

The fact that WebGLRenderer has stopped in time in managing bindings and static objects does not mean we should do the same with WebGPURenderer, the differences are quite significant in relation to performance especially if we want to use a RenderBundle, and as you may have noticed during all these months the only open issue about this was this one, which means that most projects do not need this kind of precision, so we can prioritize performance.

[gkjohnson]

I understand that WebGPURenderer is a moment to rethink some features from WebGLRenderer but this feature was also originally added for a reason. There are also a number of other solutions I provided for retaining performance improvement while still providing the new-user accessibility that has made three.js successful in the first place, so it would be great if we could discuss a few of them and why they may or may not work.

Regarding the number of issues around WebGPURenderer - as I've mentioned in other issues - there are a large swath of projects that cannot or are not ready to move to WebGPURenderer for a number of reasons. I've talked to several clients who have looked into migrating but realized something wasn't working and had to revert. Or the migration path is too complex. All to say I think pointing to the number of issues around some of these problems is leaving out a fairly huge portion of the three.js user base at the moment.

To be clear I make these issues in an effort to try to help increase the viability and usability of WebGPURenderer. I'm looking forward to the day where we can consider WebGLRenderer a legacy system.

[sunag]

I think your observation was important, so much so that I created the high precision nodes for this but there is no way to advance much in some things without modifying others, in this case since both solutions are available by changing a variable like renderer.useHighPrecision, three.js has always had this kind of change like so many other things, if the feature was being removed completely the concern would make more sense for me, but it can be solved with one line of code and will benefit most users.

[cmhhelgeson]

I think most people do expect the WebGPURenderer to work as drop-in for WebGLRenderer, which is communicated through most of the code samples being the same outside of node materials. I'll also corroborate Garrett's note on the lack of issues. The one time I worked with a client they presented many issues they had with a previous attempted migration to the WebGPURenderer, some of which they had already expressed to other Three.js contributors.

I think a solution that could address the desires for advancement and normalization with the previous renderer is setting a flag in the WebGPURenderer for calculating transformation matrices through the GPU rather than the other way around. This way, at least for this feature, WebGLRenderer users can use WebGPURenderer as a drop-in solution, while native WebGPURenderer users or those who truly need extra performance without extra precision can access a lower precision code path.

[sunag]

We will certainly include this in the migration guide as well as several other things that have changed.

About high precision being false by default for most users, I think the following way, none of the more than 150 webgpu examples use it, none of the projects I followed during these 8 months needed this feature, only this issue was opened, this shows to me that the migration problem is related to other things, sacrificing the performance of 99% of projects because of 1% does not seem like a good idea to me. Considering this CPU bit precision doesn't work with instances or skinned.

[gkjohnson]

I understand how you feel but can you help me understand why my suggestion for an automatic enabling of the flag isn't workable? This is something that should be able to be detected toggled:

Automatically toggle this kind of high precision transformation to on when the camera reaches a certain distance from the origin.

This may be something I look into at a later time.