Proceed with `Float16Array`

[petamoriken]

Float16Array is proposed in TC39 at Stage 1. Can we move in the direction of advancing that?

---

Choice of Float32Array for ImageData

There does not exist a Float16Array type, and so it is not an option.
The only other floating-point type, Float64Array, is of unnecessarily high precision.

https://github.com/w3c/ColorWeb-CG/blob/14bffcb85327b011026def4f7ebf9b88e724afad/canvas_float.md#choice-of-float32array-for-imagedata

[petamoriken]

Another option is to use Uint16Array.

---

FYI: WebGL 2.0 uses Uint16Array for half float texture.

https://registry.khronos.org/webgl/specs/latest/2.0/#3.7.6

[palemieux]

See also https://lists.w3.org/Archives/Public/public-colorweb/2022Dec/0014.html

[palemieux]

Below is consensus input from the CG per https://lists.w3.org/Archives/Public/public-colorweb/2023Jan/0000.html

CG proposal re: Float16Array + HTML Canvas

The ColorWeb CG is seeking consensus with parties interested in
discussing floating-point backing stores for canvas rendering
contexts.

The CG spec proposal is:
https://github.com/w3c/ColorWeb-CG/blob/main/canvas_float.md

The primary outstanding issue at present is how to represent values
which are read back from the canvas into typed arrays which can be
manipulated from ECMAScript.

The current spec proposal supports readbacks into both
Uint8ClampedArray and Float32Array via ImageDataColorType.

It has been suggested to also add support for readbacks into
Float16Array (#87). This
would make the specification dependent on the Float16Array type
currently being defined by the ECMAScript committee (ISO/TC 39).

The CG supports the idea of adding support for Float16Array to
ECMAScript. However, the CG feels it is most appropriate to decouple
the ongoing improvements to ECMAScript from the work on floating-point
canvases. Users of the web platform are expressing the need for
floating-point canvas backing stores now; in Chromium, implementation
is underway in crbug.com/1230619. Moreover, readback and CPU-side
manipulation paths are not high-performance, so the additional memory
bandwidth of using Float32Array for this task will not critically
impact applications.

The ColorWeb CG therefore suggests:

1. Moving ahead with
   https://github.com/w3c/ColorWeb-CG/blob/main/canvas_float.md in its
   current form, with unorm8 and float32 as options for readback.

2. In parallel, working on adding Float16Array to ECMAScript, and
   prototyping it in browsers.

3. Specifying a "float16" enum in ImageDataColorType once
   implementation experience has been achieved with Float16Array, .
   Applications can feature-detect its support in browsers by catching
   exceptions raised by getImageData or createImageData.

Background

For context, introducing a Float16Array into the Typed Array hierarchy
was discussed several years ago among members of TC39, as well as the
Khronos Group where Typed Arrays originated. At the time, the costs of
supporting this type natively in ECMAScript engines outweighed the
benefits for the following reasons:

1. For practical purposes, Float16Array could be sufficiently
   polyfilled in ECMAScript:

https://github.com/petamoriken/float16

Corner cases existed regarding denormalized values, NaNs, and
infinities - but applications could generally achieve their desired
results, with good performance on all ECMAScript engines, using
existing primitives.

2. Direct CPU support for half-float numbers did not seem to exist. At
   the time, C libraries which worked with this data type universally
   emulated it, rather than using compiler intrinsics on certain
   platforms.

3. A significant amount of work would be required in every ECMAScript
   engine to make this typed array type perform well.

4. A significant amount of work would be required to specify the
   behavior of this numeric type in ECMAScript.

The landscape has changed in recent years. Per
https://en.wikipedia.org/wiki/Half-precision_floating-point_format#Hardware_support,
ARM and x86 processors either support FP16 natively, or will in the
near future. FP16 formats are increasingly heavily used in neural
network evaluation and image processing, including on the web. The
ECMAScript editors may be able to save spec work by referencing IEEE
specifications (which, to be fair, already existed when Float16Array
was originally considered).

[chrisn]

TC39 discussed Float16Array in their March meeting - see notes.

[petamoriken]

Update: the proposal to add Float16Array to JavaScript today reached stage 3, meaning the design is finished, the committee is in favor, and engines can start implementing and shipping it.

KhronosGroup/WebGL#3531 (comment)

[mdrejhon]

Will the HDR CANVAS version of .getImageData() and .putImageData() migrate to FP16?

As a stopgap, I'd even settle for 16-bit integers (and use software-based pixel processing) and/or putting the numbers in higher-precision floating point numbers.

I eventually need to be able to HDR-colorpicker for TestUFO, an existing HDR CANVAS color swatch rendered by gradients/etc. As TestUFO HDR 2.0 is about to be launched (see my comment at #111 (comment))...