Discussion and development of web/wasm version

[falkTX]

For those unaware, https://cardinal.kx.studio/ is totally a thing now for some reason.
Also https://minicardinal.kx.studio/ for a mini/stripped-down version that loads faster.

There are some issues and things still to do.
Because this is for playground and testing and fun, and not all that serious, let's not open more than 1 ticket for web related things.
Everything web/wasm related should be on this ticket.

Stuff to do or implement:

• audio input
• touch events
• clipboard (do fake internal one if not possible to expose in browser)
• webmidi
• host time controls (on menu bar, providing fake but valid BBT values)
• intro message explaining what it all is, what to expect etc
• detect incompatible browser setups and refuse to even load (noscript, no simd, no webgl2)
• build libsndfile and fftw for extra module features
• minimum gui size (CSS is enough?)
• allow url parameters to tweak page behaviour, also load patches as base64 encoded string
• test loading all modules and disable those with custom non-GLES2 compat code

Known issues:

• Loading zst compressed project file fails to load patch.json
• opening salamonis-mono-regen example patch triggers an exception
• going full screen can mess up with the viewport size and coordinates
• ctrl+a clears all text
• audiofile player is missing regular audio support (only mp3 works)
• audiofile and midifile inside ildaeil fail to show file dialogs
• latest changes made performance in chrome worse?? firefox still fine..
• compressed data assets > 50Mb
• missing versioning to js, wasm and data downloads
  (so they are updated automatically when reopening the page as needed)

Stuff to fix or workaround:

• apply high-dpi event offset
• test and implement fractional scaling "high-dpi" (only supports 2x and 3x now)
• prevent ctrl/cmd+ shortcut keys from going to the browser
• prevent browser from zooming the page
• oddities with mouse down and leaving browser window

Undecided:

• host parameters
• host cv
• persistent storage (client-side browser DB)
• bundle a few simple LV2 plugins (mda, caps, tap, etc)

[falkTX]

Leaving a note to myself, https://github.com/Dreamtonics/juce_emscripten/blob/master/modules/juce_audio_devices/native/emscripten/RtMidi.cpp has rough but working code for webmidi.
Not that it is complicated to do, but helps to have something to compare against.

[falkTX]

high-dpi screen should work now.
tested locally with a mac mini.

if you have such a screen, let me know how it works!

[john-the-bard]

The ping pong delay from MSC makes a ton of noise if you load it, this basic patch which should play a single plucked note every second into a short and feedback heavy delay.

NOISE WARNING the output is connected and it immediately clips.

NOISEWARNING_Pluck.tar.gz

[diimdeep]

Insanity. Wowzers. Would be cool to load patch from url encoded parameter, https://cardinal.kx.studio/?patch=encoded_blob like this

[falkTX]

Insanity. Wowzers. Would be cool to load patch from url encoded parameter, https://cardinal.kx.studio/?patch=encoded_blob like this

I thought about this, and while I typically dislike such approaches, but I think it makes sense here.
A way to generate such link directly from the page would also be quite cool, so we can:

1. open cardinal web page
2. make a patch showing a specific thing
3. use file -> generate pasteable link (or whatever better name there is)
4. send that link to someone for quickly trying the patch

there are 2 things though that need to be done before we can tackle this:

1. fix opening zst compressed patches
2. allow for versioned caching of wasm data, so page reloads are instant (loading 100Mb of resources everytime to check new patches is just painful)

will add to the todo, thanks for the idea!

[pauldavisthefirst]

drag doesn't work to scroll boundaries of view area. click on output "socket", drag upward to reach module that is currently "off-screen" .. drag will break.

[pauldavisthefirst]

if scroll is used to zoom, easy to get to a point where cardinal does not occupy entire browser screen, at which point the coordinate determination seems to be messed up. drags will affect things not under the mouse.

[falkTX]

drag doesn't work to scroll boundaries of view area. click on output "socket", drag upward to reach module that is currently "off-screen" .. drag will break.

mentioned on the 1st post as

oddities with mouse down and leaving browser window

and

if scroll is used to zoom, easy to get to a point where cardinal does not occupy entire browser screen, at which point the coordinate determination seems to be messed up. drags will affect things not under the mouse.

known issue, I fixed it already locally but didnt deploy a build for it yet

[falkTX]

Small update: fixed up the ildaeil audiofile and midifile plugins, adjusted for more high-dpi screens, and updated view menu to follow official Rack 2.1.2 changes.

With the ildaeil fixes it is now possible to get some MIDI going.
Follow these steps:

1. Add ildaeil module
2. Add ildaeil expander midi output, place it touching the right side of ildaeil
3. In ildaeil, switch to internal plugins (it is JSFX by default)
4. Load "MIDI File"
5. Set "Host sync" to off, "Repeat Mode" to on
6. Press "show custom gui" button
7. Pick a MIDI file from your PC

Right-click the expander module for polyphony options.

[falkTX]

Good news, the issue with loading compressed project files is fixed. (or better said, a workaround is in place)
Somehow zstd extracts the tar with incorrect permissions, and then fails to load.
We now go for 777 for any file to be open, and all is good. The "files" are all virtual anyway, so doesn't matter on the web context.

Also, you can now pass it a patchstorage "slug" to the URL as a way to load that patch on start.
For example https://cardinal.kx.studio/?patchstorage=fern-sequence and https://cardinal.kx.studio/?patchstorage=example-patch-2
The URL parameter is the same that is used on patchstorage.com views, e.g. https://patchstorage.com/example-patch-2/

[falkTX]

Another update, this is fun 😆

Thanks to previous work done in https://github.com/DISTRHO/PawPaw I could just repurpose it to build generic opensource libs for wasm. For now that is libogg, libvorbis, flac, opus, sndfile, liblo, fftw and lv2 related libs.
This helps a lot on building opensource plugins in wasm.
Also these extra libs allow to get the extra/proper file support in the audiofile module and brings in the audio-to-cv-pitch module (which required fftw)

I did a few more tweaks and made Cardinal able to load audio plugins from the "filesystem" (JSFX dont really count, they are not binaries).
I have compiled artyfx, die-plugins (imported ardour fx), fomp and mda LV2 plugins, and added them to the site/webapp. Just open up ildaeil and they will be there for the LV2 option.

[falkTX]

ok a couple more things now:

• fixed up event key input handling a bit, ctrl+a no longer clears all text
• keyboard is no longer propagated to browser, so stuff like ctrl + scroll works now (some key combos like ctrl+n and ctrl+t cannot be overridden)
• pointer lock is implemented and enabled by default (opt-out in view menu)
• added full screen menu option

[dromer]

Going out of full screen returns to the wrong size render (x/y mouse coordinates are still correct, but don't align with the view).

[falkTX]

Going out of full screen returns to the wrong size render (x/y mouse coordinates are still correct, but don't align with the view).

I have some ideas for this, the fullscreen mode typically changes the viewport to render in "pixel perfect" mode by default, while typically contents are scaled instead. When entering fullscreen, we can request how to deal with the viewport and keep things as-is. Will try later