Is there a GLIB emscripten port available ?

[o-micron]

Hi, I had some code depending on glib and I went to build glib from source to find out it is a rabbit hole.
glib is using meson build system, no cmake or makefiles out there ...
6 hours later, it turns out the problem is building libffi for emscripten ...
Am I missing something ? Maybe there is a glib port available ?

[sbc100]

I'm not aware of any glib port I'm afraid. I imagine porting libffi might be tricky as IIRC it involves assembly code. Perhaps there is way to remove that dependency.

Feel free to add a port to https://github.com/emscripten-ports if you manage to get it working.

[kleisauke]

I had a similar problem while porting libvips to Emscripten (see libvips/libvips#192 for details). GLib and libffi currently require patches to successfully build them with Emscripten. Could you try these?:
https://gist.github.com/kleisauke/acfa1c09522705efa5eb0541d2d00887

These patches were copied from my local Git repository and tested with libffi 3.3 and GLib 2.64.2. Most of the libffi work has already been done by @Brion (thanks!), see: https://github.com/brion/libffi/commits/emscripten-work

I just made it compatible with the latest Emscripten (by swapping Module['usingWasm'] with Module['wasmMemory'] and converting the EM_ASM_ to EM_JS).

Anyways, I'll eventually try to incorporate these patches upstream.

[kripken]

That sounds great @kleisauke !

If there are issues getting them upstream, adding a port in emscripten-ports is also an option.

[o-micron]

thanks @kleisauke for your patches

I tried them on ubuntu 20.04 with :

• meson 0.54.1
• GLib 2.64.2
• libffi (the meson subproject master branch), not the brion's branch, because brion's branch doesn't contain a meson.build file which is a dependency for the main glib meson build script

Things look a bit better but I keep getting the error
ERROR: Problem encountered: Unsupported pair: system "emscripten", cpu family "wasm32"

I have applied your emscripten cross file --cross-file ./emscripten-crossfile.txt which is also what meson suggests for cross compiling as stated here starting a cross build

wasm32 is the right cpu family and not x86 of course ... but wasm32 seems unsupported
Meson is updated and so that shouldn't be the problem ?

I am exploring multiple issues at the same time I guess 😊

[kleisauke]

@o-micron I've updated the gist with an example build script. Let me know if that works for you.

[o-micron]

thanks @kleisauke that definitely works ! 💯

Also for anyone coming from the future for the same issue, here are some packages needed as well

# required tools ...
sudo apt install autoconf libtool m4 automake

# install meson from pip3
pip3 install meson
export PATH=~/.local/bin/:$PATH

# make sure you have emcc and wasm-ld in path
cd emsdk
./emsdk install latest
./emsdk activate latest
source ./emsdk_env.sh
export PATH=$(pwd)/upstream/bin/:$PATH

# download the gist https://gist.github.com/kleisauke/acfa1c09522705efa5eb0541d2d00887
# extract gist .zip and cd to that folder 
# now run ./build.sh and you should be good to go

[kripken]

Great!

If someone is interested to add this to emscripten-ports let me know - I can create a repo there and give you access. Would be good to do that since this has come up more than once.

[o-micron]

@kripken I would definitely help adding that to emscripten-ports but I would rather let @kleisauke do it and take the credit for his work

I have added above the simple steps I took to successfully build it on ubuntu

[o-micron]

@kripken Here is a script which does everything really ... could replace the whole port repository

Tested on Ubuntu 20.04

# required tools ...
sudo apt install autoconf libtool m4 automake

# install meson from pip3 because glib uses meson ..
pip3 install meson
export PATH=~/.local/bin/:$PATH

# Get the emsdk repo
git clone https://github.com/emscripten-core/emsdk.git

# Enter that directory
cd emsdk
# Fetch the latest version of the emsdk (not needed the first time you clone)
git pull

# Download and install the latest SDK tools.
./emsdk install latest

# Make the "latest" SDK "active" for the current user. (writes ~/.emscripten file)
./emsdk activate latest

# Activate PATH and other environment variables in the current terminal
. ./emsdk_env.sh

# Make sure we have mainly wasm-ld and other tools in path as well
export PATH=$(pwd)/upstream/bin/:$PATH

cd ..

# Get @kleisauke patches
git clone https://gist.github.com/acfa1c09522705efa5eb0541d2d00887.git
cd acfa1c09522705efa5eb0541d2d00887

chmod +x ./build.sh
./build.sh

[kripken]

Nice @o-micron, yeah, if we don't need a separate repo then this is quite easy. That script could be converted into a python script and put under tools/ports/glib.py basically, and integrated into ports. I think that would be great to do!

[sbc100]

For a long time we (I) have been trying to come up with a place to put "contrib" ports like this. Simply adding everything and the kitchen sink to "tools/ports" doesn't scale very well. I don't have an answer right now.. only the question.

[kleisauke]

For a long time we (I) have been trying to come up with a place to put "contrib" ports like this. Simply adding everything and the kitchen sink to "tools/ports" doesn't scale very well.

The meson build system solves this problem with the Wrap dependency system.
https://mesonbuild.com/Wrap-dependency-system-manual.html
https://github.com/mesonbuild/meson/blob/master/mesonbuild/wrap/wrap.py

Perhaps we could do something similar with Emscripten ports? For example, port files could look like this:

Details

[port-file]
directory = glib-2.64.2

source_url = https://download.gnome.org/sources/glib/2.64/glib-2.64.2.tar.xz
source_filename = glib-2.64.2.tar.xz
source_hash = 9a2f21ed8f13b9303399de13a0252b7cbcede593d26971378ec6cb90e87f2277

patch_url = https://gist.github.com/kleisauke/acfa1c09522705efa5eb0541d2d00887/raw/37e3787a16cef2b8a1b577d469406ebbb80147d7/glib-emscripten.patch
patch_filename = glib-emscripten.patch
patch_hash = bb46e867ce457d4c34551968303ffde520b5f17a54df69653e11677b7df9cfc5

[port-git]
url = https://github.com/brion/libffi.git
revision = emscripten-work

^{(This is heavily inspired from Meson Wrap files)}

The advantage of this is that updating ports can then be done without much effort, since you don't have to embed an entire project into the source tree (i.e. tests/third_party) or to maintain separate forks (i.e. the emscripten-ports organization). Any Emscripten specific patches can then be easier integrated upstream.

This is only for the source part, I'm not sure how the actual build of these ports might look like (because it can be very specific to users' needs).

[sbc100]

The immediate problem I'd like to solve is how to add ports without adding:

1. New settings such as USE_ZLIB
2. Load on the CI system (its fine if we have a separate builder that build the universe but we don't want to block evey change on this).

There are lots good ports systems out there. Its not clear if we want to block this work on choosing one and switching to it, or sticking with our existing one. Part of me would like to do all of this out-of-tree using something like pacman (from arch) the freebsd ports system: https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ports-using.html. But as an incremental step I think we could just use tools/ports/contrib with out existing ports system. I think the plan was to have a generic -s USE_PORT=foo setting to avoid adding new settings for these. Also, embuilder would not build these by default. I think is convered by an existing bug but I can't seem to find it right now.