Skip to content

Gleefre/hello-alien

Repository files navigation

Hello Alien

About

In this project I attempted to use SBCL in an android application.

Status: succeeded. (Yay!)

Huge thanks to karlosz, Shinmera, hayley and |3b| who helped me to debug problems on the #sbcl channel!

Also thanks to fstamour for the idea.

You can check out the .apk in the prebuilt/apk folder. It works only on x86_64 and arm64-v8a cpu and it is not signed (but you still can install it without ADB).

See also Simple Repl App for a slightly bigger example.

Design

SBCL can be built as a shared library (see SBCL news v2.1.10). To use it as a part of an android app, it needs to be compiled using Android NDK, which can be done with make-android.sh. (see SBCL news v2.3.4).

The source consists of a lisp file that gets compiled into a core on the target device, a C wrapper that allows Java to call into lisp, and a very simple Java frontend for the application.

Lisp side

alien.lisp defines a simple function (hello) that returns a string to be displayed by the Java frontend. This function is registered as an alien callback corresponding to a C function (hello) with define-alien-callable (see SBCL manual 9.8 Calling Lisp From C). The C symbol hello is contained in the C wrapper wrap.c (see below).

C side

The C wrapper wrap.c defines Java native methods and the hello symbol to be used by the lisp callback. The native method HelloActivity.initLisp calls SBCL’s initialize_lisp, while the HelloActivity.getAlien calls the hello function defined in alien.lisp.

The C wrapper also redirects the process stdout and stderr output streams to the android’s logging facility that can be viewed with adb logcat. (See this article.)

Java side

Finally, the Java frontend HelloActivity.java defines the main activity for the app. It is simple – it shows a single button. When you press it, it calls the getAlien native method that calls the lisp hello function to get a string that is then set to be the button’s text. When you press it for the first time, the app also loads the C wrapper and initializes lisp.

Problems

dlsym

At first, the application proccess simply died when calling initialize_lisp. At the same time, the lisp initialization scheme worked for a standalone executable (running through adb shell).

It turned out that when SBCL tried initialize the alien callable hello it couldn’t find the corresponding foreign symbol. It was present in the lib.gleefre.wrap.so, but dlsym couldn’t find it. When calling dlsym on the *runtime-dlhandle* which is a handle for the main program (the java process).

The solution is to load the lib.gleefre.wrap.so library during the startup, adding a lambda to the *init-hooks*, so that SBCL could find external symbols in the wrapper.

dlsym-2

It later turned out that on older androids loading the wrapper library would load a second copy instead of finding the handle to the already loaded library.

This was fixed by getting the handle in the wrapper, passing it to lisp through a custom pair of functions from an additional SBCL patch (pass_pointer_to_lisp and get-pointer-from-c) (see this branch from my fork of SBCL).

float-traps

This is a problem mostly on x86_64 and not on arm64 (not sure why).

When initializing, SBCL enables float traps. That being said, it doesn’t disable them when initialize_lisp finishes. Most C libraries assume that float traps are disabled, which leads to crashes due to SIGFPE being signaled (this might happen, for example, in harfbuzz_ng.so).

The workaround is to disable float traps manually in the *init-hooks*. This means that float traps will be disabled in all of the lisp callbacks as well, so enabling them for the duration of the callback might be a good idea.

Setting up Android SDK & Android NDK & emulator.

This section describes how to set up command line tools from the Android SDK without installing Android Studio.

It requires quite a bit of free space on the disk (about 2-5 GB).

Download

You need to download Android SDK from the Android Studio’s download page. The link is in the bottom of the page.

After the extraction you will also need to restructure your folders. Command line tools are not bundled properly for some reason.

The desired structure is

.
├── android-sdk
│   └── cmdline-tools
│   │   └── tools
|   |   |   └── ...

Here is a little script that might be useful. It automatically downloads and restructures the folders.

mkdir android-sdk && cd android-sdk
# This link might be broken in the future.
# Replace it or download the .zip archive manually
wget https://dl.google.com/android/repository/commandlinetools-linux-13114758_latest.zip
unzip commandlinetools-linux-13114758_latest.zip
rm commandlinetools-linux-13114758_latest.zip
mv cmdline-tools tools
mkdir cmdline-tools
mv tools cmdline-tools

Environment variables

Next step is to setup your environment variables.

Here is a script for that. (To be loaded with source.) It must be placed next to the android-sdk folder.

# setup.sh script
unset $CDPATH
SCRIPT_DIR="$(cd $(dirname ${BASH_SOURCE[0]}) >/dev/null 2>&1 && pwd)"

# These are required for command line tools.
export ANDROID_HOME="$SCRIPT_DIR/android-sdk"
export ANDROID_SDK_ROOT="$ANDROID_HOME"

# These tell emulator to store its data NOT in the $USER_HOME/.android folder.
# Unfortunately, some other tools will still store their data there.
export ANDROID_USER_HOME="$SCRIPT_DIR/.android"
export ANDROID_EMULATOR_HOME="$ANDROID_USER_HOME"
export ANDROID_AVD_HOME="$ANDROID_USER_HOME/avd"

# Used by SBCL / C wrapper build scripts.
export NDK="$ANDROID_HOME/ndk/28.1.13356709"

export PATH="$PATH:$ANDROID_HOME/cmdline-tools/tools/bin"  # sdkmanager and avdmanager
export PATH="$PATH:$ANDROID_HOME/platform-tools"  # adb
export PATH="$PATH:$ANDROID_HOME/emulator"  # emulator

# That sets up tools used during the build of the .apk file.
# You might want to replace the version.
export PATH="$PATH:$ANDROID_HOME/build-tools/34.0.0-rc3"

You can also put it into .bashrc for conveniece.

# in your .bashrc
source "path/to/the/setup.sh"

To install / find packages for Android SDK use sdkmanager program.

Needed packages

You need to install platform-tools, platforms;android-36 and build-tools;30.0.3 packages from sdkmanager:

sdkmanager --install "platform-tools" "platforms;android-36" "build-tools;36.0.0"

NDK

Install it with sdkmanager:

sdkmanager --install "ndk;28.1.13356709"

Emulator

Install it with sdkmanager:

sdkmanager --install "emulator"

To run the emulator you need to create an Android Virtual Device first.

Install the needed packages with sdkmanager:

# You can choose other versions.
# The SDK version (33 here) must be the same.
# See sdkmanager --list
sdkmanager --install "system-images;android-33;google_apis;x86_64"
sdkmanager --install "platforms;android-33"

Create the AVD with avdmanager:

# You can use another name (-n flag).
# You can use different device (--device flag), list possible devices with
#   avdmanager list device
avdmanager -s create avd -f -n image \
           -k "system-images;android-33;google_apis;x86_64" \
           -p $ANDROID_AVD_HOME \
           --device "pixel_4"

Run the emulator:

emulator @image

And you can connect to the shell with ADB:

adb shell

Compiling the project

Note: all build scripts must be called from the root of the project.

Requirements

  • Java version 21 It is needed for the gradle 8.14 (used as build system).
  • An android device connected with ADB. You can use an emulator instead. It is required to build the c/lisp/java code yourself, but SBCL is already built. You can forcefully recompile it by setting the SBCL_REBUILD env variable.

Simple

Just run make-all.sh to build everything:

./build-scripts/make-all.sh

To only build the .apk file you can use make-java.sh or gradlew. This does not require a connected android device.

./build-scripts/make-java.sh
# or
./gradlew assembleDebug

If the SBCL is already present on the connected device, and libsbcl.so is copied to prebuilt/libs together with other libraries used by SBCL, you can use make-app.sh to skip the compilation and copying of SBCL.

./build-scripts/make-app.sh

You can also use gradlew to install the apk via ADB on the connected device:

./gradlew installDebug

More control

All build files are located in the build/ directory, both those that are used by gradle and those that aren’t. At the same time, prebuilt/libs is used for native libraries to be included in the app, and libsbcl.so, libzstd.so, lib.gleefre.core.so and lib.gleefre.wrap.so are copied there during the build process.

The compilation steps are the following:

  • SBCL

    First the SBCL is compiled and copied over to the device. This is done by the build-scripts/make-sbcl.sh script. It automatically detects if a prebuilt SBCL exists, in which case it only takes care of copying shared libraries used by SBCL to prebuilt/libs and copying SBCL to the device.

    When compiling SBCL, the sbcl-android-upd-pptl branch of my fork is used.

    The shared libraries and headers from prebuilt/sbcl-android-libs are copied to android-libs. This includes libzstd (can be compiled with this script) and its headers that are used for sb-core-compression. You can also compile other libraries for SBCL to use and put them there; namely libgmp, libmpfr, and libcapstone for the corresponding sbcl contribs.

  • Core

    Then the lisp code is compiled and saved into a non-executable core. This is done by the build-scripts/make-core.sh script. This requires SBCL to be installed on the connected android device.

  • C

    The the C wrapped is compiled into a shared library and copied to prebuilt/libs. This is done by the build-scripts/make-c.sh script. This does not require a connected android device.

    Requires the NDK environment variable to be set.

  • Java (.apk file)

    The the Java frontend is compiled with gradlew into an apk that also includes all shared libraries from prebuilt/libs. This is done by the build-scripts/make-jave.sh script. This does not require a connected android device.

Bugs & Contributions

Feel free to report bugs or make suggestions by filing an issue on github.

Feel free to submit pull requests on github as well.

License

Copyright 2023-2025 Gleefre

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

About

SBCL *works* on android as part of an application!

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published