LLVM Dependency

Overview

SwiftShader's Reactor library uses LLVM as one of its JIT-compiler backends. This page contains notes about building and upgrading LLVM.

Directory structure

The current version of LLVM we use is 10, and can be found in third_party/llvm-10.0.

In this folder you will find the following directories:

  • configs : Contains per-platform headers that LLVM sources include to configure the build. These are generated by running scripts/update.py (more on that below).
  • llvm : Contains a subset of the LLVM source code needed to build the JIT support required by SwiftShader.
  • scripts : Contains update.py, which is used to update the files in the configs folder. More on that below.

Updating the current version of LLVM to latest

Updating to the latest version of LLVM can be tricky to do manually, especially because the llvm-project repo includes much more than just LLVM (e.g. it includes all the Clang source). Furthermore, we may have local changes to our copy of LLVM that must be maintained, or at least considered across updates.

To ease this pain, run the script third_party/update-llvm-10.sh on Linux. This script works by updating a separate branch of SwiftShader, llvm10-clean, on which the latest snapshot of LLVM is fetched and committed, and then this branch is merged back into master. During the merge, if there are conflicts to resolve because of local changes we've made, these can be resolved in the usual manner, and the merge can be resumed.

The script is configured to fetch from the branch in LLVM_REPO_BRANCH, and will automatically grab the latest commit on that branch.

Although not always necessary, if there were new configuration variables added or modified, you may need to run update.py as described below. Otherwise, if all goes well, the update to LLVM can be committed and pushed.

Updating LLVM configuration files

The script third_party/llvm-10.0/scripts/update.py is used to update the config files in third_party/llvm-10.0/configs.

Before running this script, you must make sure to update two variables in it (and commit this change):

# LLVM_BRANCH must match the value of the same variable in third_party/update-llvm-10.sh
LLVM_BRANCH = "release/10.x"

# LLVM_COMMIT must be set to the commit hash that we last updated to when running third_party/update-llvm-10.sh.
# Run 'git show -s origin/llvm10-clean' and look for 'llvm-10-update: <hash>' to retrieve it.
LLVM_COMMIT = "d32170dbd5b0d54436537b6b75beaf44324e0c28"

The script takes a platform as argument, and extra CMake args. For example, to update the Linux configs, run:

python3 update.py linux -j 200

This script does the following:

  • Clones the LLVM repo and checks out LLVM_COMMIT from LLVM_BRANCH.
  • Builds LLVM specifically for the target architectures specified in LLVM_TRIPLES dictionary.
  • Copies the specified platform config files to third_party/llvm-10.0/configs, applying certain transformations to the files, such as undefining macros listed in LLVM_UNDEF_MACROS (see the copy_platform_file function).

Note that certain configuration options depend on the host OS, you will need to run the script on the right host OS. See the LLVM_PLATFORM_TO_HOST_SYSTEM dictionary for the mapping, which looks like this at the time of this writing:

# Mapping of target platform to the host it must be built on
LLVM_PLATFORM_TO_HOST_SYSTEM = {
    'android': 'Linux',
    'darwin': 'Darwin',
    'linux': 'Linux',
    'windows': 'Windows',
    'fuchsia': 'Linux'
}

Generally, Windows to build Window, Darwin to build Darwin (MacOS), and Linux for everything else. Also note that for android and fuchsia, the config is closest to that of Linux, but you will likely have to manually tweak the configs (in particular, configs/<platform>/include/llvm/Config/config.h).

Supported platforms, architectures, and build systems

SwiftShader is used by many products on many architectures:

  • OS: Windows, Linux, MacOS, Android, Fuchsia
  • Architecture: x64, x86, ARM, ARM64, MIPS, MIPS64
  • Build systems: CMake, GN, Soong, Blaze

Upgrading/updating LLVM usually entails making sure it builds for all of these.