| =================================================================== |
| How to Cross Compile Compiler-rt Builtins For Arm |
| =================================================================== |
| |
| Introduction |
| ============ |
| |
| This document contains information about building and testing the builtins part |
| of compiler-rt for an Arm target, from an x86_64 Linux machine. |
| |
| While this document concentrates on Arm and Linux the general principles should |
| apply to other targets supported by compiler-rt. Further contributions for other |
| targets are welcome. |
| |
| The instructions in this document depend on libraries and programs external to |
| LLVM, there are many ways to install and configure these dependencies so you |
| may need to adapt the instructions here to fit your own local situation. |
| |
| Prerequisites |
| ============= |
| |
| In this use case we'll be using CMake on a Debian-based Linux system, |
| cross-compiling from an x86_64 host to a hard-float Armv7-A target. We'll be |
| using as many of the LLVM tools as we can, but it is possible to use GNU |
| equivalents. |
| |
| * ``A build of LLVM/clang for the llvm-tools and llvm-config`` |
| * ``The qemu-arm user mode emulator`` |
| * ``An arm-linux-gnueabihf sysroot`` |
| |
| See https://compiler-rt.llvm.org/ for more information about the dependencies |
| on clang and LLVM. |
| |
| ``qemu-arm`` should be available as a package for your Linux distribution. |
| |
| The most complicated of the prequisites to satisfy is the arm-linux-gnueabihf |
| sysroot. The :doc:`HowToCrossCompileLLVM` has information about how to use the |
| Linux distributions multiarch support to fulfill the dependencies for building |
| LLVM. Alternatively, as building and testing just the compiler-rt builtins |
| requires fewer dependencies than LLVM, it is possible to use the Linaro |
| arm-linux-gnueabihf gcc installation as our sysroot. |
| |
| Building compiler-rt builtins for Arm |
| ===================================== |
| We will be doing a standalone build of compiler-rt using the following cmake |
| options. |
| |
| * ``path/to/llvm/projects/compiler-rt`` |
| * ``-DCOMPILER_RT_BUILD_BUILTINS=ON`` |
| * ``-DCOMPILER_RT_BUILD_SANITIZERS=OFF`` |
| * ``-DCOMPILER_RT_BUILD_XRAY=OFF`` |
| * ``-DCOMPILER_RT_BUILD_LIBFUZZER=OFF`` |
| * ``-DCOMPILER_RT_BUILD_PROFILE=OFF`` |
| * ``-DCMAKE_C_COMPILER=/path/to/clang`` |
| * ``-DCMAKE_AR=/path/to/llvm-ar`` |
| * ``-DCMAKE_NM=/path/to/llvm-nm`` |
| * ``-DCMAKE_RANLIB=/path/to/llvm-ranlib`` |
| * ``-DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld"`` |
| * ``-DCMAKE_C_COMPILER_TARGET="arm-linux-gnueabihf"`` |
| * ``-DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON`` |
| * ``-DLLVM_CONFIG_PATH=/path/to/llvm-config`` |
| * ``-DCMAKE_C_FLAGS="build-c-flags"`` |
| |
| The build-c-flags need to be sufficient to pass the C-make compiler check and |
| to compile compiler-rt. When using a GCC 7 Linaro arm-linux-gnueabihf |
| installation the following flags are needed: |
| |
| * ``--target=arm-linux-gnueabihf`` |
| * ``--march=armv7a`` |
| * ``--gcc-toolchain=/path/to/dir/toolchain`` |
| * ``--sysroot=/path/to/toolchain/arm-linux-gnueabihf/libc`` |
| |
| Depending on how your sysroot is laid out, you may not need ``--gcc-toolchain``. |
| For example if you have added armhf as an architecture using your Linux |
| distributions multiarch support then you should be able to use ``--sysroot=/``. |
| |
| Once cmake has completed the builtins can be built with ``ninja builtins`` |
| |
| Testing compiler-rt builtins using qemu-arm |
| =========================================== |
| To test the builtins library we need to add a few more cmake flags to enable |
| testing and set up the compiler and flags for test case. We must also tell |
| cmake that we wish to run the tests on ``qemu-arm``. |
| |
| * ``-DCOMPILER_RT_EMULATOR="qemu-arm -L /path/to/armhf/sysroot`` |
| * ``-DCOMPILER_RT_INCLUDE_TESTS=ON`` |
| * ``-DCOMPILER_RT_TEST_COMPILER="/path/to/clang"`` |
| * ``-DCOMPILER_RT_TEST_COMPILER_CFLAGS="test-c-flags"`` |
| |
| The ``/path/to/armhf/sysroot`` should be the same as the one passed to |
| ``--sysroot`` in the "build-c-flags". |
| |
| The "test-c-flags" can be the same as the "build-c-flags", with the addition |
| of ``"-fuse-ld=lld`` if you wish to use lld to link the tests. |
| |
| Once cmake has completed the tests can be built and run using |
| ``ninja check-builtins`` |
| |
| Modifications for other Targets |
| =============================== |
| |
| Arm Soft-Float Target |
| --------------------- |
| The instructions for the Arm hard-float target can be used for the soft-float |
| target by substituting soft-float equivalents for the sysroot and target. The |
| target to use is: |
| |
| * ``-DCMAKE_C_COMPILER_TARGET=arm-linux-gnueabi`` |
| |
| Depending on whether you want to use floating point instructions or not you |
| may need extra c-flags such as ``-mfloat-abi=softfp`` for use of floating-point |
| instructions, and ``-mfloat-abi=soft -mfpu=none`` for software floating-point |
| emulation. |
| |
| AArch64 Target |
| -------------- |
| The instructions for Arm can be used for AArch64 by substituting AArch64 |
| equivalents for the sysroot, emulator and target. |
| |
| * ``-DCMAKE_C_COMPILER_TARGET=aarch64-linux-gnu`` |
| * ``-DCOMPILER_RT_EMULATOR="qemu-aarch64 -L /path/to/aarch64/sysroot`` |
| |
| The CMAKE_C_FLAGS and COMPILER_RT_TEST_COMPILER_CFLAGS may also need: |
| ``"--sysroot=/path/to/aarch64/sysroot --gcc-toolchain=/path/to/gcc-toolchain"`` |
| |
| Armv6-m, Armv7-m and Armv7E-M targets |
| ------------------------------------- |
| If you wish to build, but not test compiler-rt for Armv6-M, Armv7-M or Armv7E-M |
| then the easiest way is to use the BaremetalARM.cmake recipe in |
| clang/cmake/caches. |
| |
| You will need a bare metal sysroot such as that provided by the GNU ARM |
| Embedded toolchain. |
| |
| The libraries can be built with the cmake options: |
| |
| * ``-DBAREMETAL_ARMV6M_SYSROOT=/path/to/bare/metal/sysroot`` |
| * ``-DBAREMETAL_ARMV7M_SYSROOT=/path/to/bare/metal/sysroot`` |
| * ``-DBAREMETAL_ARMV7EM_SYSROOT=/path/to/bare/metal/sysroot`` |
| * ``-C /path/to/llvm/source/tools/clang/cmake/caches/BaremetalARM.cmake`` |
| |
| **Note** that for the recipe to work the compiler-rt source must be checked out |
| into the directory llvm/runtimes and not llvm/projects. |
| |
| To build and test the libraries using a similar method to Armv7-A is possible |
| but more difficult. The main problems are: |
| |
| * There isn't a ``qemu-arm`` user-mode emulator for bare-metal systems. The ``qemu-system-arm`` can be used but this is significantly more difficult to setup. |
| * The target to compile compiler-rt have the suffix -none-eabi. This uses the BareMetal driver in clang and by default won't find the libraries needed to pass the cmake compiler check. |
| |
| As the Armv6-M, Armv7-M and Armv7E-M builds of compiler-rt only use instructions |
| that are supported on Armv7-A we can still get most of the value of running the |
| tests using the same ``qemu-arm`` that we used for Armv7-A by building and |
| running the test cases for Armv7-A but using the builtins compiled for |
| Armv6-M, Armv7-M or Armv7E-M. This will not catch instructions that are |
| supported on Armv7-A but not Armv6-M, Armv7-M and Armv7E-M. |
| |
| To get the cmake compile test to pass the libraries needed to successfully link |
| the test application will need to be manually added to ``CMAKE_CFLAGS``. |
| Alternatively if you are using version 3.6 or above of cmake you can use |
| ``CMAKE_TRY_COMPILE_TARGET=STATIC_LIBRARY`` to skip the link step. |
| |
| * ``-DCMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY`` |
| * ``-DCOMPILER_RT_OS_DIR="baremetal"`` |
| * ``-DCOMPILER_RT_BUILD_BUILTINS=ON`` |
| * ``-DCOMPILER_RT_BUILD_SANITIZERS=OFF`` |
| * ``-DCOMPILER_RT_BUILD_XRAY=OFF`` |
| * ``-DCOMPILER_RT_BUILD_LIBFUZZER=OFF`` |
| * ``-DCOMPILER_RT_BUILD_PROFILE=OFF`` |
| * ``-DCMAKE_C_COMPILER=${host_install_dir}/bin/clang`` |
| * ``-DCMAKE_C_COMPILER_TARGET="your *-none-eabi target"`` |
| * ``-DCMAKE_AR=/path/to/llvm-ar`` |
| * ``-DCMAKE_NM=/path/to/llvm-nm`` |
| * ``-DCMAKE_RANLIB=/path/to/llvm-ranlib`` |
| * ``-DCOMPILER_RT_BAREMETAL_BUILD=ON`` |
| * ``-DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON`` |
| * ``-DLLVM_CONFIG_PATH=/path/to/llvm-config`` |
| * ``-DCMAKE_C_FLAGS="build-c-flags"`` |
| * ``-DCMAKE_ASM_FLAGS="${arm_cflags}"`` |
| * ``-DCOMPILER_RT_EMULATOR="qemu-arm -L /path/to/armv7-A/sysroot"`` |
| * ``-DCOMPILER_RT_INCLUDE_TESTS=ON`` |
| * ``-DCOMPILER_RT_TEST_COMPILER="/path/to/clang"`` |
| * ``-DCOMPILER_RT_TEST_COMPILER_CFLAGS="test-c-flags"`` |
| |
| The Armv6-M builtins will use the soft-float ABI. When compiling the tests for |
| Armv7-A we must include ``"-mthumb -mfloat-abi=soft -mfpu=none"`` in the |
| test-c-flags. We must use an Armv7-A soft-float abi sysroot for ``qemu-arm``. |
| |
| Unfortunately at time of writing the Armv7-M and Armv7E-M builds of |
| compiler-rt will always include assembler files including floating point |
| instructions. This means that building for a cpu without a floating point unit |
| requires something like removing the arm_Thumb1_VFPv2_SOURCES from the |
| arm_Thumb1_SOURCES in builtins/CMakeLists.txt. The float-abi of the compiler-rt |
| library must be matched by the float abi of the Armv7-A sysroot used by |
| qemu-arm. |
| |
| Depending on the linker used for the test cases you may encounter BuildAttribute |
| mismatches between the M-profile objects from compiler-rt and the A-profile |
| objects from the test. The lld linker does not check the BuildAttributes so it |
| can be used to link the tests by adding -fuse-ld=lld to the |
| ``COMPILER_RT_TEST_COMPILER_CFLAGS``. |
