C++11 support on Android with g++ 4.8

Summary

The C++11 support on Android when using g++ 4.8 is pretty good. The important concurrency features that were missing from g++ 4.6 are now available in the Standard Template Library.

Testing libgnustl_shared.so

After examining the LLVM libc++ library on Android, I wanted to check which C++11 features are currently available using g++ 4.8 and the GNU Standard Template Library (libstdc++). g++ 4.8 improved support for C++11 significantly (4.6 vs. 4.8), so the question was whether that support also made it into the Android-specific compilers. With the LLVM libc++ library, a lot of very specific platform code such as support for atomic operations was stubbed out, which would cascade down into problems with the mutual exclusion mechanisms and threads.

Based on a review of an existing codebase, the features most interesting to me were contained in the following headers: atomic, chrono, condition_variable (condition_variable), memory (shared_ptr, unique_ptr), mutex (lock_guard, mutex, unique_lock), thread (thread). Also of interest was compiler support for lambdas and auto keyword support.

So let’s see what’s available, by modifying the android-ndk-r9/samples/test-libstdc++ native executable to utilize a number of C++11 features to perform the same race condition-prone task. Namely, the program generates a random number of threads which each attempt to simultaneously increment some global variable by one, one hundred times.

C++ gives you a few ways to solve the problem, and I’ve tried each of them here. The logic of the incrementation isn’t exactly identical, and in some cases incredibly inefficient (the conditional_variable case basically has all threads waiting for a single thread to complete, before they can acquire a shared resource mutex and begin executing). In any case, this is more a proof of library completeness than a proof of algorithmic efficiency.

The test source code looks like this:

I added an Application.mk file to the jni folder. Debugging is activated, so that the assert() macro isn’t disabled.

APP_ABI    := armeabi
APP_STL    := gnustl_shared
APP_CFLAGS := --std=c++11
APP_OPTIM  := debug

NDK_TOOLCHAIN_VERSION := 4.8

Building works fine:

$ ndk-build
Compile++ thumb  : test-libstl /sources/cxx-stl/gnu-libstdc++/4.8/libs/armeabi/
Executable     : test-libstl
Install        : test-libstl => libs/armeabi/test-libstl
Install        : libgnustl_shared.so => libs/armeabi/libgnustl_shared.so
$ adb push libs/armeabi/libgnustl_shared.so /data/tmp
366 KB/s (804484 bytes in 2.143s)
$ adb push libs/armeabi/test-libstl /data/tmp
284 KB/s (50536 bytes in 0.173s)

After pushing the test-libstl executable to the device, make sure the /data/tmp folder is added to the LD_LIBRARY_PATH:

localhost tmp # LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/data/tmp

Then just run the executable, and you should see:

localhost tmp # ./test-libstl 
std::atomic + std::random + std::thread + auto + ranged-for test
  -- std::random rolled: 11
  -- atomicInt final value: 1100
std::condition_variable + std::mutex + std::unique_lock + std::random + std::thread + lambda test
  -- std::random rolled: 15
  -- nonAtomicInt final value: 1500
std::lock_guard + std::mutex + std::random + std::thread + lambda test
  -- std::random rolled: 7
  -- lockGuardInt final value: 700
std::unique_ptr + std::shared_ptr + std::make_shared tests
  --  PtrTest(): std::unique_ptr
  --  PtrTest(): std::shared_ptr
  -- ~PtrTest(): std::shared_ptr
  -- ~PtrTest(): std::unique_ptr

To run the tests, I used a rooted phone running Android 2.3.7.

In real deployments where you’re building a loadable shared-library using the Java Native Interface, you’ll need to follow the instructions from the NDK’s CPLUSPLUS-SUPPORT.html file and call System.loadLibrary on libgnustl_shared.so before loading the eventual shared library form of your app. (Managing standalone executable lifecycles tends to be a pain in the ass inside of Android apps and services, at least based on the experiences we had at my previous company.)

Overall, I’m pretty positive about the support for concurrency in g++ 4.8 on Android, and am thrilled to see that it may now be possible to write a single modern C++ codebase across all of the major desktop and mobile platforms.

Mucking around with libc++ on Android

tl;dr

libc++ on Android doesn’t work yet as of October 2013.

Getting the test suite running

The LLVM project-created libc++ library isn’t yet fully supported on Android, but I wanted to see how much of its test suite would currently work on an Android phone. Considering the weaknesses of all the other C++ Standard Template Libraries out there, libc++ becoming available would be a huge boon to developing a modern codebase. There are a few notes about this in one long-running feature request, and according to the git logs there are maybe 2 people actually working full-time on the NDK at Google, but there’s not much else to note.

Easier said than done, as usual, and it’s not really in Google’s interest to get this working, since everyone is supposed to write things specificially in Java. But it is the one stumbling block that prevents a company from writing their core logic a single time in modern C++11 (with conveniences like support for atomics, threads, conditional variables, and mutexes) and having that run across all existing desktop and mobile platforms. I’ll have to take a look at whether the g++ 4.8 STL support is complete enough to cover the C++11 cases I want to use. Everyone else (Microsoft, Apple) has a usable STL implementation on their desktop and mobile platforms. A functioning libc++ might also let people avoid pulling in the entirety of a library like Qt to plug the gaps, and as it is Qt is in an alpha stage of support for Android and iOS.

Anyway, these are some of the notes I took while getting the test suite going on a rooted smartphone.

Clone the latest Google NDK repository sources in $HOME:

git clone https://android.googlesource.com/platform/ndk git-ndk

Grab a copy of the prepackaged NDK from the Android developer website, at the time of this writing version r9. Unpack this in $HOME.

Create a symbolic link from sources/cxx-stl/llvm-libc++ folder in the Git checkout to the sources/cxx-stl folder in the officially packaged NDK folder. This saves you from having to build the cross-compilers yourself.

ln -s git-ndk/sources/cxx-stl/llvm-libc++ android-ndk-r9/sources/cxx-stl/llvm-libc++

Edit the android-ndk-r9/sources/cxx-stl/llvm-libc++/test/../../gabi++/include/gabixx_config.h file to use the __typeof__ operator instead of typeof, because of Problems 3 and 4 (detailed below) encountered in --std=c++11 mode:

After making this change, go to the android-ndk-r9/tests/device/test-libc++ folder and try building the test libc++ executable, but limit the build to armeabi, otherwise the ndk-build will try building 8 versions of the libc++ libraries (static, shared) x (armeabi, armeabi-v7a, mips, x86):

android-ndk-r9/tests/device/test-libc++$ ndk-build APP_ABI=armeabi

The code compiles, but there’s still a ton of stuff in the library not yet implemented, and you’ll see a lot of #warning Not implemented messages, wherever functionality has been stubbed out.

[...]
Compile++ thumb  : c++_static  libs/armeabi/test_1_static

OK, so the library builds, now to get the testit_android test runner running in android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test:

Set up the libc++ library include path to look like:

The sysroot setting needs to be fixed up in the testit_android script, in this case I set it to the android-9 platform, which reaches back to roughly the Android 2.2 release:

Add ANDROID_SYSROOT to the two compilation commands:

Now connect a rooted Android phone to your development computer.

Running testit_android should work, first pushing a copy of the libc++ shared library to the smartphone, then compiling, pushing, and running the test executables on the smartphone:

android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test$ ./testit_android --abi=armeabi
passed 1 tests in /home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test
passed 1 tests in /home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test/algorithms
passed 1 tests in /home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test/algorithms/alg.c.library
passed 1 tests in /home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test/algorithms/alg.modifying.operations
[...]

Unfortunately, after running several thousand tests for over an hour, testit_android seemed to hang when attempting to run tests within the support.dynamic test suite:

[...]
failed 1 tests in /home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test/language.support/support.dynamic/alloc.errors/set.new.handler
passed 1 tests in /home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test/language.support/support.dynamic/alloc.errors/set.new.handler

real	68m19.512s
user	0m0.004s
sys	0m0.000s

I removed the llvm-libc++/libcxx/test/support.dynamic test suite and reran testit_android, which resulted in the full suite completing:

****************************************************
Results for /home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test:
using arm-linux-androideabi-g++ (GCC) 4.8
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
with -std=c++11 -I/home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test/support -I/home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/include -I/home/builder/android-ndk-r9/sources/android/support/include -L/home/builder/android-ndk-r9/tests/device/test-libc++/obj/local/armeabi
----------------------------------------------------
sections without tests   : 0
sections with failures   : 199
sections without failures: 861
                       +   ----
total number of sections : 1060
----------------------------------------------------
number of tests failed   : 683
number of tests passed   : 3716
                       +   ----
total number of tests    : 4399
****************************************************

real	141m23.980s
user	67m18.716s
sys	21m46.932s

As usual, your mileage may vary, but this is the state of libc++ on Android in October 2013.

Notes

1. How about using Clang, since it should work better?

According to the NDK page, you can specify the compiler to use via the NDK_TOOLCHAIN_VERSION variable:

“Fixed the ndk-build script to enable you to specify a version of Clang as a command line option (e.g., NDK_TOOLCHAIN_VERSION=clang3.2). Previously, only specifying the version as an environment variable worked.”

So this works too:

ndk-build NDK_TOOLCHAIN_VERSION=clang3.3 APP_ABI=armeabi

And produces output like:

[...]
Compile++ thumb  : c++_shared 

The problem is that the testit_android script is currently hard-coded to use arm-linux-androideabi-g++ to build the test suite executables. The LLVM compiler that ships with Android NDK r9 seems to be an all-in-one cross-compiler, which can generate code for ARM, MIPS, and x86 depending on which config.mk files are supplied to configure the current build. So getting testit_android to build ARM executables with Clang would be a bit more work, though probably not that much more work.

2. Relation between the Android.mk files currently

android-ndk-r9/tests/device/test-libc++/jni/Android.mk (immediately imports test executable makefile) -> 
android-ndk-r9/sources/cxx-stl/llvm-libc++/test/Android.mk (builds the test executables) -> 
android-ndk-r9/sources/cxx-stl/llvm-libc++/Android.mk (builds the actual libc++ static and shared libraries)

Problems encountered

1. Attempting to build the tests in the Git working directory (git-ndk) without first creating the symbolic link in the release NDK directory (android-ndk-r9)

git-ndk/tests/device/test-libc++$ ndk-build
jni/Android.mk:1: /home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/test/Android.mk: No such file or directory
jni/Android.mk:1: /home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/test/Android.mk: No such file or directory
jni/Android.mk:1: /home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/test/Android.mk: No such file or directory
jni/Android.mk:1: /home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/test/Android.mk: No such file or directory
/home/builder/android-ndk-r9/build/core/build-all.mk:89: Android NDK: WARNING: There are no modules to build in this project!    
make: *** No rule to make target `/home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/test/Android.mk'.  Stop.

2. Attempting to build the tests by specifying NDK_ROOT with ndk-build on the PATH

git-ndk/tests/device/test-libc++$ ndk-build NDK_ROOT=$HOME/git-ndk
Android NDK: Could not find platform files (headers and libraries)    
Android NDK: Please run build/tools/build-platforms.sh to build the corresponding directory.    
/home/builder/git-ndk/build/core/init.mk:471: *** Android NDK: Aborting    .  Stop.

Problems 1 and 2 are solved by creating the symbolic link as described above and running the ndk-build command from within the android-ndk-r9 folder structure.

3. Attempting to build with GNU g++ fails due to the use in GAbi++ of a GNU-specific 'typeof' keyword which isn't recognized in --std=c++11 mode

Compile++ thumb  : c++_shared 

4. Attempting to build with Clang 3.3 fails due to the use in GAbi++ of a GNU-specific 'typeof' keyword which isn't recognized in --std=c++11 mode

Compile++ thumb  : c++_shared 

typeof is a GNU compiler-specific keyword extension to the C standard, meaning that other compilers won't like it very much. Darn it. The compilation commands for g++ and clang++ use --std=c++11, which does not allow the use of the typeof keyword.

However, according to the Clang user manual:

The parser recognizes "asm" and "typeof" as keywords in gnu* modes; the variants "__asm__" and "__typeof__" are recognized in all modes.

The same appears to be true of the GNU compiler, therefore replacing typeof with __typeof__ fixes the problem on both.

Running testit_android

1. Missing prebuilt library

android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test$ ./testit_android --abi=armeabi
ERROR: Missing prebuilt library: /home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libs/armeabi/libc++_static.a
Please run: build/tools/build-cxx-stl.sh --stl=libc++

But the library is already built here:

android-ndk-r9/sources/cxx-stl/llvm-libc++/libs/armeabi/libc++_static.a

So, the testit_android script just needs to be modified, where the line:

LIBCXX_LIBS=$(cd $LIBCXX_ROOT/.. && pwd)/libs/$TARGET_ABI

Needs to be changed to:

LIBCXX_LIBS=$(cd $LIBCXX_ROOT/../../../.. && pwd)/tests/device/test-libc++/obj/local/$TARGET_ABI

2. Missing C++ compiler

./testit_android --abi=armeabi
ERROR: Missing C++ compiler: arm-linux-androideabi-g++

Now, put that compiler into the PATH:

export PATH=$HOME/android-ndk-r9/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86_64/bin:$PATH

3. Missing platform headers and libraries

The build command doesn't know which Android platform to compile and link against:

/home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test/containers/container.adaptors/queue/queue.defn/assign_move.pass.cpp failed to compile
Compile line was: ( cd /home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test/containers/container.adaptors/queue/queue.defn && arm-linux-androideabi-g++ -std=c++11 -I/home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test/support -I/home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/include -I/home/builder/android-ndk-r9/sources/android/support/include -L/home/builder/android-ndk-r9/tests/device/test-libc++/obj/local/armeabi assign_move.pass.cpp -lc++_shared )
android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test$ cd /home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test/containers/container.adaptors/queue/queue.defn && arm-linux-androideabi-g++ -std=c++11 -I/home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test/support -I/home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/include -I/home/builder/android-ndk-r9/sources/android/support/include -L/home/builder/android-ndk-r9/tests/device/test-libc++/obj/local/armeabi assign_move.pass.cpp -lc++_shared
In file included from /home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/include/queue:168:0,
                 from assign_move.pass.cpp:14:
/home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/include/__config:99:21: fatal error: endian.h: No such file or directory
 # include <endian.h>
                     ^
compilation terminated.
android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test$ cd /home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test/algorithms/alg.modifying.operations/alg.copy && arm-linux-androideabi-g++ -std=c++11 -I/home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/test/support -I/home/builder/android-ndk-r9/sources/cxx-stl/llvm-libc++/libcxx/include -I/home/builder/android-ndk-r9/sources/android/support/include -I/home/builder/android-ndk-r9/sources/android/support/../../../platforms/android-9/arch-arm/usr/include -L/home/builder/android-ndk-r9/tests/device/test-libc++/obj/local/armeabi copy_n.pass.cpp -lc++_shared
/home/builder/android-ndk-r9/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.8/../../../../arm-linux-androideabi/bin/ld: error: cannot open crtbegin_dynamic.o: No such file or directory
/home/builder/android-ndk-r9/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.8/../../../../arm-linux-androideabi/bin/ld: error: cannot open crtend_android.o: No such file or directory
/home/builder/android-ndk-r9/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.8/../../../../arm-linux-androideabi/bin/ld: error: cannot find -lstdc++
/home/builder/android-ndk-r9/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.8/../../../../arm-linux-androideabi/bin/ld: error: cannot find -lm
/home/builder/android-ndk-r9/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.8/../../../../arm-linux-androideabi/bin/ld: error: cannot find -lc
/home/builder/android-ndk-r9/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.8/../../../../arm-linux-androideabi/bin/ld: error: cannot find -ldl
/tmp/cc55fvWI.o:copy_n.pass.cpp:function void test, output_iterator >(): error: undefined reference to 'memset'
/tmp/cc55fvWI.o:copy_n.pass.cpp:function void test, output_iterator >(): error: undefined reference to '__assert2'
/tmp/cc55fvWI.o:copy_n.pass.cpp:function void test, output_iterator >(): error: undefined reference to '__assert2'
/tmp/cc55fvWI.o:copy_n.pass.cpp:function void test, input_iterator >(): error: undefined reference to 'memset'
/tmp/cc55fvWI.o:copy_n.pass.cpp:function void test, input_iterator >(): error: undefined reference to '__assert2'
/tmp/cc55fvWI.o:copy_n.pass.cpp:function void test, input_iterator >(): error: undefined reference to '__assert2'
/tmp/cc55fvWI.o:copy_n.pass.cpp:function void test, forward_iterator >(): error: undefined reference to 'memset'
/tmp/cc55fvWI.o:copy_n.pass.cpp:function void test, bidirectional_iterator >(): error: undefined reference to 'memset'
/tmp/cc55fvWI.o:copy_n.pass.cpp:function void test(): error: undefined reference to 'memmove'
collect2: error: ld returned 1 exit status

As specified above, this problem is fixed by the addition of the ANDROID_SYSROOT variable to the build command, to indicate which Android platform includes and libraries the compiler should use.

Chrome USB Web Debugging Is Awesome

I’ve been doing a little bit of tuning for mobile devices on some web work that I’d been doing, and wanted to give the Chrome USB Web Debugging a try. For those living under a rock for the past two years, Google added the ability to live-debug websites on mobile devices using the standard Android Debug Bridge and a copy of mobile Chrome. This means you can debug your websites using a real browser, running on real Android tablets and phones. And, because most mobile browsers are based on some form of Webkit (or Blink), this should lead to reasonable market coverage.

In this case, I tested it using my Nexus 7, by activating the option in mobile Chrome:

usb-web-debugging-3-small

After installing the ADB plugin for Google Chrome on the Desktop, the following extension icon shows up in the browser window:

usb-web-debugger-4

Once you’ve started ADB via the plugin menu, you can then “View Inspection Targets” and the following list of inspection targets will open up:

usb-web-debugging-5

Inspecting the Chrome instance on the Nexus 7 device will open up the familiar Developer Tools window, acting on the mobile device target:

usb-web-debugging-1

Inspecting a DOM node on the desktop, highlights it on the mobile device, like you’re used to:

usb-web-debugging-2-small

The great thing is that the debugging session is seamless. You are put in full control of an interactive debugger, all of the CSS inspection and modification is there, you can modify anything you want the Javascript console, set breakpoints, inspect, modify, and continue running stopped code. It all just works, which is pretty much the nicest thing you can say about a piece of software (and incidentally the hardest thing to achieve in software engineering).

In short, it’s a great way to code, debug, and optimize a website using a real mobile web browser.

Genius-Level Software Developers

Unfortunately, there aren’t more of them around. For anyone who has ever wondered why their data plan gets chewed up so quickly, look no further than the idiots who program smartphone apps that force you to opt-out of 3G / 4G data synchronization over the phone networks:

gallery-idiots

Because I really wanted my multi-year-old photos from Picasa, which I’m about to delete, to sync down onto my phone over HSPA. Thanks guys. I really like it when a program punishes me for its bad behavior. Of the top-three data consumers on my phone, the top two had better just be email and web, because that is the stuff I actually care about and need immediate, high-speed access to. Also: Why do I want to sync these photos anyway? Isn’t that what the cloud is for? So I don’t have to lug around a full-sized (!) local copy, if I don’t want?

opt-out

So they’re smart enough to know that I might mind, but not smart enough to explicitly ask me whether or not I want to burn up my high-speed data quota first.

Straightforward Android Native Executables

I spent some time trying to figure out how to build a native code “Hello, world!” program for Android and noticed that prior to the r4b release of the Android NDK, there is a lot of bad info out there on how to do this. Usually, it involves using some kind of unofficial cross-compiler, plus the entire source tree of the Android OS, and/or whatever other kludgy hacks like linking the executable statically to some kind of ARM libc. This seems bad. With the newest version of the NDK generating a proper gdbserver executable and gdb setup stubs, why anyone would want to do this by hand is beyond me.

There was a script that made sense at one point: agcc.pl (here), which essentially takes all of the Makefile fragments and command-line switches and wraps them for you, letting you treat the NDK’s cross-compilers almost like a normal gcc, but this is probably still more work than it’s worth.

The Easy Way

The easiest way to get the NDK to build a native executable for you is just to use

at the end of your Android.mk, and then build like normal.

Anything else will just cause premature baldness.

Read on to see a fully-worked example…
Continue reading Straightforward Android Native Executables