Building OpenJDK 8 on Mac OS X Mavericks

tech

java make gcc cmake apple openjdk mavericks osx homebrew xcode ffs

So You Have Set Out To Build OpenJDK

Everyone enjoys some good Adventure from time to time. One of my latest is building OpenJDK on Mac OS X Mavericks. This article is a mix of building instructions and my Adventure Log. If you are not interested in the latter, there’s a TL;DR version available. I cannot guarantee that it will work in every single case, however. You may or may not encounter the same issues as I have. To have a better understanding of what’s going on I recommend you to at least skim the whole of it.

Getting Started

The first thing we have to do is get the source code. All you need is a mercurial installation. It can be installed from homebrew, which is the awesome package manager for OSX in case you were not aware of the fact. If you are not using it, you should probably start doing so (especially if you are a MacPorts user)… But anyway, we have some cloning to indulge in.

$ hg clone http://hg.openjdk.java.net/jdk8/jdk8
[...]
$ cd jdk8
$ bash ./get_source.sh
[...]

The most recent revision at the time this article was written was 933:4f8fa4724c14. If you find yourself unable to reproduce some of the problems, the chances are that things have been fixed. Now, if we inspect our shiny new sourcebase, we will observe that in the root directory there lies a file named README-builds.html, which contains the detailed instructions on how to build OpenJDK.

What we need to ensure is that we have:

  • GNU make of version 3.81 or newer. Run make -version to check that.
  • A so-called Bootstrap JDK. The required version is ≥ 1.7.0_7 (but not JDK 8).

Now, the platform-specific things come. The funny thing is that the manual has around 10 lines for Linux, around 40 for Solaris, and a couple of pages for Windows. For OS X, however, we have just two sentences. Here they are:

Install Xcode 4.5.2 and also install the “Command line tools” found under the preferences pane “Downloads”.

Make sure you get the right Xcode version.

Sounds very promising and easy, right? What could possibly go wrong?

All You Had To Do Was Follow The Damn Train, CJ!

Well, you might already have Xcode of the wrong version installed. The likelihood of this increases greatly if you have a new device, which comes with Mavericks by default. And when you require Xcode for the first time, the OS is going to helpfully offer you to install the latest version. Which is 5.0.2 at the time of writing this article.

Nah, why would Apple break backwards compatibility? We should just try it regardless, nobody will get hurt either way:

$ bash ./configure
[...]

checking for gcc... /usr/bin/gcc
configure: Resolving CC (as /usr/bin/gcc) failed, using /usr/bin/gcc directly.
checking resolved symbolic links for CC... /usr/bin/gcc
checking if CC is disguised ccache... no, keeping CC
configure: The C compiler (located as /usr/bin/gcc) does not seem to be the required GCC compiler.
configure: The result from running with --version was: "Configured with: --prefix=/Applications/Xcode-5.0.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1"
configure: error: GCC compiler is required. Try setting --with-tools-dir.
configure exiting with result code 1

Oh. Still, it was worth trying. Come to think of it, it’s actually a little bit embarrassing. All the manual has asked us was to have the right version of Xcode, and we failed at that. In our defense, we could say that it should be possible to build with the latest version. The problem is that Xcode 5 uses clang and has no support for gcc anymore. OpenJDK has plans to add clang support in the future, and work is being done. Allegedly. If someone is in fact working hard on the issue, please do not take offense. I appreciate it, I really do.

Uh. Anyway, what we could do is roll back to an older version of Xcode, but hey, that is hardly an acceptable solution. And even if it were, the process is in fact no walk in the park either.

A more proper approach would be to install gcc separately, and us it instead of clang:

$ brew install gcc48
[...]

$ sudo mkdir /usr/bin/backup && sudo mv /usr/bin/gcc /usr/bin/g++ /usr/bin/backup
$ sudo ln -s /usr/local/bin/gcc-4.8 /usr/bin/gcc
$ sudo ln -s /usr/local/bin/g++-4.8 /usr/bin/g++
$ bash ./configure
[...]

configure: error: Could not find freetype!
configure exiting with result code 1

Freetype is a library that renders fonts. It is also very conveniently shipped with XQuartz. Just go to their site and download its latest version. After the installation is complete, we can have another go at it:

$ bash ./configure
[...]

A new configuration has been successfully created in
jdk8/build/macosx-x86_64-normal-server-release
using default settings.

Yay! We can now move on to building the JDK.

Just A Couple Of Fixes Here And There And Also There And There

$ make
[...]

jdk8/hotspot/src/os/bsd/vm/os_bsd.cpp:1150:7: error: "__FreeBSD__" is not defined [-Werror=undef]
jdk8/hotspot/src/os/bsd/vm/os_bsd.cpp:1152:7: error: "__OpenBSD__" is not defined [-Werror=undef]
jdk8/hotspot/src/os/bsd/vm/os_bsd.cpp:1154:7: error: "__NetBSD__" is not defined [-Werror=undef]
[...]

Alright, if it looks like a bug and breaks builds, then it should probably be fixed. Here’s a patch for this:

Let’s apply it and try again:

$ pushd hotspot
$ curl https://gist.githubusercontent.com/gvsmirnov/8634644/raw > os_bsd_cpp_defined_fix.patch
$ hg import os_bsd_cpp_defined_fix.patch
applying os_bsd_cpp_defined_fix.patch
$ popd
$ make
[...]
Making SA debugger back-end...

jdk8/hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m: In function 'Java_sun_jvm_hotspot_debugger_bsd_BsdDebuggerLocal_lookupByName0':
jdk8/hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m:213:1: error: '-fobjc-exceptions' is required to enable Objective-C exception syntax
[...]

This has been running for a number of minutes before the error showed up. I was almost beginning to hope it would work. But alas, there are more things to fix. According to the manual, this options enables Objective-C compiler to use the @try, @throw, @catch, @finally and @synchronized keywords. I wonder why it is off by default. The forums say that there are good reasons, so there probably are. Whatever, I am not an Objective-C guy, so it’s not my call to judge. My call to enable this thing, though!

It should be as easy as setting the OBJCFLAGS="-fobjc-exceptions" env variable, but it is not. Doing so does not resolve the issue, so we have do go deeper.

$ grep -rn "Making SA debugger back-end" .
./build/macosx-x86_64-normal-server-release/build.log:670:Making SA debugger back-end...
./hotspot/make/bsd/makefiles/saproc.make:117:       @echo Making SA debugger back-end...
./hotspot/make/linux/makefiles/saproc.make:83:      @echo Making SA debugger back-end...
./hotspot/make/solaris/makefiles/saproc.make:98:    @echo Making SA debugger back-end...

What we are looking for is saproc.make for bsd. Here’s an excerpt from it:

$(LIBSAPROC): $(SASRCFILES) $(SAMAPFILE)
    $(QUIETLY) if [ "$(BOOT_JAVA_HOME)" = "" ]; then \
      echo "ALT_BOOTDIR, BOOTDIR or JAVA_HOME needs to be defined to build SA"; \
      exit 1; \
    fi
    @echo Making SA debugger back-end...
    $(QUIETLY) $(CC) -D$(BUILDARCH) -D_GNU_SOURCE                   \
               $(SYMFLAG) $(SAARCH) $(SHARED_FLAG) $(PICFLAG)       \
               -I$(SASRCDIR)                                        \
               -I$(GENERATED)                                       \
               $(BOOT_JAVA_INCLUDES)                                \
               $(SASRCFILES)                                        \
               $(SA_LFLAGS)                                         \
               $(SA_DEBUG_CFLAGS)                                   \
               -o $@                                                \
               $(SALIBS)

By adding some @echo invocations and trying to make again, we can see what actually gets executed, and even establish what values which variables have:

/usr/local/Cellar/gcc48/4.8.2/bin/gcc-4.8 -Damd64 -D_GNU_SOURCE -g -m64

    # $(SHARED_FLAG)
    -Wl,-install_name,@rpath/libsaproc.dylib -dynamiclib -compatibility_version 1.0.0 -current_version 1.0.0 -fPIC

    # $(PICFLAG)
    -fPIC

    # $(SASRCDIR)
    -Ijdk8/hotspot/agent/src/os/bsd

    # $(GENERATED)
    -I../generated

    # $(BOOT_JAVA_INCLUDES)
    -I/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/include
    -I/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/include/darwin
    -I/System/Library/Frameworks/JavaVM.framework/Headers

    # $(SASRCFILES)
    jdk8/hotspot/agent/src/os/bsd/symtab.c
    jdk8/hotspot/agent/src/os/bsd/libproc_impl.c
    jdk8/hotspot/agent/src/os/bsd/ps_core.c
    jdk8/hotspot/agent/src/os/bsd/MacosxDebuggerLocal.m
    jdk8/hotspot/agent/src/share/native/sadis.c

    -o libsaproc.dylib

    # $(SALIBS)
    -g
    -framework Foundation -F/System/Library/Frameworks/JavaVM.framework/Frameworks
    -framework JavaNativeFoundation -framework Security -framework CoreFoundation

Both $(SA\_LFLAGS) and $(SA\_DEBUG\_CFLAGS) are empty, despite the fact that our magic -fobjc-exceptions flag should have been set in one of them. A closer inspection of the file shows us that the latter is set within the script, and is either -g or empty; and the former is loaded from $(LDFLAGS\_HASH\_STYLE). Which is, in turn, is set… nowhere. Cool!

$ pushd hotspot
$ curl https://gist.githubusercontent.com/gvsmirnov/8664413/raw > saproc_make_fobjc_exceptions_flag_fix.patch
$ hg import saproc_make_fobjc_exceptions_flag_fix.patch
applying private_extern_fix.patch
$ popd
$ make
[...]

jdk8/jdk/src/solaris/native/java/io/io_util_md.c:46:1: error: unknown type name '__private_extern__'
 __private_extern__
 ^
jdk8/jdk/src/solaris/native/java/io/io_util_md.c:47:9: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'newStringPlatform'
 jstring newStringPlatform(JNIEnv *env, const char* str)
         ^
[...]

Apparently, \_\_private\_extern\_\_ has been deprecated, and we should use \_\_attribute\_\_((visibility("hidden"))) instead. Dr. Deprecator is very happy, and so are we. Yay! More patching!

$ pushd jdk
$ curl https://gist.githubusercontent.com/gvsmirnov/8664662/raw > private_extern_fix.patch
$ hg import private_extern_fix.patch
applying private_extern_fix.patch
$ popd
$ make
[...]

/System/Library/Frameworks/Foundation.framework/Headers/NSUserNotification.h:16:45: error: expected ',' or '}' before '__attribute__'
     NSUserNotificationActivationTypeReplied NS_AVAILABLE(10_9, NA) = 3
                                             ^
[...]

Now, this is getting a bit ridiculous. Working around this one requires us to patch a file outside of OpenJDK. It is discussed in some detail in this thread.

# In /System/Library/Frameworks/Foundation.framework/Headers/NSUserNotification.h:16

      NSUserNotificationActivationTypeActionButtonClicked = 2,
-     NSUserNotificationActivationTypeReplied NS_AVAILABLE(10_9, NA) = 3
+     NSUserNotificationActivationTypeReplied /*NS_AVAILABLE(10_9, NA)*/ = 3
} NS_ENUM_AVAILABLE(10_8, NA);

$ make
[...]

In file included from /System/.../JavaNativeFoundation.framework/Headers/JavaNativeFoundation.h:11:0,
                 from jdk8/jdk/src/macosx/native/apple/applescript/AppleScriptEngine.m:29:
jdk8/jdk/src/macosx/native/apple/applescript/AppleScriptEngine.m: In function 'Java_apple_applescript_AppleScriptEngine_createContextFrom':
/System/.../JNFJNI.h:51:6: error: '-fobjc-exceptions' is required to enable Objective-C exception syntax
[...]

Really? Again? Ugh, whatever. Here’s yet another patch.

$ pushd jdk
$ curl https://gist.githubusercontent.com/gvsmirnov/8718663/raw > platform_libraries_fobjc_exceptions_fix.patch
$ hg import platform_libraries_fobjc_exceptions_fix.patch
applying platform_libraries_fobjc_exceptions_fix.patch
$ popd
$ make
[...]

/Users/gvsmirnov/sources/jdk8/jdk/src/macosx/native/sun/osxapp/ThreadUtilities.h:132:60: error: expected ')' before '^' token
 + (void)performOnMainThreadWaiting:(BOOL)wait block:(void (^)())block;
                                                            ^
[...]
/Users/gvsmirnov/sources/jdk8/jdk/src/macosx/native/sun/osxapp/ThreadUtilities.m:53:62: error: expected ')' before '(' token
 + (void)performOnMainThreadWaiting:(BOOL)wait block:(void (^)())block {
                                                            ^
</pre>

Now, what kind of syntax is that?! Apparently, the block syntax, which is not officially supported by gcc.
Apple has patches for it, so we need to either patch our gcc (yikes!), or get the appropriate binaries. The "yikes!"
option is all the more yikes if one knows that Apple only has patches for gcc 4.2. I do happen to know, so it definitely is the binaries.


$ brew install apple-gcc42
[...]

[... don't forget to make symlinks ...]

$ bash ./configure
$ make clean && make
[...]

----- Build times -------
Start 2014-02-06 21:20:36
End   2014-02-06 21:30:50
00:00:18 corba
00:06:57 hotspot
00:00:13 jaxp
00:00:18 jaxws
00:02:03 jdk
00:00:25 langtools
00:10:14 TOTAL
-------------------------
Finished building OpenJDK for target 'default'

$ build/macosx-x86_64-normal-server-release/jdk/bin/java -version
openjdk version "1.8.0-internal"
OpenJDK Runtime Environment (build 1.8.0-internal-gvsmirnov_2014_02_06_21_20-b00)
OpenJDK 64-Bit Server VM (build 25.0-b67, mixed mode)




Wheeeee! Actually, hold on a second! Did we *really* need to make all those changes? Would it work right
out of the box with the right compiler? But that's a different story to tell.


# TL;DR

## If you have XCode ≤ 4.*

$ hg clone http://hg.openjdk.java.net/jdk8/jdk8
$ cd jdk8 && bash ./get_source.sh

$ bash ./configure && make
$ ./build/macosx-x86_64-normal-server-release/jdk/bin/java -version


That's it!

## If you have XCode 5

1. Install [XQuartz](http://xquartz.macosforge.org/landing/)
2. Install apple-gcc42 from homebrew
3. Make symlinks: /usr/bin/gcc -> /usr/local/Cellar/apple-gcc42/4.2.1-5666.3/bin/gcc-4.2 (same for g++)
4. Get the sources (see above)
5. Make sure bash ./configure displays no errors
4. Apply patches: [1](https://gist.githubusercontent.com/gvsmirnov/8634644), [2](https://gist.githubusercontent.com/gvsmirnov/8664413),
   [3](https://gist.githubusercontent.com/gvsmirnov/8664662), [4](https://gist.githubusercontent.com/gvsmirnov/8718663)
5. Comment out NS\_AVAILABLE(10\_9, NA) in line 16 of
   /System/Library/Frameworks/Foundation.framework/Headers/NSUserNotification.h
6. Run make
7. Enjoy!

# An afterword
What has happened here is a dirty workaround. OpenJDK should support XCode 5. It most likely will, but probably
not in a while. My bet is that when Java 9 development hits an active phase, its support will be implemented, and then
backported to OpenJDK 8. Until then, we can build with an older version of XCode, or do some tricks. I hope mine work
for everyone. If they do not, feel free to comment below!
comments powered by Disqus

CC0 Freely use anything from this website in any way you can imagine