JVM之编译OpenJDK

学习JVM时看到书里讲到自己编译OpenJDK。记录一下过程

• Mac系统版本：High Sierra 10.13.6
• 源码版本：jdk8u-dev

一、准备源码

我是从官网下载网站的，openJDK源码是用mercurial进行管理的，所以首先使用homebrew安装mercurial

brew install mercurial

安装完成之后，再运行命令克隆jdk源码，之后通过运行脚本get_source.sh获取所有的源代码

hg clone http://hg.openjdk.java.net/jdk8u/jdk8u-dev/
cd jdk8u-dev
sh ./get_source.sh 

在获取源码的过程中可能会遇到一些问题，如下所示。我遇到的问题主要是网络的问题，后来通过切换网络，使用手机热点解决了。

abort: stream ended unexpectedly (got 1596 bytes, expected 10164)

二、设置环境变量

在编译开始之前需要设置一些变量。这里与书中所讲的有一些不一样，只因为我用的是Mac平台，所以编译器之类的可能会有一些不一样。可以将下面的内容拷贝建立一个脚本运行。

这里的环境变量设置采用了博客https://www.jianshu.com/p/d9a1e1072f37中的设置

# 语言选项
export LANG=C
# Mac平台，C编译器不再是GCC，是clang
export CC=clang
# 跳过clang的一些严格的语法检查，不然会将N多的警告作为Error
export COMPILER_WARNINGS_FATAL=false
# 链接时使用的参数
export LFLAGS='-Xlinker -lstdc++'
# 是否使用clang
export USE_CLANG=true
# 使用64位数据模型
export LP64=1
# 告诉编译平台是64位，不然会按32位来编译
export ARCH_DATA_MODEL=64
# 允许自动下载依赖
export ALLOW_DOWNLOADS=true
# 并行编译的线程数，编译时间长，为了不影响其他工作，我选择为2
export HOTSPOT_BUILD_JOBS=2
# 是否跳过与先前版本的比较
export SKIP_COMPARE_IMAGES=true
# 是否使用预编译头文件，加快编译速度
export USE_PRECOMPILED_HEADER=true
# 是否使用增量编译
export INCREMENTAL_BUILD=true
# 编译内容
export BUILD_LANGTOOLS=true
export BUILD_JAXP=true
export BUILD_JAXWS=true
export BUILD_CORBA=true
export BUILD_HOTSPOT=true
export BUILD_JDK=true
# 编译版本
export SKIP_DEBUG_BUILD=true
export SKIP_FASTDEBUG_BUILD=false
export DEBUG_NAME=debug
# 避开javaws和浏览器Java插件之类的部分的build
export BUILD_DEPLOY=false
export BUILD_INSTALL=false
unset JAVA_HOME

三、编译

设置好环境变量之后，就可以开始编译了，首先运行configure，这个脚本会调用很多其他的脚本进行配置并检查编译环境。

bash ./configure

运行成功后的显示如下：

A new configuration has been successfully created in
/Users/zhengshuangxi/mercurial/temp/build/macosx-x86_64-normal-server-release
using default settings.

Configuration summary:
* Debug level:    release
* JDK variant:    normal
* JVM variants:   server
* OpenJDK target: OS: macosx, CPU architecture: x86, address length: 64

Tools summary:
* Boot JDK:       java version "1.8.0_212" Java(TM) SE Runtime Environment (build 1.8.0_212-b10) Java HotSpot(TM) 64-Bit Server VM (build 25.212-b10, mixed mode)  (at /Library/Java/JavaVirtualMachines/jdk1.8.0_212.jdk/Contents/Home)
* Toolchain:      gcc (GNU Compiler Collection)
* C Compiler:     Version Apple LLVM version 10.0.0 (clang-1000.11.45.5) Target: x86_64-apple-darwin17.7.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin (at /usr/bin/clang)
* C++ Compiler:   Version Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1 Apple LLVM version 10.0.0 (clang-1000.11.45.5) Target: x86_64-apple-darwin17.7.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin (at /usr/bin/g++)

Build performance summary:
* Cores to use:   2
* Memory limit:   8192 MB

也有可能会遇到一些问题，这里我遇到的问题有：

• Xcode版本不符合，需要Xcode 4
• 无法找到gcc编译器

具体解决办法在本文最后提到。如果没有遇到问题就可以进一步编译了。

直接使用make all命令进行编译

make all

编译成功之后显示如下：

## Finished docs (build time 00:02:31)

----- Build times -------
Start 2019-06-21 16:43:05
End   2019-06-21 16:57:48
00:00:27 corba
00:00:55 demos
00:02:31 docs
00:03:43 hotspot
00:01:34 images
00:00:15 jaxp
00:00:24 jaxws
00:04:10 jdk
00:00:31 langtools
00:00:12 nashorn
00:14:43 TOTAL
-------------------------
Finished building OpenJDK for target 'all'

在编译过程中还可能出现一些问题，目前我只遇到这一个问题：

• 参数错误“-std=gnu++98”，具体解决办法在本文最后提到。

四、运行

编译完成之后，在build文件夹中会生成一个名为macosx-x86_64-normal-server-release的文件夹，这里就是我们编译生成的结果。在这个文件夹中jdk就是OpenJDK编译成功的产物。

我们可以通过如下命令运行虚拟机，查看版本号，如果没什么问题终端会显示如下信息：

cd ./build/macosx-x86_64-normal-server-release/jdk/bin
./java -version
openjdk version "1.8.0-internal"
OpenJDK Runtime Environment (build 1.8.0-internal-zhengshuangxi_2019_06_21_17_25-b00)
OpenJDK 64-Bit Server VM (build 25.71-b00, mixed mode)

运行之后我又遇到一个问题，在运行过程中会报错，提示 libjvm.dylib 动态链接库相关的错误。具体解决在本文最后。

五、遇到的问题及解决办法

5.1、运行configure之后，提示Xcode版本问题

checking for xcodebuild... /usr/bin/xcodebuild
configure: error: Xcode 4 is required to build JDK 8, the version found was 10.1. Use --with-xcode-path to specify the location of Xcode 4 or make Xcode 4 active by using xcode-select.
configure exiting with result code 1

我们只需要将文件 jdk8u-dev/common/autoconf/generated-configure.sh 进行一些修改，找到以下代码并进行注释，代码大约在26780行左右。注释之后就不会提示版本问题了

    # Fail-fast: verify we're building on Xcode 4, we cannot build with Xcode 5 or later
    XCODE_VERSION=`$XCODEBUILD -version | grep '^Xcode ' | sed 's/Xcode //'`
    XC_VERSION_PARTS=( ${XCODE_VERSION//./ } )
    if test ! "${XC_VERSION_PARTS[0]}" = "4"; then
      as_fn_error $? "Xcode 4 is required to build JDK 8, the version found was $XCODE_VERSION. Use --with-xcode-path to specify the location of Xcode 4 or make Xcode 4 active by using xcode-select." "$LINENO" 5
    fi

5.2、gcc编译器找不到

configure: Will use user supplied compiler CC=clang
checking for clang... /usr/bin/clang
checking resolved symbolic links for CC... no symlink
configure: The C compiler (located as /usr/bin/clang) does not seem to be the required gcc compiler.
configure: The result from running with --version was: ""
configure: error: A gcc compiler is required. Try setting --with-tools-dir.
configure exiting with result code 1

解决办法也是在generated-configure.sh文件中注释掉以下的条件代码。注释的地方一共有两处：

# 第一处代码，27936行左右
  elif test  "x$TOOLCHAIN_TYPE" = xgcc; then
    # gcc --version output typically looks like
    #     gcc (Ubuntu/Linaro 4.8.1-10ubuntu9) 4.8.1
    #     Copyright (C) 2013 Free Software Foundation, Inc.
    #     This is free software; see the source for copying conditions.  There is NO
    #     warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
    COMPILER_VERSION_OUTPUT=`$COMPILER --version 2>&1`
    # Check that this is likely to be GCC.
    $ECHO "$COMPILER_VERSION_OUTPUT" | $GREP "Free Software Foundation" > /dev/null
# 条件语句注释掉
#     if test $? -ne 0; then
#       { $as_echo "$as_me:${as_lineno-$LINENO}: The $COMPILER_NAME compiler (located as $COMPILER) does not seem to be the required $TOOLCHAIN_TYPE compiler." >&5
# $as_echo "$as_me: The $COMPILER_NAME compiler (located as $COMPILER) does not seem to be the required $TOOLCHAIN_TYPE compiler." >&6;}
#       { $as_echo "$as_me:${as_lineno-$LINENO}: The result from running with --version was: "$COMPILER_VERSION"" >&5
# $as_echo "$as_me: The result from running with --version was: "$COMPILER_VERSION"" >&6;}
#       as_fn_error $? "A $TOOLCHAIN_TYPE compiler is required. Try setting --with-tools-dir." "$LINENO" 5
#     fi

# 第二处代码，29677行左右
  elif test  "x$TOOLCHAIN_TYPE" = xgcc; then
    # gcc --version output typically looks like
    #     gcc (Ubuntu/Linaro 4.8.1-10ubuntu9) 4.8.1
    #     Copyright (C) 2013 Free Software Foundation, Inc.
    #     This is free software; see the source for copying conditions.  There is NO
    #     warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
    COMPILER_VERSION_OUTPUT=`$COMPILER --version 2>&1`
    # Check that this is likely to be GCC.
    $ECHO "$COMPILER_VERSION_OUTPUT" | $GREP "Free Software Foundation" > /dev/null
# 条件语句注释掉
#     if test $? -ne 0; then
#       { $as_echo "$as_me:${as_lineno-$LINENO}: The $COMPILER_NAME compiler (located as $COMPILER) does not seem to be the required $TOOLCHAIN_TYPE compiler." >&5
# $as_echo "$as_me: The $COMPILER_NAME compiler (located as $COMPILER) does not seem to be the required $TOOLCHAIN_TYPE compiler." >&6;}
#       { $as_echo "$as_me:${as_lineno-$LINENO}: The result from running with --version was: "$COMPILER_VERSION"" >&5
# $as_echo "$as_me: The result from running with --version was: "$COMPILER_VERSION"" >&6;}
#       as_fn_error $? "A $TOOLCHAIN_TYPE compiler is required. Try setting --with-tools-dir." "$LINENO" 5
#     fi

注释完之后，重新运行configure

5.3、make过程中遇到参数错误“-std=gnu++98”

Making signal interposition lib...
error: invalid argument '-std=gnu++98' not allowed with 'C'
make[6]: *** [libjsig.dylib] Error 1
make[6]: *** Waiting for unfinished jobs....
make[5]: *** [the_vm] Error 2
make[4]: *** [product] Error 2
make[3]: *** [generic_build2] Error 2
make[2]: *** [product] Error 2
make[1]: *** [/Users/zhengshuangxi/mercurial/temp/build/macosx-x86_64-normal-server-release/hotspot/_hotspot.timestamp] Error 2
make: *** [hotspot-only] Error 2

在generated-configure.sh文件中找到参数“-std=gnu++98”，将代码注释如下：

      LDFLAGS_JDK="$LDFLAGS_JDK -Wl,-z,relro"
      LEGACY_EXTRA_LDFLAGS="$LEGACY_EXTRA_LDFLAGS -Wl,-z,relro"
    fi
    此处注释掉
    #CXXSTD_CXXFLAG="-std=gnu++98"

之后重新运行configure，再执行make all

5.4、编译时遇到 lstdc++ 库无法找到

ld: library not found for -lstdc++
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[6]: *** [libsaproc.dylib] Error 1
make[6]: *** Waiting for unfinished jobs....
make[5]: *** [the_vm] Error 2
make[4]: *** [product] Error 2
make[3]: *** [generic_build2] Error 2
make[2]: *** [product] Error 2
make[1]: *** [/Users/zhengshuangxi/mercurial/temp/build/macosx-x86_64-normal-server-release/hotspot/_hotspot.timestamp] Error 2
make: *** [hotspot-only] Error 2

这个原因是Xcode升级到10以后就没有包含lstdc++库了，解决办法是将/usr/lib中的库文件拷贝到Xcode相应的文件夹中，并设置相关的软连接。

sudo cp /usr/lib/libstdc++.6.0.9.dylib /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib/
cd /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib/
sudo ln -s libstdc++.6.0.9.dylib libstdc++.6.dylib
sudo ln -s libstdc++.6.dylib libstdc++.dylib

然后重新运行make all命令进行编译。

5.5、编译完成之后，运行虚拟机时出现错误

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGILL (0x4) at pc=0x000000010fe801bf, pid=48116, tid=0x0000000000002203
#
# JRE version: OpenJDK Runtime Environment (8.0) (build 1.8.0-internal-zhengshuangxi_2019_06_21_16_42-b00)
# Java VM: OpenJDK 64-Bit Server VM (25.71-b00 mixed mode bsd-amd64 compressed oops)
# Problematic frame:
# V  [libjvm.dylib+0x4801bf]  PerfDataManager::destroy()+0xab
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /Users/zhengshuangxi/mercurial/temp/build/macosx-x86_64-normal-server-release/jdk/bin/hs_err_pid48116.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#

[error occurred during error reporting , id 0x4]

Abort trap: 6

根据这个错误提示，我们可以知道错误发生在动态链接库 libjvm.dylib中，根据提示，错误代码时PerDataManager::destroy()函数中。因此我们需要对这个函数进行修改，即文件 hotspot/src/share/vm/runtime/perfData.cpp，在这个文件中将代码"delete p"注释掉。

问题的解决办法是在博客https://www.jianshu.com/p/34c8a8c37169中看到的。

// 大约在287行
void PerfDataManager::destroy() {

  if (_all == NULL)
    // destroy already called, or initialization never happened
    return;

  for (int index = 0; index < _all->length(); index++) {
    PerfData* p = _all->at(index);
    // 将delete p注释掉
    //delete p;
  }

  delete(_all);
  delete(_sampled);
  delete(_constants);

  _all = NULL;
  _sampled = NULL;
  _constants = NULL;
}

然后重新运行configure以及make all进行编译。之后就不会出现问题了。