• CMake库搜索函数居然不搜索LD_LIBRARY_PATH


    摘要: 本文通过编译后运行找不到库文件的问题引入,首先分析了find_package(JNI)的工作流程,而后针对cmake不搜索LD_LIBRARY_PATH的问题,提出了一种通用的解决办法。

    本文分享自华为云社区《CMake库搜索函数居然不搜索LD_LIBRARY_PATH? 由编译工具使用体验而引发的思考》,作者: 蜉蝣与海 。

    最近产品要使用JNI技术,CMake编译C++代码时需要对外链接libjvm.so库。代码编译倒是正常,系统中也有libjvm.so, 然而使用时却报了如下异常:

    error while loading shared libraries: libjvm.so: cannot open shared object file: No such file or directory

    这个报错表示,操作系统并没有找到libjvm.so, 我们的操作系统是从LD_LIBRARY_PATH中搜索这些动态链接库,很显然目前libjvm.so并不在这个目录下。

    问题的解决倒是简单,直接在LD_LIBRARY_PATH里加入libjvm.so的库即可。但是这却引发了我的思考:

    为什么构建时可以找到libjvm.so, 运行时却找不到呢?

    这个问题的回答,既可以有简明扼要版解释,又可以刨根问底深挖。

    先来看简明扼要版解释:

    代码的CMakeList中使用了下列语句,在编译过程中寻找并链接libjvm.so,这个搜索方式和操作系统的搜索方式不同:

    find_package(JNI)
    get_filename_component(JVM_LIB_PATH ${JAVA_JVM_LIBRARY} DIRECTORY)
    get_filename_component(JAVA_LIB_PATH ${JVM_LIB_PATH} DIRECTORY)
    link_directories(${JVM_LIB_PATH} ${JAVA_LIB_PATH})
    set_target_properties(${NAME} PROPERTIES LINK_FLAGS "-ljvm")

    其中find_package(JNI)会搜索libjvm.so可能存在的路径,通过get_filename_component来获得libjvm.so的文件夹,并把这个文件夹设为默认搜索库路径。而后set_target_properties会进行链接工作。

    这个答案只能告诉我们“是什么”,但是作为一只程序猿,还要了解“为什么”,这里引申几个问题讨论:

    1. find_package(JNI)的工作过程是怎样的?为什么LD_LIBRARY_PATH里没找到的依赖库,cmake可以找到
    2. cmake的库搜索函数find_library会搜索LD_LIBRARY_PATH吗,如果不会,可以通过设置来搜索LD_LIBRARY_PATH吗?

    问题一:find_package(JNI)的工作过程是怎样的

    为了方便开发者引用外部包,cmake官方预定义了许多寻找依赖包的Module, 他们存储在cmake的/share/-cmake-<version>/Modules目录下。每个以Find<LibraryName>.cmake命名的文件都可以帮我们找到一个包[1]。在本地计算机执行以下指令,即可找到find_package(JNI)使用的脚本文件。

    find / -name FindJNI.cmake

    打开自己的cmake对应的FindJNI文件,可以看到密密麻麻的注释和脚本,通过阅读这些脚本,我们得以得知FindJNI是如何工作的。

    分析问题前,先看问题带来的结果,文件最上方注释有如下说明:

    This module sets the following result variables:
    ``JNI_INCLUDE_DIRS``
     the include dirs to use
    ``JNI_LIBRARIES``
      the libraries to use (JAWT and JVM)
    ``JNI_FOUND``
      TRUE if JNI headers and libraries were found.
    Cache Variables
    ^^^^^^^^^^^^^^^
    The following cache variables are also available to set or use:
    ``JAVA_AWT_LIBRARY``
      the path to the Java AWT Native Interface (JAWT) library
    ``JAVA_JVM_LIBRARY``
      the path to the Java Virtual Machine (JVM) library
    ``JAVA_INCLUDE_PATH``
     the include path to jni.h
    ``JAVA_INCLUDE_PATH2``
     the include path to jni_md.h and jniport.h
    ``JAVA_AWT_INCLUDE_PATH``
     the include path to jawt.h

    这段代码表明,执行find_package(JNI)之后,会有一系列变量被设置,其中包括表示JNI是否被找到的变量JNI_FOUND,以及表示libjvm.so的变量JAVA_JVM_LIBRARY。这些变量在设定之后,通过FindPackageHandleStandardArgs导出,返回调用处,FindPackageHandleStandardArgs是cmake专门用来导出变量的宏[2]:

    include(${CMAKE_CURRENT_LIST_DIR}/FindPackageHandleStandardArgs.cmake)
    FIND_PACKAGE_HANDLE_STANDARD_ARGS(JNI  DEFAULT_MSG  JAVA_AWT_LIBRARY
                                                        JAVA_JVM_LIBRARY
                                                        JAVA_INCLUDE_PATH
                                                        JAVA_INCLUDE_PATH2
                                                        JAVA_AWT_INCLUDE_PATH)

    在文件中定位JAVA_JVM_LIBRARY, 可以追踪到下述代码片段:

    foreach(search ${_JNI_SEARCHES})
     find_library(JAVA_JVM_LIBRARY ${_JNI_${search}_JVM})
     find_library(JAVA_AWT_LIBRARY ${_JNI_${search}_JAWT})
     if(JAVA_JVM_LIBRARY)
     break()
     endif()
    endforeach()

    由此可知,JAVA_JVM_LIBRARY这个变量,是通过逐个搜索${_JNI_${search}_JVM}里的文件夹进而确定JAVA_JVM_LIBRARY的。而${_JNI_${search}_JVM}相关的定义语句如图:

    set(_JNI_FRAMEWORK_JVM NAMES JavaVM)
    set(_JNI_NORMAL_JVM
      NAMES jvm
      PATHS ${JAVA_JVM_LIBRARY_DIRECTORIES}
      )

    其中JAVA_JVM_LIBRARY_DIRECTORIES中涉及了大量可能的libjvm.so存在的路径。

    set(JAVA_JVM_LIBRARY_DIRECTORIES)
    foreach(dir ${JAVA_AWT_LIBRARY_DIRECTORIES})
     list(APPEND JAVA_JVM_LIBRARY_DIRECTORIES
     "${dir}"
     "${dir}/client"
     "${dir}/server"
     # IBM SDK, Java Technology Edition, specific paths
     "${dir}/j9vm"
     "${dir}/default"
        )
    endforeach()
    set(JAVA_AWT_LIBRARY_DIRECTORIES)
    if(_JAVA_HOME)
      JAVA_APPEND_LIBRARY_DIRECTORIES(JAVA_AWT_LIBRARY_DIRECTORIES
     ${_JAVA_HOME}/jre/lib/{libarch}
     ${_JAVA_HOME}/jre/lib
     ${_JAVA_HOME}/lib/{libarch}
     ${_JAVA_HOME}/lib
     ${_JAVA_HOME}
        )
    endif()
    JAVA_APPEND_LIBRARY_DIRECTORIES(JAVA_AWT_LIBRARY_DIRECTORIES
     ${_JNI_JAVA_AWT_LIBRARY_TRIES}
      )
    foreach(_java_dir IN LISTS _JNI_JAVA_DIRECTORIES_BASE)
     list(APPEND _JNI_JAVA_AWT_LIBRARY_TRIES
     ${_java_dir}/jre/lib/{libarch}
     ${_java_dir}/jre/lib
     ${_java_dir}/lib/{libarch}
     ${_java_dir}/lib
     ${_java_dir}
      )
     list(APPEND _JNI_JAVA_INCLUDE_TRIES
     ${_java_dir}/include
      )
    endforeach()

    如上图所示,变量依赖顺序如下:

    JAVA_JVM_LIBRARY_DIRECTORIES => JAVA_AWT_LIBRARY_DIRECTORIES => _JNI_JAVA_AWT_LIBRARY_TRIES & _JAVA_HOME => _JNI_JAVA_DIRECTORIES_BASE

    最终发现JAVA_JVM_LIBRARY_DIRECTORIES变量的值,是由JAVA_HOME变量的值和_JNI_JAVA_DIRECTORIES_BASE变量的值共同决定的。而JNI_JAVA_DIRECTORY_BASE预置了大量预定义路径:

    set(_JNI_JAVA_DIRECTORIES_BASE
      /usr/lib/jvm/java
      /usr/lib/java
      /usr/lib/jvm
      /usr/local/lib/java
      /usr/local/share/java
      /usr/lib/j2sdk1.4-sun
      /usr/lib/j2sdk1.5-sun
      /opt/sun-jdk-1.5.0.04
      /usr/lib/jvm/java-6-sun
      /usr/lib/jvm/java-1.5.0-sun
      /usr/lib/jvm/java-6-sun-1.6.0.00       # can this one be removed according to #8821 ? Alex
      /usr/lib/jvm/java-6-openjdk
      /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0        # fedora
     # Debian specific paths for default JVM
      /usr/lib/jvm/default-java
     # Arch Linux specific paths for default JVM
      /usr/lib/jvm/default
     # Ubuntu specific paths for default JVM
      /usr/lib/jvm/java-11-openjdk-{libarch}    # Ubuntu 18.04 LTS
      /usr/lib/jvm/java-8-openjdk-{libarch}    # Ubuntu 15.10
      /usr/lib/jvm/java-7-openjdk-{libarch}    # Ubuntu 15.10
      /usr/lib/jvm/java-6-openjdk-{libarch}    # Ubuntu 15.10
     # OpenBSD specific paths for default JVM
      /usr/local/jdk-1.7.0
      /usr/local/jre-1.7.0
      /usr/local/jdk-1.6.0
      /usr/local/jre-1.6.0
     # SuSE specific paths for default JVM
      /usr/lib64/jvm/java
      /usr/lib64/jvm/jre
      )

    通过以上分析可以看出,JAVA_JVM_LIBRARY的搜索,依赖JAVA_HOME和大量预定义路径。

    问题二:cmake库搜索函数find_library会搜索LD_LIBRARY_PATH吗

    通过阅读Does CMake's find_library search LD_LIBRARY_PATH可以知道,find_library默认不搜索LD_LIBRARY_PATH, 并且网上也找不到让cmake搜索LD_LIBRARY_PATH的文章。

    那cmake能搜索LD_LIBRARY_PATH吗?

    答案是可以的,通过cmake获取LD_LIBRARY_PATH环境变量,并转为cmake可理解的list格式,而后注入find_library即可,代码如下:

    string(REPLACE ":" ";" RUNTIME_PATH "$ENV{LD_LIBRARY_PATH}")
    find_library(JVM_API NAMES jvm HINTS ${RUNTIME_PATH})
    if (JVM_API STREQUAL "JVM_API-NOTFOUND")
     message(WARNING "found libjvm.so only in ${JAVA_JVM_LIBRARY} but not in LD_LIBRARY_PATH. environment variable LD_LIBRARY_PATH must include its' directory.")
    endif()

    如果希望找不到这个库时编译失败,可以将WARNING改为fatal_error, 代码如下:

    string(REPLACE ":" ";" RUNTIME_PATH "$ENV{LD_LIBRARY_PATH}")
    find_library(JVM_API NAMES jvm HINTS ${RUNTIME_PATH})
    if (JVM_API STREQUAL "JVM_API-NOTFOUND")
     message(FATAL_ERROR "found libjvm.so only in ${JAVA_JVM_LIBRARY} but not in LD_LIBRARY_PATH. environment variable LD_LIBRARY_PATH must include its' directory.")
    endif()

    小结

    本文通过编译后运行找不到库文件的问题引入,首先分析了find_package(JNI)的工作流程,而后针对cmake不搜索LD_LIBRARY_PATH的问题,提出了一种通用的解决办法。

    参考文献:

    [1] Cmake之深入理解find_package()的用法:https://zhuanlan.zhihu.com/p/97369704?utm_source=wechat_session

    [2] Cmake中find_package命令的搜索模式之模块模式(Module mode):https://www.jianshu.com/p/f983a90bcf91

    [3]Does CMake's find_library search LD_LIBRARY_PATH?:https://stackoverflow.com/questions/41566316/does-cmakes-find-library-search-ld-library-path

     

    点击关注,第一时间了解华为云新鲜技术~

  • 相关阅读:
    Quartz2.0以上版本的单机和集群
    Mysql的Haproxy反向代理和负载均衡
    spring AOP原理解析
    Restful接口调用方法超详细总结
    mysql数据库主从同步
    数据备份的OSS接口
    读取properties配置文件的方法
    算法学习——堆排序(二叉树排序)
    回溯算法的实现
    冒泡排序
  • 原文地址:https://www.cnblogs.com/huaweiyun/p/16534610.html
Copyright © 2020-2023  润新知