• 编译加速 remote cache


    remote cache
    大致原理就是根据编译输入的sha码与编译输出建立(key,value)对,存储在远端cache.编译过程如果能匹配到key,则获取GET到value.
    如果hit不到key,则客户端在本地编译,并控制是否更新远端缓存库.
    https://blog.csdn.net/periodtang/article/details/80249769
    Bazel Remote Caching实践
    https://github.com/buchgr/bazel-remote/
    (
    https://golang.org/pkg/net/http/
    ) KEY: bazel-remote/server/http.go
    .......
    docker run -d -v `pwd`/bazel_cache/:/data -p 8080:8080 buchgr/bazel-remote-cache --max_size=20

      build:remote --experimental_strict_action_env --remote_http_cache=http://192.168.1.102:8080
      build:remote --experimental_execution_log_file=/tmp/exec1.log
      build:remote --build_event_text_file=/tmp/bep.txt
      build:remote --remote_upload_local_results=false
      build:remote_cache --remote_upload_local_results=true

      [3,541 / 3,565] 12 actions, 7 running
      ...lation_world:simulation_world_service; 0s remote-cache
      .../backend/sim_control:sim_control; 0s remote-cache
      ...lation_world:simulation_world_service; 0s remote-cache
      Compiling .../dreamview/backend/main.cc; 0s remote-cache
      ...lation_world:simulation_world_updater; 0s remote-cache
      ...lation_world:simulation_world_updater; 0s remote-cache
      .../handlers:websocket_handler_test; 0s remote-cache
      [-----] .../backend/handlers:image_handler_cpplint ...

      ...  

      INFO: Elapsed time: 68.628s, Critical Path: 8.32s
      INFO: 1212 processes: 1212 remote cache hit.
      INFO: Build completed successfully, 3700 total actions
      modules/perception/proto/perception_lane.proto: warning: Import modules/common/proto/error_code.proto but not used.
      1============================
      [ OK ] Build passed!
      [INFO] Took 234 seconds
      ============================

    ===================================================================================================
    distribute compilation 

      Some conclusions:
      Buildfarm: Now it maybe still exists much problems. The work plan of project Buildfarm https://github.com/bazelbuild/bazel/projects/1.
      Buildbarn: The implement of Buildfarm in go. So see Buildfarm.
      BuildGrid: This project is an open source proj by python. It seems may work for us, and not try yet.
      distcc: This proj may be mature, but cannot use in bazel directly.And the bazel team suggests that remote execution has made huge leaps in the past year and a half and should be ready for you to try as a replacement of distcc.

      In a word, try distcc or BuildGrid, and wait for Buildfarm.

    在所有您想用来编译的机器上安装 distcc
    启动每台机器上的 distcc 新进程
    用变量名导出 DISTCC_HOSTS 环境变量
    启动 distcc 监视器(这样就可以观察正在发生什么事!)
    使用 CC=distcc ./configure ,而不是使用 ./configure 来进行配置
    使用 make -j n ,而不是使用 make 或 make -j 2 来进行编译,其中,n 是 DISTCC_HOSTS中机器数目的两到三倍
    link
    https://github.com/bazelbuild/bazel/issues/3875
    
    https://github.com/bazelbuild/bazel-buildfarm
    
    https://github.com/distcc/distcc/blob/master/INSTALL

  • 相关阅读:
    APDU:APDU常用指令
    其他:IntelliJ IDEA设置运行内存
    Linux:获取Linux离线postgresql数据库安装包并部署
    资源:Postgresql数据库下载路径
    免费版:Xshell和Xftp下载路径
    Raspberry Pi:树莓派开发板配置USB启动系统
    Raspberry Pi:树莓派安装基础系统 Raspberry Pi(树莓派系统)
    Raspberry Pi:树莓派安装Kali2021新版本
    Docker:虚拟机挂起后,再启动docker容器连接失败
    Docker:docker搭建redis一主多从集群(配置哨兵模式)
  • 原文地址:https://www.cnblogs.com/cjyp/p/10622545.html
Copyright © 2020-2023  润新知