[administrator] rpmbuild

rpmbuild

1.  rpm是什么

   RPM = RPM Package Manager = Redhat Package Manager

https://en.wikipedia.org/wiki/Rpm_(software)

2. two forks

　　1. RPM.org  redhat 维护的，

This version is used by distributions such as Fedora, Red Hat Enterprise Linux and derivatives, openSUSE, SUSE Linux Enterprise, Unity Linux, Mageia[16], OpenEmbedded, Tizen and OpenMandriva Lx(formerly Mandriva).

　　2.  RPM V5  红帽前雇员维护

This version is used by distributions such as Wind River Linux (until Wind River Linux 10), Rosa Linux, and OpenMandriva Lx (former Mandriva Linux which switched to rpm5 in 2011[17]) and also by the 
OpenPKG project which provides packages for other common UNIX-platforms. OpenMandriva Lx is going to switch back to rpm.org[18] for 4.0 release.

OpenEmbedded switched back to rpm.org due to issues in RPM5.[19]

3. 官网

http://rpm.org/

4. rpm packaging文档

https://rpm-packaging-guide.github.io/

高级用法：https://rpm-packaging-guide.github.io/#triggers-and-scriptlets

5. spec file 是 rpmbuild 的配置文件，配置rpmbuild的行为，以及定制rpm包的内容。

A SPEC file can be thought of as the "recipe" that the rpmbuild utility uses to actually build an RPM. It tells the build system what to do by defining instructions in a series of sections. 
The sections are defined in the Preamble and the Body. The Preamble contains a series of metadata items that are used in the Body. The Body contains the main part of the instructions.

6。 查看各种默认宏的值

[root@D129 j-dpdk]# rpm --eval %{buildroot}
/root/rpmbuild/BUILDROOT/%{name}-%{version}-%{release}.x86_64
[root@D129 j-dpdk]# rpm --eval %{_bindir}
/usr/bin

 　　查看设置和环境变量。

[root@test ~]# rpm --showrc
ARCHITECTURE AND OS:
build arch            : x86_64
compatible build archs: x86_64 noarch
build os              : Linux
... ...

7. 安装rpmdev tools。

　　其实不装也是可以的。这个工具包主要帮助编写spec文件。和配置等？

[root@D129 j]# yum install rpmdevtools

77.  rpm包需要与systemd serivce联动。见 https://fedoraproject.org/wiki/Packaging:Systemd

8.  quick start

8.1  用rpmdev-newspec工具生成一个空 spec 模板，或者到什么地方去随便考一个过来。

[root@D129 copyright]# man rpmdev-newspec

[root@D129 j]# rpmdev-newspec 
newpackage.spec created; type minimal, rpm version >= 4.11.

8.2  改 spec文件

　　写法可以读前边提到的文档，或者靠天分自己推理。

8.3  拷贝源代码的 gz tar包 到 $ROOT/rpmbuild/SOURCES

　　要求包名与解压后的文件夹名相同，且格式为 $NAME-$VERSION

　　rpmbuild在解压，找目录的时候，根据：定义在spec文件的Source变量：Source: http://dpdk.org/browse/dpdk/snapshot/%{name}-%{version}.tar.gz

　　如果不喜欢，也可以自己定义这个过程：https://rpm-packaging-guide.github.io/#setup

8.4  使用命令开始做包：

[root@D129 pkg]# rpmbuild -ba dpdk.spec 

成功之后，生成的包，在这里：

[root@D129 pkg]# ll /root/rpmbuild/RPMS/x86_64/
total 3544
-rw-r--r-- 1 root root 2778612 Jul 13 21:14 dpdk-16.07-1.x86_64.rpm
-rw-r--r-- 1 root root  147028 Jul 13 21:14 dpdk-debuginfo-16.07-1.x86_64.rpm
-rw-r--r-- 1 root root  699036 Jul 13 21:14 dpdk-devel-16.07-1.x86_64.rpm
[root@D129 pkg]# ll /root/rpmbuild/SRPMS/
total 16548
-rw-r--r-- 1 root root 16941800 Jul 13 21:14 dpdk-16.07-1.src.rpm

9. 最终，我是为dpdk打了个轻量的rpm包，做了些裁剪定制。修改过的内容如下：

dpdk.spec

Name: xxxx-dpdk
Version: 16.07
Release: 1
Packager: packaging@6wind.com
URL: http://dpdk.org
Source: http://dpdk.org/browse/dpdk/snapshot/%{name}-%{version}.tar.gz

Summary: Data Plane Development Kit core
Group: System Environment/Libraries
License: BSD and LGPLv2 and GPLv2

ExclusiveArch: x86_64
%global target %{_arch}-native-linuxapp-gcc
%global config %{target}

BuildRequires: kernel-devel, kernel-headers

%description
DPDK core includes kernel modules, core libraries and tools.
testpmd application allows to test fast packet processing environments
on x86 platforms. For instance, it can be used to check that environment
can support fast path applications such as 6WINDGate, pktgen, rumptcpip, etc.
More libraries are available as extensions in other packages.

%package devel
Summary: Data Plane Development Kit for development
Requires: %{name}%{?_isa} = %{version}-%{release}
%description devel
DPDK devel is a set of makefiles, headers and examples
for fast packet processing on x86 platforms.

%prep
%setup -q

%build
make O=%{target} T=%{config} config
sed -ri 's,(RTE_APP_TEST=).*,1n,'         %{target}/.config
sed -ri 's,(RTE_BUILD_SHARED_LIB=).*,1y,' %{target}/.config
sed -ri 's,(RTE_NEXT_ABI=).*,1n,'         %{target}/.config
sed -ri 's,(LIBRTE_VHOST=).*,1y,'         %{target}/.config
make O=%{target} %{?_smp_mflags}

%install
rm -rf %{buildroot}
make install O=%{target} DESTDIR=%{buildroot} 
        prefix=%{_prefix} bindir=%{_bindir} sbindir=%{_sbindir} 
        includedir=%{_includedir}/dpdk libdir=%{_libdir} 
        datadir=%{_datadir}/dpdk docdir=%{_docdir}/dpdk

%files
%dir %{_datadir}/dpdk
%{_datadir}/dpdk/tools
/lib/modules/%(uname -r)/extra/*
%{_sbindir}/*
%{_bindir}/*
%{_libdir}/*

%files devel
%{_includedir}/dpdk
%{_datadir}/dpdk/mk
%{_datadir}/dpdk/scripts
%{_datadir}/dpdk/%{target}
%{_datadir}/dpdk/examples

%post
/sbin/ldconfig
/sbin/depmod

%postun
/sbin/ldconfig
/sbin/depmod

strip.sh

#! /bin/bash
#
#       Descriptions:
#               strap rpmbuild of xxxx-dpdk
#
#       created @ 2018-07-13 
#

PACKAGE_NAME="xxxx-dpdk"
DPDK_VERSION="16.07"

cd ..
TOPDIR=`rpmbuild --eval '%_topdir' 2>/dev/null`
git archive --format=tar.gz 
        --prefix=$PACKAGE_NAME-$DPDK_VERSION/ 
        -o $TOPDIR/SOURCES/$PACKAGE_NAME-$DPDK_VERSION.tar.gz HEAD
cd ./pkg

rpmbuild -ba dpdk.spec

　

所以，如果你已经读到这里了。那么你至少应该，像我一样理解了以上两个脚本中的每一行。同时能够应付日常的rpmbuild工作，或者懂了更多。

10. QA

10.1

error: Bad owner/group

+ /usr/lib/rpm/redhat/brp-python-hardlink
+ /usr/lib/rpm/redhat/brp-java-repack-jars
error: Bad owner/group: /root/Src/copyright/j/dpdk/pkg/dpdk.spec


RPM build errors:
    Bad owner/group: /root/Src/copyright/j/dpdk/pkg/dpdk.spec
[root@D129 pkg]# ll

因为我的环境是nfs，改了dpdk.spec的owner。

10.2  在编译dpdk的app程序时，由于以下错误，导致只能出正常bin包，不能出debug包。

+ /usr/lib/rpm/find-debuginfo.sh --strict-build-id -m --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 /root/rpmbuild/BUILD/jer
extracting debug info from /root/rpmbuild/BUILDROOT/jb-1.0.0-1627.el7.x86_64/usr/bin/jb
/usr/lib/rpm/debugedit -b "$RPM_BUILD_DIR" -d /usr/src/debug                          -i -l "$SOURCEFILE" $f"
/root/rpmbuild/BUILD
/root/rpmbuild/BUILD/jr/debugsources.list
/usr/lib/rpm/debugedit: canonicalization unexpectedly shrank by one character
error: Bad exit status from /var/tmp/rpm-tmp.f7o4DC (%install)

debug一下工具 debugedit

编译rpm工具的源码：

clone rpm的代码：https://github.com/rpm-software-management/rpm

[root@D129 rpm.git]# yum search nss-devel nspr-devel file-devel popt-devel

编译：

./autogen.sh CPPFLAGS="-I/usr/include/nspr4 -I/usr/include/nss3" --without-lua

很遗憾没有编译出debugedit的可执行文件，研究半天，也不知道怎么编。

第二招，通过rpm安装 debuginfo

[root@D129 rpm.git]# yum install rpm-debuginfo

步骤：

1.　　执行脚本/usr/lib/rpm/find-debuginfo.sh， 在debugedit 调用前，sleep 特别特别长的时间。

2.　　到另一个konsole，用gdb执行debugedi程序。

3. 　　设立断点，到错误的行。

结论：定位到了错误，但是不理解逻辑，所以也很难定位原因。觉得理解一下debugedit的逻辑：

参数说明：

static struct poptOption optionsTable[] = {                                                                   
    { "base-dir",  'b', POPT_ARG_STRING, &base_dir, 0,                          
      "base build directory of objects", NULL },                                                              
    { "dest-dir",  'd', POPT_ARG_STRING, &dest_dir, 0,                          
      "directory to rewrite base-dir into", NULL },                             
    { "list-file",  'l', POPT_ARG_STRING, &list_file, 0,                        
      "file where to put list of source and header file names", NULL },         
    { "build-id",  'i', POPT_ARG_NONE, &do_build_id, 0,                         
      "recompute build ID note and print ID on stdout", NULL },                 
      POPT_AUTOHELP                                                                                           
    { NULL, 0, 0, NULL, 0, NULL, NULL }                                                                       
};                

也就是说，debugedit工具利用 list-file中的信息，将dest obj中的debug信息（源文件的绝对路径）修改。将其内容从base-dir修改为dest-dir。

大概意思是这样的。所以，可以将上边提到的obj文件copy出来，每次复现。因为obj文件每次执行都会被修改，所以。第二次执行相同的文件是，结果会与第一次不同（我这里的现象是，第二次不复现错误。）

第三招

读源码，熟悉代码逻辑，继续深入debug。

通过yum安装源码：

 yumdownloader --source rpm

不行。。。。 down下来的是patch。还是要用git代码。

/usr/lib/rpm/debugedit -b "/root/rpmbuild/BUILD" -d /usr/src/debug -i -l newdebug.list vrouter 

至此，通过阅读代码，已经可以确定，是二进制程序的问题。debugedit在二进制程序的.debuginfo区检查目录名的时候，发现了它认为非法的目录名。

详见debugedit的代码：~/Src/thirdparty/rpm.git/tools/debugedit.c

 675       if (shrank > 0)                                                           
 676         {                                                                       
 677           if (--shrank == 0)                                                    
 678             error (EXIT_FAILURE, 0,                                             
 679                    "canonicalization unexpectedly shrank by one character");    
 680           else                                                                  
 681             {                                                                   
 682               memset (ptr, 'X', shrank);                                        
 683               ptr += shrank;                                                    
 684               *ptr++ = '';                                                    
 685             }                                                                   
 686         }                                                  

。。。 10个小时之后 。。。

最终。。。。 是我的spec里，在make前，设置了环境变量，环境变量的值是一个path，结尾有个 ’ / ‘， 如下图：

然后把path结尾的 ‘ / ’， 去掉就好了。

[root@D129 j-vrouter]# ldd build/vrouter
        linux-vdso.so.1 =>  (0x00007fffff9ed000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f4a54d6c000)
        libthrift-0.9.1.so => /lib64/libthrift-0.9.1.so (0x00007f4a54ac5000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f4a547bd000)
        libevent-2.0.so.5 => /lib64/libevent-2.0.so.5 (0x00007f4a54575000)
        libjemalloc.so.2 => /lib64/libjemalloc.so.2 (0x00007f4a540f0000)
        librte_kni.so.2 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_kni.so.2 (0x00007f4a53eeb000)
        librte_pipeline.so.3 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_pipeline.so.3 (0x00007f4a53ce5000)
        librte_table.so.2 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_table.so.2 (0x00007f4a53ac5000)
        librte_port.so.3 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_port.so.3 (0x00007f4a538a8000)
        librte_pdump.so.1 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_pdump.so.1 (0x00007f4a5349e000)
        librte_distributor.so.1 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_distributor.so.1 (0x00007f4a5329a000)
        librte_reorder.so.1 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_reorder.so.1 (0x00007f4a53096000)
        librte_ip_frag.so.1 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_ip_frag.so.1 (0x00007f4a52e8d000)
        librte_meter.so.1 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_meter.so.1 (0x00007f4a52c8a000)
        librte_sched.so.1 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_sched.so.1 (0x00007f4a52a82000)
        librte_lpm.so.2 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_lpm.so.2 (0x00007f4a5287b000)
        librte_acl.so.2 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_acl.so.2 (0x00007f4a52665000)
        librte_jobstats.so.1 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_jobstats.so.1 (0x00007f4a52462000)
        librte_power.so.1 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_power.so.1 (0x00007f4a52251000)
        librte_timer.so.1 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_timer.so.1 (0x00007f4a52046000)
        librte_hash.so.2 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_hash.so.2 (0x00007f4a51e36000)
        librte_vhost.so.3 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_vhost.so.3 (0x00007f4a51c19000)
        librte_kvargs.so.1 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_kvargs.so.1 (0x00007f4a51a15000)
        librte_mbuf.so.2 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_mbuf.so.2 (0x00007f4a51812000)
        libethdev.so.4 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/libethdev.so.4 (0x00007f4a51585000)
        librte_cryptodev.so.1 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_cryptodev.so.1 (0x00007f4a5137b000)
        librte_mempool.so.2 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_mempool.so.2 (0x00007f4a51174000)
        librte_ring.so.1 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_ring.so.1 (0x00007f4a50f71000)
        librte_eal.so.2 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_eal.so.2 (0x00007f4a50d12000)
        librte_cmdline.so.2 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_cmdline.so.2 (0x00007f4a50b08000)
        librte_cfgfile.so.2 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_cfgfile.so.2 (0x00007f4a50904000)
        librte_pmd_bond.so.1 => /root/Src/copyright/j/dpdk//x86_64-native-linuxapp-gcc/lib/librte_pmd_bond.so.1 (0x00007f4a506ef000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f4a504d9000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f4a502d5000)
        libprotobuf-c.so.1 => /lib64/libprotobuf-c.so.1 (0x00007f4a500cb000)
        libjansson.so.4 => /lib64/libjansson.so.4 (0x00007f4a4febe000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f4a4fca2000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f4a4f8de000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f4a55078000)
        libssl.so.10 => /lib64/libssl.so.10 (0x00007f4a4f671000)
        libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f4a4f28a000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f4a4f081000)
        libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f4a4ee33000)
        libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f4a4eb4b000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f4a4e947000)
        libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f4a4e713000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f4a4e4fd000)
        libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f4a4e2ef000)
        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f4a4e0ea000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f4a4ded0000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f4a4dca8000)
        libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f4a4da46000)