Ceph Luminous手动解决pg分布不均衡问题

原文链接：

https://www.jianshu.com/p/afb6277dbfd6

1.设置集群仅支持 Luminous（或者L之后的）客户端

具体命令：
ceph osd set-require-min-compat-client luminous
或
ceph osd set-require-min-compat-client luminous --yes-i-really-mean-it

[root@controller ~]# ceph features
{
    "mon": [
        {
            "features": "0x3ffddff8ffacffff",
            "release": "luminous",
            "num": 1
        }
    ],
    "osd": [
        {
            "features": "0x3ffddff8ffacffff",
            "release": "luminous",
            "num": 14
        }
    ],
    "client": [
        {
            "features": "0x3ffddff8ffacffff",
            "release": "luminous",
            "num": 20
        }
    ],
    "mgr": [
        {
            "features": "0x3ffddff8ffacffff",
            "release": "luminous",
            "num": 1
        }
    ]
}

2.获取集群当前的osdmap信息

命令
ceph osd getmap {osdmap_filename}

[root@controller ~]# ceph osd getmap -o osd.map
got osdmap epoch 52619

3.获取当前集群数据均衡后的优化信息

命令
osdmaptool {osdmap_filename} --upmap out.txt [--upmap-pool <pool>] [--upmap-max <max-count>] [--upmap-deviation <max-deviation>]
其中

• upmap-pool ：指定需要优化均衡的存储池名

• upmap-max： 指定一次优化的数据条目，默认100，可根据环境业务情况调整该值，一次调整的条目越多，数据迁移会越多，可能对环境业务造成影响。

• max-deviation：最大偏差值，默认为0.01（即1％）。如果OSD利用率与平均值之间的差异小于此值，则将被视为完美。

假设只优化2条，即：

[root@controller ~]# osdmaptool osd.map --upmap out.txt --upmap-pool volumes --upmap-max=2 
osdmaptool: osdmap file 'osd.map'
writing upmap command output to: out.txt
checking for upmap cleanups
upmap, max-count 2, max deviation 0.01
 limiting to pools volumes (2)

[root@controller ~]# cat out.txt 
ceph osd pg-upmap-items 2.2de 10 5
ceph osd pg-upmap-items 2.2e1 8 5

从上述数据结果中
ceph osd pg-upmap-items 2.2de 10 5
表明pg 12.2de需从osd.10重新映射到osd.5

4.均衡

[root@controller ~]# set 2.2de  pg_upmap_items mapping to [10->5]   
[root@controller ~]# set 2.2d1  pg_upmap_items mapping to [8->5]