Redis缩容卡槽Slots迁移算法实现

背景

当一个Redis不再需要过多节点时,要进行节点缩容。此时要删除节点,所以要迁移卡槽,迁移卡槽后还想让卡槽落在某些特定机器,且比较均匀。下面用 n主->3主 进行举例。

迁移卡槽命令

```python
redis-cli --cluster reshard <host>:<port> --cluster-from <node-id> --cluster-to <node-id> --cluster-slots <number of slots> --cluster-yes
```

参数解释:

  • <host>:<port>: redis任意一个可以访问的节点
  • <node-id>: 节点的节点Id(RunId)
  • <number of slots>:迁移的卡槽数量。迁移卡槽时,默认先迁移低地址卡槽

算法思想

(1). 假设要迁移的最终结果为:

```
node0: 0-5460
node1: 5461-10922
node2: 10923-16383
```

(2). 假设当前slot的分配情况如下所示

节点 slot
N0 0 - X1
N1 X1 - X2
N2 X2 - X3
... ...
Nn Xn - 16383

(3). 假设 X1 <= 5460,则命令如下

```python
--cluster-from N0 --cluster-to node0 --cluster-slots X1 --cluster-yes
```

之后待分配卡槽为:

节点 slot
N1 X1 - X2
N2 X2 - X3
... ...
Nn Xn - 16383

(4). 若 X1 > 5460,则命令如下

```python
--cluster-from N0 --cluster-to node0 --cluster-slots 5461 --cluster-yes
```

分配卡槽后,待分配卡槽为

节点 slot
N0 5461 - X1
N1 X1 - X2
N2 X2 - X3
... ...
Nn Xn - 16383

(5). 按照此方法以此类推,直到所有节点都分配完毕

Java代码实现

```java
public static void computeSlotsPlan(JedisCluster jedisCluster) {
     List clusterSlots = null;
     for (JedisPool pool: jedisCluster.getClusterNodes().values()) {
         try(Jedis jedis = pool.getResource()) {
             clusterSlots = jedis.clusterSlots();
             if (clusterSlots != null) break;
         }
     } // 获取各个节点的Slots信息

     if (clusterSlots == null) {
         throw new RuntimeException("获取集群Slot信息失败");
     }

     /**
      * 将Slots信息组成以下数据结构,方便操作:
      * [
      *   [startSlot, endSlots, nodeId] // 表示该 startSlot - endSlots 这个段卡槽在节点nodeId上
      *   ...
      * ]
      * [
      *  [0, 3277, "e0637fafd06b2c8a0156b332e804b37842601da0"],
      *  [3278, 6553, "af7c59487546f1d12fe62ca03e8854f265dc563e"],
      *  ...
      *  [13107, 16383, "3f3c4f5c9a18088e6a65d6e8f9883988df26734b"]
      * ]
      */
     List<List> slotsInfo = new ArrayList<>();
     for (Object clusterSlot : clusterSlots) {
         List slotInfo = ((List)clusterSlot);

         List singleSlotInfo = new ArrayList(3);
         singleSlotInfo.add(slotInfo.get(0));
         singleSlotInfo.add(slotInfo.get(1));
         singleSlotInfo.add(new String((byte[]) ((List)slotInfo.get(2)).get(2)));
         slotsInfo.add(singleSlotInfo);
     }

     slotsInfo.sort((o1, o2) -> (Long) o1.get(0) > (Long)o2.get(0) ? 1 : -1); // 将slots信息从小到大排序

     // 认定最低地址的三个节点作为最终节点,实际做时,可以修改该处todo
     String[] nodeIds = new String[3];
     for (int i = 0; i < nodeIds.length; i++) {
         nodeIds[i] = (String) slotsInfo.get(i).get(2);
     }

     Long currentSlot = 0L;
     int currentNodeIndex = 0; // 当前计算到第几个nodeIds了
     String temp = "redis-cli --cluster reshard 127.0.0.1:7001 ";

     while (!slotsInfo.isEmpty()) {
         List item = slotsInfo.get(0);
         Long waitForCompareSlot;
         if (currentSlot <= 5460L) {
             currentNodeIndex = 0;
             waitForCompareSlot = 5460L;
         } else if (currentSlot <= 10922) {
             currentNodeIndex = 1;
             waitForCompareSlot = 10922L;
         } else {
             currentNodeIndex = 2;
             waitForCompareSlot = 16383L;
         }

         Long numOfSlots = 0L;  // 要迁移slot的数量
         if ((Long)item.get(1) > waitForCompareSlot) {

             numOfSlots = waitForCompareSlot - (Long)item.get(0) + 1;
             item.set(0, waitForCompareSlot + 1);
             currentSlot = waitForCompareSlot + 1;
         } else {
             numOfSlots = (Long)item.get(1) - (Long)item.get(0) + 1;
             slotsInfo.remove(0);
         }

         if (item.get(2).equals(nodeIds[currentNodeIndex]) || numOfSlots <= 0) continue;
         System.out.println(temp + "--cluster-from " + item.get(2) + " --cluster-to " + nodeIds[currentNodeIndex] + " --cluster-slots " + numOfSlots + " --cluster-yes");
     }
 }
```

最后输出为:

```
redis-cli --cluster reshard 127.0.0.1:7001 --cluster-from e0637fafd06b2c8a0156b332e804b37842601da0 --cluster-to c37737fca87286c7d158bea45285d9f137ebfa96 --cluster-slots 2184 --cluster-yes
redis-cli --cluster reshard 127.0.0.1:7001 --cluster-from af7c59487546f1d12fe62ca03e8854f265dc563e --cluster-to e0637fafd06b2c8a0156b332e804b37842601da0 --cluster-slots 3276 --cluster-yes
redis-cli --cluster reshard 127.0.0.1:7001 --cluster-from e22b661ea4f416c6eda146ade8170fbd5878a598 --cluster-to e0637fafd06b2c8a0156b332e804b37842601da0 --cluster-slots 1093 --cluster-yes
redis-cli --cluster reshard 127.0.0.1:7001 --cluster-from e22b661ea4f416c6eda146ade8170fbd5878a598 --cluster-to af7c59487546f1d12fe62ca03e8854f265dc563e --cluster-slots 2184 --cluster-yes
redis-cli --cluster reshard 127.0.0.1:7001 --cluster-from 4e5452745931505d326a7a251314951cc251b4bc --cluster-to af7c59487546f1d12fe62ca03e8854f265dc563e --cluster-slots 3277 --cluster-yes
```

按照这些命令一个一个执行即可。注意两次执行之间需要间隔一定时间。

Next Post Previous Post
No Comment
Add Comment
comment url