Redis缩容卡槽Slots迁移算法实现
背景
当一个Redis不再需要过多节点时,要进行节点缩容。此时要删除节点,所以要迁移卡槽,迁移卡槽后还想让卡槽落在某些特定机器,且比较均匀。下面用 n主->3主 进行举例。
迁移卡槽命令
```python redis-cli --cluster reshard <host>:<port> --cluster-from <node-id> --cluster-to <node-id> --cluster-slots <number of slots> --cluster-yes ```
参数解释:
<host>:<port>
: redis任意一个可以访问的节点<node-id>
: 节点的节点Id(RunId)<number of slots>
:迁移的卡槽数量。迁移卡槽时,默认先迁移低地址卡槽
算法思想
(1). 假设要迁移的最终结果为:
``` node0: 0-5460 node1: 5461-10922 node2: 10923-16383 ```
(2). 假设当前slot的分配情况如下所示
节点 | slot |
---|---|
N0 | 0 - X1 |
N1 | X1 - X2 |
N2 | X2 - X3 |
... | ... |
Nn | Xn - 16383 |
(3). 假设 X1 <= 5460,则命令如下
```python --cluster-from N0 --cluster-to node0 --cluster-slots X1 --cluster-yes ```
之后待分配卡槽为:
节点 | slot |
---|---|
N1 | X1 - X2 |
N2 | X2 - X3 |
... | ... |
Nn | Xn - 16383 |
(4). 若 X1 > 5460,则命令如下
```python --cluster-from N0 --cluster-to node0 --cluster-slots 5461 --cluster-yes ```
分配卡槽后,待分配卡槽为
节点 | slot |
---|---|
N0 | 5461 - X1 |
N1 | X1 - X2 |
N2 | X2 - X3 |
... | ... |
Nn | Xn - 16383 |
(5). 按照此方法以此类推,直到所有节点都分配完毕
Java代码实现
```java public static void computeSlotsPlan(JedisCluster jedisCluster) { List clusterSlots = null; for (JedisPool pool: jedisCluster.getClusterNodes().values()) { try(Jedis jedis = pool.getResource()) { clusterSlots = jedis.clusterSlots(); if (clusterSlots != null) break; } } // 获取各个节点的Slots信息 if (clusterSlots == null) { throw new RuntimeException("获取集群Slot信息失败"); } /** * 将Slots信息组成以下数据结构,方便操作: * [ * [startSlot, endSlots, nodeId] // 表示该 startSlot - endSlots 这个段卡槽在节点nodeId上 * ... * ] * [ * [0, 3277, "e0637fafd06b2c8a0156b332e804b37842601da0"], * [3278, 6553, "af7c59487546f1d12fe62ca03e8854f265dc563e"], * ... * [13107, 16383, "3f3c4f5c9a18088e6a65d6e8f9883988df26734b"] * ] */ List<List> slotsInfo = new ArrayList<>(); for (Object clusterSlot : clusterSlots) { List slotInfo = ((List)clusterSlot); List singleSlotInfo = new ArrayList(3); singleSlotInfo.add(slotInfo.get(0)); singleSlotInfo.add(slotInfo.get(1)); singleSlotInfo.add(new String((byte[]) ((List)slotInfo.get(2)).get(2))); slotsInfo.add(singleSlotInfo); } slotsInfo.sort((o1, o2) -> (Long) o1.get(0) > (Long)o2.get(0) ? 1 : -1); // 将slots信息从小到大排序 // 认定最低地址的三个节点作为最终节点,实际做时,可以修改该处todo String[] nodeIds = new String[3]; for (int i = 0; i < nodeIds.length; i++) { nodeIds[i] = (String) slotsInfo.get(i).get(2); } Long currentSlot = 0L; int currentNodeIndex = 0; // 当前计算到第几个nodeIds了 String temp = "redis-cli --cluster reshard 127.0.0.1:7001 "; while (!slotsInfo.isEmpty()) { List item = slotsInfo.get(0); Long waitForCompareSlot; if (currentSlot <= 5460L) { currentNodeIndex = 0; waitForCompareSlot = 5460L; } else if (currentSlot <= 10922) { currentNodeIndex = 1; waitForCompareSlot = 10922L; } else { currentNodeIndex = 2; waitForCompareSlot = 16383L; } Long numOfSlots = 0L; // 要迁移slot的数量 if ((Long)item.get(1) > waitForCompareSlot) { numOfSlots = waitForCompareSlot - (Long)item.get(0) + 1; item.set(0, waitForCompareSlot + 1); currentSlot = waitForCompareSlot + 1; } else { numOfSlots = (Long)item.get(1) - (Long)item.get(0) + 1; slotsInfo.remove(0); } if (item.get(2).equals(nodeIds[currentNodeIndex]) || numOfSlots <= 0) continue; System.out.println(temp + "--cluster-from " + item.get(2) + " --cluster-to " + nodeIds[currentNodeIndex] + " --cluster-slots " + numOfSlots + " --cluster-yes"); } } ```
最后输出为:
``` redis-cli --cluster reshard 127.0.0.1:7001 --cluster-from e0637fafd06b2c8a0156b332e804b37842601da0 --cluster-to c37737fca87286c7d158bea45285d9f137ebfa96 --cluster-slots 2184 --cluster-yes redis-cli --cluster reshard 127.0.0.1:7001 --cluster-from af7c59487546f1d12fe62ca03e8854f265dc563e --cluster-to e0637fafd06b2c8a0156b332e804b37842601da0 --cluster-slots 3276 --cluster-yes redis-cli --cluster reshard 127.0.0.1:7001 --cluster-from e22b661ea4f416c6eda146ade8170fbd5878a598 --cluster-to e0637fafd06b2c8a0156b332e804b37842601da0 --cluster-slots 1093 --cluster-yes redis-cli --cluster reshard 127.0.0.1:7001 --cluster-from e22b661ea4f416c6eda146ade8170fbd5878a598 --cluster-to af7c59487546f1d12fe62ca03e8854f265dc563e --cluster-slots 2184 --cluster-yes redis-cli --cluster reshard 127.0.0.1:7001 --cluster-from 4e5452745931505d326a7a251314951cc251b4bc --cluster-to af7c59487546f1d12fe62ca03e8854f265dc563e --cluster-slots 3277 --cluster-yes ```
按照这些命令一个一个执行即可。注意两次执行之间需要间隔一定时间。