Redis缩容卡槽Slots迁移算法实现
背景
当一个Redis不再需要过多节点时,要进行节点缩容。此时要删除节点,所以要迁移卡槽,迁移卡槽后还想让卡槽落在某些特定机器,且比较均匀。下面用 n主->3主 进行举例。
迁移卡槽命令
```python redis-cli --cluster reshard <host>:<port> --cluster-from <node-id> --cluster-to <node-id> --cluster-slots <number of slots> --cluster-yes ```
参数解释:
<host>:<port>: redis任意一个可以访问的节点<node-id>: 节点的节点Id(RunId)<number of slots>:迁移的卡槽数量。迁移卡槽时,默认先迁移低地址卡槽
算法思想
(1). 假设要迁移的最终结果为:
``` node0: 0-5460 node1: 5461-10922 node2: 10923-16383 ```
(2). 假设当前slot的分配情况如下所示
| 节点 | slot |
|---|---|
| N0 | 0 - X1 |
| N1 | X1 - X2 |
| N2 | X2 - X3 |
| ... | ... |
| Nn | Xn - 16383 |
(3). 假设 X1 <= 5460,则命令如下
```python --cluster-from N0 --cluster-to node0 --cluster-slots X1 --cluster-yes ```
之后待分配卡槽为:
| 节点 | slot |
|---|---|
| N1 | X1 - X2 |
| N2 | X2 - X3 |
| ... | ... |
| Nn | Xn - 16383 |
(4). 若 X1 > 5460,则命令如下
```python --cluster-from N0 --cluster-to node0 --cluster-slots 5461 --cluster-yes ```
分配卡槽后,待分配卡槽为
| 节点 | slot |
|---|---|
| N0 | 5461 - X1 |
| N1 | X1 - X2 |
| N2 | X2 - X3 |
| ... | ... |
| Nn | Xn - 16383 |
(5). 按照此方法以此类推,直到所有节点都分配完毕
Java代码实现
```java
public static void computeSlotsPlan(JedisCluster jedisCluster) {
List clusterSlots = null;
for (JedisPool pool: jedisCluster.getClusterNodes().values()) {
try(Jedis jedis = pool.getResource()) {
clusterSlots = jedis.clusterSlots();
if (clusterSlots != null) break;
}
} // 获取各个节点的Slots信息
if (clusterSlots == null) {
throw new RuntimeException("获取集群Slot信息失败");
}
/**
* 将Slots信息组成以下数据结构,方便操作:
* [
* [startSlot, endSlots, nodeId] // 表示该 startSlot - endSlots 这个段卡槽在节点nodeId上
* ...
* ]
* [
* [0, 3277, "e0637fafd06b2c8a0156b332e804b37842601da0"],
* [3278, 6553, "af7c59487546f1d12fe62ca03e8854f265dc563e"],
* ...
* [13107, 16383, "3f3c4f5c9a18088e6a65d6e8f9883988df26734b"]
* ]
*/
List<List> slotsInfo = new ArrayList<>();
for (Object clusterSlot : clusterSlots) {
List slotInfo = ((List)clusterSlot);
List singleSlotInfo = new ArrayList(3);
singleSlotInfo.add(slotInfo.get(0));
singleSlotInfo.add(slotInfo.get(1));
singleSlotInfo.add(new String((byte[]) ((List)slotInfo.get(2)).get(2)));
slotsInfo.add(singleSlotInfo);
}
slotsInfo.sort((o1, o2) -> (Long) o1.get(0) > (Long)o2.get(0) ? 1 : -1); // 将slots信息从小到大排序
// 认定最低地址的三个节点作为最终节点,实际做时,可以修改该处todo
String[] nodeIds = new String[3];
for (int i = 0; i < nodeIds.length; i++) {
nodeIds[i] = (String) slotsInfo.get(i).get(2);
}
Long currentSlot = 0L;
int currentNodeIndex = 0; // 当前计算到第几个nodeIds了
String temp = "redis-cli --cluster reshard 127.0.0.1:7001 ";
while (!slotsInfo.isEmpty()) {
List item = slotsInfo.get(0);
Long waitForCompareSlot;
if (currentSlot <= 5460L) {
currentNodeIndex = 0;
waitForCompareSlot = 5460L;
} else if (currentSlot <= 10922) {
currentNodeIndex = 1;
waitForCompareSlot = 10922L;
} else {
currentNodeIndex = 2;
waitForCompareSlot = 16383L;
}
Long numOfSlots = 0L; // 要迁移slot的数量
if ((Long)item.get(1) > waitForCompareSlot) {
numOfSlots = waitForCompareSlot - (Long)item.get(0) + 1;
item.set(0, waitForCompareSlot + 1);
currentSlot = waitForCompareSlot + 1;
} else {
numOfSlots = (Long)item.get(1) - (Long)item.get(0) + 1;
slotsInfo.remove(0);
}
if (item.get(2).equals(nodeIds[currentNodeIndex]) || numOfSlots <= 0) continue;
System.out.println(temp + "--cluster-from " + item.get(2) + " --cluster-to " + nodeIds[currentNodeIndex] + " --cluster-slots " + numOfSlots + " --cluster-yes");
}
}
```
最后输出为:
``` redis-cli --cluster reshard 127.0.0.1:7001 --cluster-from e0637fafd06b2c8a0156b332e804b37842601da0 --cluster-to c37737fca87286c7d158bea45285d9f137ebfa96 --cluster-slots 2184 --cluster-yes redis-cli --cluster reshard 127.0.0.1:7001 --cluster-from af7c59487546f1d12fe62ca03e8854f265dc563e --cluster-to e0637fafd06b2c8a0156b332e804b37842601da0 --cluster-slots 3276 --cluster-yes redis-cli --cluster reshard 127.0.0.1:7001 --cluster-from e22b661ea4f416c6eda146ade8170fbd5878a598 --cluster-to e0637fafd06b2c8a0156b332e804b37842601da0 --cluster-slots 1093 --cluster-yes redis-cli --cluster reshard 127.0.0.1:7001 --cluster-from e22b661ea4f416c6eda146ade8170fbd5878a598 --cluster-to af7c59487546f1d12fe62ca03e8854f265dc563e --cluster-slots 2184 --cluster-yes redis-cli --cluster reshard 127.0.0.1:7001 --cluster-from 4e5452745931505d326a7a251314951cc251b4bc --cluster-to af7c59487546f1d12fe62ca03e8854f265dc563e --cluster-slots 3277 --cluster-yes ```
按照这些命令一个一个执行即可。注意两次执行之间需要间隔一定时间。