由于master宕机等导致resource调用出现异常，直接将该resource返回到pool以便其他代码使用会导致得到不可预期的结果，导致返回数据混乱。

Posted 2020-09-05

由于master宕机等导致resource调用出现异常，直接将该resource返回到pool以便其他代码使用会导致得到不可预期的结果，导致返回数据混乱。

实现一：
public String get(final String key) {         
     Jedis resource = null;         
     try {                   
        resource = pool.getResource();                   
         return resource.get(key);       
    } finally {                   
        pool.returnResourceObject(resource);        
    }
}

实现二：
public String get(final String key) {         
     Jedis resource = null;         
     try {                   
           resource = pool.getResource();                   
           return resource.get(key);         
     } finally {                   
           resource.close();         
     }
 }

哨兵的监控：
 手动关闭 master: [email protected]:6379
29350:X 06 Aug 09:31:36.184 # +sdown master mymaster 10.91.230.120 6379
29350:X 06 Aug 09:31:36.251 # +odown master mymaster 10.91.230.120 6379 #quorum 4/2
29350:X 06 Aug 09:31:36.251 # +new-epoch 71
哨兵发现master宕机，他们会选举一个slave成为master
29350:X 06 Aug 09:31:36.251 # +try-failover master mymaster 10.91.230.120 6379
29350:X 06 Aug 09:31:36.253 # +vote-for-leader 4619691f4385e4098ed8eba52dd254827d33e44b 71
29350:X 06 Aug 09:31:36.258 # 10.91.230.120:26380 voted for 4619691f4385e4098ed8eba52dd254827d33e44b 71
29350:X 06 Aug 09:31:36.259 # 10.91.230.119:26380 voted for 4619691f4385e4098ed8eba52dd254827d33e44b 71
29350:X 06 Aug 09:31:36.260 # 10.91.230.120:26379 voted for 4619691f4385e4098ed8eba52dd254827d33e44b 71
29350:X 06 Aug 09:31:36.354 # +elected-leader master mymaster 10.91.230.120 6379
29350:X 06 Aug 09:31:36.354 # +failover-state-select-slave master mymaster 10.91.230.120 6379
29350:X 06 Aug 09:31:36.444 # +selected-slave slave 10.91.230.119:6379 10.91.230.119 6379 @ mymaster 10.91.230.120 6379
29350:X 06 Aug 09:31:36.444 * +failover-state-send-slaveof-noone slave 10.91.230.119:6379 10.91.230.119 6379 @ mymaster 10.91.230.120 6379
29350:X 06 Aug 09:31:36.500 * +failover-state-wait-promotion slave 10.91.230.119:6379 10.91.230.119 6379 @ mymaster 10.91.230.120 6379
29350:X 06 Aug 09:31:36.979 # +promoted-slave slave 10.91.230.119:6379 10.91.230.119 6379 @ mymaster 10.91.230.120 6379
29350:X 06 Aug 09:31:36.980 # +failover-state-reconf-slaves master mymaster 10.91.230.120 6379
29350:X 06 Aug 09:31:37.031 # +failover-end master mymaster 10.91.230.120 6379
// 哨兵切换被选中的slave成为master
29350:X 06 Aug 09:31:37.031 # +switch-master mymaster 10.91.230.120 6379 10.91.230.119 6379
// 哨兵切换旧master为slave
29350:X 06 Aug 09:31:37.031 * +slave slave 10.91.230.120:6379 10.91.230.120 6379 @ mymaster 10.91.230.119 6379
// 手动开启旧的master，哨兵切换旧master为slave
29350:X 06 Aug 09:31:47.975 * +convert-to-slave slave 10.91.230.120:6379 10.91.230.120 6379 @ mymaster 10.91.230.119 6379

症状：
见实现一，master宕机，哨兵选举一个slave成为master，则调用某些resource的代码会报TIMEOUT的异常（此类resource已经由于master宕机等异常而broken，由于resource对网络socket进行了封装和buffer，异常的发生毁导致下一次返回出来的数据不可预期，需要主动关闭它，而不是官方示例代码那样直接在final块里面将它返回到pool中），在哨兵切换选中的slave成为新的master之后，调用此resource的代码就会获得不可预期的返回。

解决方案：
见实现二，在异常发生的时候，主动关闭该resource，则返回数据混乱的问题不会发生。