可实现的全局唯一有序ID生成策略

Posted slowcity

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了可实现的全局唯一有序ID生成策略相关的知识，希望对你有一定的参考价值。

在博客园搜素全局唯一有序ID，罗列出来的文章大致讲述了以下几个问题，常见的生成全局唯一id的常见方法 ：使用数据库自动增长序列实现 ； 使用UUID实现；  使用redis实现； 使用Twitter的snowflake算法实现；使用数据库+本地缓存实现。作为一个记录性质的博客，简单总结一下。

在实际的生产场景中，经常会出现如下的情况比方说订单号：D channelNo 流水号 样例PSDK1600000001, PSDK1600000002, PSDK1600000003... 这种具有业务意义的全局唯一id且有序自增。先来看一下使用比较多的Twitter的snowflake算法，snowflake生成的ID整体上按照时间自增排序，并且整个分布式系统内不会产生ID碰撞（由datacenter和workerId作区分），并且效率较高。经测试snowflake每秒能够产生26万个ID。一个简单的实现如下：

/**
 * Twitter的分布式自增ID雪花算法snowflake
 **/
public class SnowFlake {
    /**
     * 起始的时间戳
     */
    private final static long START_STMP = 1480166465631L;
    /**
     * 每一部分占用的位数
     */
    private final static long SEQUENCE_BIT = 12; //序列号占用的位数
    private final static long MACHINE_BIT = 5;   //机器标识占用的位数
    private final static long DATACENTER_BIT = 5;//数据中心占用的位数
    /**
     * 每一部分的最大值
     */
    private final static long MAX_DATACENTER_NUM = -1L ^ (-1L << DATACENTER_BIT);
    private final static long MAX_MACHINE_NUM = -1L ^ (-1L << MACHINE_BIT);
    private final static long MAX_SEQUENCE = -1L ^ (-1L << SEQUENCE_BIT);
    /**
     * 每一部分向左的位移
     */
    private final static long MACHINE_LEFT = SEQUENCE_BIT;
    private final static long DATACENTER_LEFT = SEQUENCE_BIT + MACHINE_BIT;
    private final static long TIMESTMP_LEFT = DATACENTER_LEFT + DATACENTER_BIT;

    private long datacenterId;  //数据中心
    private long machineId;     //机器标识
    private long sequence = 0L; //序列号
    private long lastStmp = -1L;//上一次时间戳

    public SnowFlake(long datacenterId, long machineId) {
        if (datacenterId > MAX_DATACENTER_NUM || datacenterId < 0) {
            throw new IllegalArgumentException("datacenterId can‘t be greater than MAX_DATACENTER_NUM or less than 0");
        }
        if (machineId > MAX_MACHINE_NUM || machineId < 0) {
            throw new IllegalArgumentException("machineId can‘t be greater than MAX_MACHINE_NUM or less than 0");
        }
        this.datacenterId = datacenterId;
        this.machineId = machineId;
    }

    /**
     * 产生下一个ID
     *
     * @return
     */
    public synchronized long nextId() {
        long currStmp = getNewstmp();
        if (currStmp < lastStmp) {
            throw new RuntimeException("Clock moved backwards.  Refusing to generate id");
        }

        if (currStmp == lastStmp) {
            //相同毫秒内，序列号自增
            sequence = (sequence + 1) & MAX_SEQUENCE;
            //同一毫秒的序列数已经达到最大
            if (sequence == 0L) {
                currStmp = getNextMill();
            }
        } else {
            //不同毫秒内，序列号置为0
            sequence = 0L;
        }

        lastStmp = currStmp;

        return (currStmp - START_STMP) << TIMESTMP_LEFT //时间戳部分
                | datacenterId << DATACENTER_LEFT       //数据中心部分
                | machineId << MACHINE_LEFT             //机器标识部分
                | sequence;                             //序列号部分
    }

    private long getNextMill() {
        long mill = getNewstmp();
        while (mill <= lastStmp) {
            mill = getNewstmp();
        }
        return mill;
    }

    private long getNewstmp() {
        return System.currentTimeMillis();
    }

    public static void main(String[] args) {
        SnowFlake snowFlake = new SnowFlake(1, 1);
        long start = System.currentTimeMillis();
        for (int i = 0; i < 10; i++) {
            System.out.println(snowFlake.nextId());
        }
        System.out.println(System.currentTimeMillis() - start);
    }
}

算法中引入了时间因子，所以可以保证生成的id唯一且有序，但是满足不了业务字段+流水号有序自增的要求。如果在此基础上再配合使用数据库本地缓存自然也是可以实现的，不过复杂化了。上述代码执行两次结果如下： 

385063405393940480
385063405393940481
385063405393940482
385063405393940483
385063405393940484
385063405393940485
385063405393940486
385063405393940487
385063405398134784
385063405398134785 

385064572152844288
385064572152844289
385064572152844290
385064572152844291
385064572152844292
385064572152844293
385064572152844294
385064572152844295
385064572152844296
385064572152844297

 

简单的方法就是我们放弃自己造轮子的思想。mongodb中数据的基本单元称为document，在一个特定集合内部需要唯一的标识文档，因此mongdb中存储的文档都由一个‘_id’键，这个键的值可以是任意类型的ObjectId，要求不同的机器都能用全局唯一的同种方法方便的生成它。因此不能使用自增主键，ObjectId 底层也是借鉴了雪花算法，使用12字节的存储空间  |0|1|2|3|4|5|6 |7|8|9|10|11|  |时间戳  |机器ID|PID|计数器|  前四个字节时间戳是从标准纪元开始的时间戳，单位为秒 。时间戳保证秒级唯一，机器ID保证设计时考虑分布式，避免时钟同步，PID保证同一台服务器运行多个mongod实例时的唯一性，最后的计数器保证同一秒内的唯一性。

 

 

 

mongo在spring boot中的引入和配置，此处不再介绍。

创建model类

package com.slowcity.admin.generate.dbmodel;

import java.io.Serializable;

public class BaseSequence implements Serializable{

    private static final long serialVersionUID = 475722757687764546L;
    private String id;
    private String name;
    private Long sequence;

    public String getId() {
        return id;
    }
    public void setId(String id) {
        this.id = id;
    }
    public String getName() {
        return name;
    }
    public void setName(String name) {
        this.name = name;
    }
    public Long getSequence() {
        return sequence;
    }
    public void setSequence(Long sequence) {
        this.sequence = sequence;
    }
    @Override
    public int hashCode() {
        final int prime = 31;
        int result = 1;
        result = prime * result + ((id == null) ? 0 : id.hashCode());
        result = prime * result + ((name == null) ? 0 : name.hashCode());
        result = prime * result + ((sequence == null) ? 0 : sequence.hashCode());
        return result;
    }
    @Override
    public boolean equals(Object obj) {
        if (this == obj)
            return true;
        if (obj == null)
            return false;
        if (getClass() != obj.getClass())
            return false;
        BaseSequence other = (BaseSequence) obj;
        if (id == null) {
            if (other.id != null)
                return false;
        } else if (!id.equals(other.id))
            return false;
        if (name == null) {
            if (other.name != null)
                return false;
        } else if (!name.equals(other.name))
            return false;
        if (sequence == null) {
            if (other.sequence != null)
                return false;
        } else if (!sequence.equals(other.sequence))
            return false;
        return true;
    }
    @Override
    public String toString() {
        return "BaseSequence [id=" + id + ", name=" + name + ", sequence=" + sequence + "]";
    }

}
public class DigitalTaskSequence extends BaseSequence{
    private static final long serialVersionUID = -7287622688931253780L;
}
import org.springframework.data.annotation.Id;
import org.springframework.data.mongodb.core.mapping.Document;
import org.springframework.stereotype.Component;


@Component
@Document(collection = "dm_id_task")
public class DigitalTaskSequenceMG extends DigitalTaskSequence {
    private static final long serialVersionUID = -425011291271386371L;
    @Id
    @Override
    public String getId() {
        return super.getId();
    }
}

service

import java.util.List;

import com.slowcity.admin.generate.dbmodel.BaseSequence;

public interface SequenceGenericService {
    public String generateId(Class<? extends BaseSequence> clazz);
    List<BaseSequence> initAllId();
}
import java.util.ArrayList;
import java.util.List;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import com.slowcity.admin.generate.dbmodel.BaseSequence;
import com.slowcity.admin.generate.dbmodel.DigitalTaskSequenceMG;
import com.slowcity.admin.generate.repository.SequenceGenericRepository;
import com.slowcity.admin.generate.service.SequenceGenericService;


@Service
@Transactional
public class SequenceGenericServiceImpl implements SequenceGenericService {
    private static final Logger log = LoggerFactory.getLogger(SequenceGenericServiceImpl.class);
    private SequenceGenericRepository sequenceGenericRepository;

    public SequenceGenericServiceImpl(SequenceGenericRepository sequenceGenericRepository) {
        this.sequenceGenericRepository = sequenceGenericRepository;
    }

    @Override
    public String generateId(Class<? extends BaseSequence> clazz) {
        String id = sequenceGenericRepository.generateId(clazz);
        log.info("{} generate {}", clazz.getName(), id);
        return id;
    }

    @Override
    public List<BaseSequence> initAllId() {

        List<BaseSequence> baseSequenceList = new ArrayList<>(),
        baseSequenceResultList = new ArrayList<>();
        DigitalTaskSequenceMG digitalTaskSequenceMG = new DigitalTaskSequenceMG();
        digitalTaskSequenceMG.setName("sequence");
        digitalTaskSequenceMG.setSequence(1210000000000000000L); //1210可以代表业务号 000000000000000代表自增流水号

        baseSequenceList.add(digitalTaskSequenceMG);
        
        for (BaseSequence baseSequence:baseSequenceList) {
            BaseSequence resultSequence = sequenceGenericRepository.initAllId(baseSequence);
            if(resultSequence != null){
                baseSequenceResultList.add(resultSequence);
            }
        }
        return baseSequenceResultList;
    }

   
}

数据实现层

import com.slowcity.admin.generate.dbmodel.BaseSequence;

public interface SequenceGenericRepository {

    public String generateId(Class<? extends BaseSequence> clazz);
    BaseSequence initAllId(BaseSequence Sequence);
    
}
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;

import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.data.mongodb.core.query.Query;
import org.springframework.data.mongodb.core.query.Update;
import org.springframework.stereotype.Component;

import com.slowcity.admin.generate.dbmodel.BaseSequence;
import com.slowcity.admin.generate.repository.SequenceGenericRepository;

@Component
public class SequenceMongoGenericRepository implements SequenceGenericRepository {
    private Map<Class,Class<? extends BaseSequence>> baseSequenceMap;
    private MongoTemplate mongoTemplate;
    public SequenceMongoGenericRepository(List<BaseSequence> baseSequences, MongoTemplate mongoTemplate){
        baseSequenceMap = baseSequences.stream()
            .collect(Collectors.toMap(baseSequence -> baseSequence.getClass().getSuperclass(),
                BaseSequence::getClass));
        this.mongoTemplate = mongoTemplate;
    }


    @Override
    public String generateId(Class<? extends BaseSequence> clazz) {
        Class<? extends BaseSequence> childClazz = baseSequenceMap.get(clazz);
        if(childClazz != null) {
            Query query = new Query(Criteria.where("name").is("sequence"));
            Update update = new Update().inc("sequence", 1);
            Object dbm = mongoTemplate.findAndModify(query, update, childClazz);
            if(dbm != null) {
                BaseSequence bs = (BaseSequence)dbm;
                return String.valueOf(bs.getSequence());
            }
        }
        return null;
    }

    @Override
    public BaseSequence initAllId(BaseSequence Sequence) {

        Query query = new Query(Criteria.where("name").is("sequence"));
        Class clazz = Sequence.getClass();
        List<? extends BaseSequence> list = mongoTemplate.find(query,clazz);
        if(list.isEmpty()){
            mongoTemplate.save(Sequence);
            return Sequence;
        } 
            return null;
    }
}

controller

import java.util.List;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

import com.slowcity.admin.generate.dbmodel.BaseSequence;
import com.slowcity.admin.generate.dbmodel.DigitalTaskSequence;
import com.slowcity.admin.generate.service.SequenceGenericService;

/**
 * id生成器 
 * @author moona
 *
 */
@RestController
@RequestMapping("/generateId/task")
public class TaskGenerateIdController {

    @Autowired
    private SequenceGenericService sequenceGenericService;
    @RequestMapping(value = "/taskId", method = RequestMethod.GET)
    public String generateTaskId() {
        return sequenceGenericService.generateId(DigitalTaskSequence.class);
    }
    
    @RequestMapping(value = "/init", method = RequestMethod.GET)
    public  List<BaseSequence> generateTaskIdinit() {
        return sequenceGenericService.initAllId();
    }
    
}

 

执行初始化调用方法

技术图片

 

 对应数据库

技术图片

 

 

开始测试生成id

第一次调用：

技术图片

第2次调用

技术图片

 

 第10次调用

技术图片

 

 此时再查看数据库，序列已经到1210000000000000011 下次调用直接取值了。真正做到了了分布式满足业务的自增全局唯一索引。mongo底层是原子性的，所以也不会出现并发的问题。如果将id生成策略部署成单台机器服务，则可以满足不同服务不同业务的需求，真正做到可定制可扩展。尽可放心使用。

技术图片

 

 

【end】

 

以上是关于可实现的全局唯一有序ID生成策略的主要内容，如果未能解决你的问题，请参考以下文章

Redis场景拓展秒杀问题-全局唯一ID生成策略

唯一ID生成算法剖析

唯一ID生成算法剖析

常见分布式全局唯一ID生成策略

常见分布式全局唯一ID生成策略

全局唯一ID生成方案