寮€婧愬伐鍏?| hbase-sdk ORM妗嗘灦

Posted 2021-04-18 澶х尶灏忕尶鍚戝墠鍐?/a>

hbase-sdk

鍩轰簬hbase-client鐨勭浉鍏矨PI寮€鍙戣€屾潵鐨勪竴娆捐交閲忕骇鐨凥Base ORM妗嗘灦銆?馃構

hbase-sdk鍒嗕负spring-boot-starter-hbase鍜宧base-sdk-core涓ら儴鍒嗐€?/p>

SpringBoot椤圭洰涓紩鍏pring-boot-starter-hbase锛屽湪鏅€氱殑Java椤圭洰涓垯鍙互浣跨敤hbase-sdk-core銆?/p>

hbase-sdk

hbase-sdk 鏄竴娆捐交閲忕骇鐨?ORM 妗嗘灦锛屽皝瑁呬簡瀵?HBase 鐨勬暟鎹鍐欏拰瀵归泦缇ょ殑杩愮淮绠＄悊绛夋搷浣溿€?/p>

鍔熻兘鐗规€?/span>

褰撳墠鍔熻兘

• 鍦ㄤ繚鐣欏師鏈?hbase-client 鍔熻兘 API 鐨勫熀纭€涓婃彁渚涗簡 ORM 鐨勭壒鎬?
• 涓?SpringBoot 闆嗘垚锛屽緢濂藉湴鍒╃敤浜?SpringBoot 鐨勪紭鑹壒鎬?
• JDK8+

鏈潵鎯宠鏂板鐨勫姛鑳?span class="mq-32">

• 涓诲闆嗙兢 API 灞傞潰鐨勭啍鏂満鍒?
• 璇诲啓娴侀噺鎺у埗
• ... ...

蹇€熷紑濮?/span>

1. 鏅€氶」鐩?span class="mq-45">

Maven 閰嶇疆锛?/p>

鍒涘缓涓€涓熀纭€鐨?Maven 宸ョ▼锛孒Base SDK API 寮€鍙戞椂鍩轰簬鐨?HBase 鐗堟湰涓昏鏄?1.4.3 鍜?2.1.0銆?/p>

鎵€浠ワ紝濡傛灉浣犵殑 HBase 鐗堟湰鏄?1.x锛屽彲浠ヤ娇鐢ㄥ涓嬩緷璧栥€?/p>

<dependency>
    <groupId>com.github.CCweixiao</groupId>
    <artifactId>hbase-sdk-core_1.4</artifactId>
    <version>1.0.5</version>
</dependency>

濡傛灉浣犵殑 HBase 鐗堟湰鏄?2.x锛屽垯鍙互浣跨敤濡備笅渚濊禆銆?/p>

<dependency>
    <groupId>com.github.CCweixiao</groupId>
    <artifactId>hbase-sdk-core_2.1</artifactId>
    <version>1.0.5</version>
</dependency>

hbase-sdk鐩墠鏈€鏂扮殑鐗堟湰鏄?code class="mq-92">1.0.5銆備綘鍙互鍦?maven 浠撳簱涓悳绱?CCweixiao 鏉ヨ幏鍙?hbase-sdk 鐩稿叧 jar 鍖呯殑鏈€鏂扮増鏈€?/p>

https://mvnrepository.com/artifact/com.github.CCweixiao

褰撶劧锛屽鏋滀綘鎯抽噸鏂扮紪璇戯紝浠ラ€傞厤浣犺嚜宸辩殑 HBase 鐗堟湰锛屼篃鍙互閫夋嫨涓嬭浇婧愮爜锛屼慨鏀归」鐩?pom.xml 鏂囦欢涓殑 hbase.version 鏉ヨ繍琛屽涓嬬紪璇戝懡浠わ細

git clone https://github.com/CCweixiao/hbase-sdk.git
git clone https://gitee.com/weixiaotome/hbase-sdk.git
cd hbase-sdk
mvn clean install -Dmaven.test.skip=true

2. 椤圭洰缁撴瀯

project

3. 鍦?SpringBoot 椤圭洰涓娇鐢?span class="mq-107">

Maven 閰嶇疆锛?/p>

鍒涘缓涓€涓熀浜?code class="mq-111">Maven鐨?spring boot 宸ョ▼銆?/p>

<dependency>
    <groupId>com.github.CCweixiao</groupId>
    <artifactId>spring-boot-starter-hbase_1.4</artifactId>
    <version>1.0.5</version>
</dependency>

spring-boot-starter-hbase 杩欎釜妯″潡涓凡缁忓寘鍚簡 hbase-sdk-core銆?/p>

4. 寮曞叆 hbase-client 鐨勪緷璧?span class="mq-134">

闄や簡寮曞叆hbase-sdk鐨勭浉鍏充緷璧栦箣澶栵紝浣犺繕闇€瑕佸紩鍏?code class="mq-137">hbase-client鐨勪緷璧栵紝hbase-client鐨勭増鏈洰鍓嶅缓璁负1.2.x銆?code class="mq-140">1.4.x銆?code class="mq-141">2.1.x銆傚叾瀹?hbase-client 鏂版棫 API 鏈夋墍宸紓銆傛湭鏉ワ紝hbase-sdk鍦ㄥ hbase 鐨勭増鏈敮鎸佹柟闈細鏇村姞瀹屽杽銆?/p>

<dependency>
    <groupId>org.apache.hbase</groupId>
    <artifactId>hbase-client</artifactId>
    <version>1.4.3</version>
</dependency>

5. 閰嶇疆 HBase 鏁版嵁搴撹繛鎺?span class="mq-164">

鏅€氶」鐩?/strong>

// 鏁版嵁璇诲啓鎿嶄綔
HBaseTemplate hBaseTemplate = new HBaseTemplate("node1", "2181");
//闆嗙兢绠＄悊鎿嶄綔
HBaseAdminTemplate hBaseAdminTemplate = new HBaseAdminTemplate("node1", "2181");

spring boot 椤圭洰

application.yaml

spring:
  data:
    hbase:
      quorum: node1

@Service
public class UserService {
    @Autowired
    private HBaseTemplate hBaseTemplate;
    @Autowired
    private HBaseAdminTemplate hBaseAdminTemplate;
}

闆嗙兢绠＄悊

鐩墠锛孒BaseAdminTemplate 鍙彁渚涗簡 HBaseAdmin 鐨勫父鐢ㄦ搷浣滐紝姣斿 namespace 鐨勭鐞嗐€佽〃鐨勭鐞嗙瓑绛夛紝涓庡師鐢?HBaseAdmin 鐨?API 鐩告瘮锛屽畠鐨勫姛鑳藉彲鑳戒笉鏄緢鍏ㄩ潰锛屼絾浠ュ悗涓€瀹氫細鏇村姞瀹屽杽銆?/p>

鍒涘缓 namespace

HBase 绠＄悊鍛樻搷浣?API 涓庡師 API 鍏跺疄宸紓骞朵笉澶с€?/p>

    @Test
    public void testCreateNamespace() {
        String namespaceName = "LEO_NS2";
        Map<String, String> para = new HashMap<>();
        para.put("tag", "娴嬭瘯鍛藉悕绌洪棿");
        para.put("createBy", "leo");
        para.put("updateBy", "");
        NamespaceDescriptor namespaceDescriptor = NamespaceDescriptor.create(namespaceName)
                .addConfiguration(para)
                .build();
        hBaseTemplate.createNamespace(namespaceDescriptor);
    }

鍒涘缓琛?span class="mq-228">

    @Test
    public void testCreateTable() {
        String tableName = "LEO_NS:USER";

        HTableDescriptor tableDescriptor = new HTableDescriptor(TableName.valueOf(tableName));

        tableDescriptor.setValue("tag", "娴嬭瘯鐢ㄦ埛琛?);
        tableDescriptor.setValue("createUser", "leo");


        HColumnDescriptor columnDescriptor = new HColumnDescriptor("INFO");
        columnDescriptor.setScope(1);
        columnDescriptor.setCompressionType(Compression.Algorithm.SNAPPY);
        columnDescriptor.setTimeToLive(2147483647);
        columnDescriptor.setMaxVersions(3);

        HColumnDescriptor columnDescriptor2 = new HColumnDescriptor("INFO2");
        columnDescriptor2.setScope(0);
        columnDescriptor2.setTimeToLive(864000);
        columnDescriptor2.setMaxVersions(3);

        tableDescriptor.addFamily(columnDescriptor).addFamily(columnDescriptor2);

        hBaseTemplate.createTable(tableDescriptor, Bytes.toBytes(0), Bytes.toBytes(100), 10);
    }

鏇村鎿嶄綔

鍙互鍙傝€冩祴璇曠敤渚嬩腑鐨勭浉鍏矨PI銆?/p>

鏁版嵁璇诲啓

绫讳技浜?Hibernate锛屼綘涔熷彲浠ヤ娇鐢?hbase-sdk 妗嗘灦鎵€鎻愪緵鐨?ORM 鐗规€э紝鏉ュ疄鐜板 HBase 鐨勮鍐欐搷浣溿€?/p>

鍒涘缓鏁版嵁妯″瀷绫?span class="mq-267">

public class ModelEntity {
    private String createBy;
    private Long createTime;

    public String getCreateBy() {
        return createBy;
    }

    public void setCreateBy(String createBy) {
        this.createBy = createBy;
    }

    public Long getCreateTime() {
        return createTime;
    }

    public void setCreateTime(Long createTime) {
        this.createTime = createTime;
    }
}

@HBaseTable(schema = "TEST", name = "LEO_USER", uniqueFamily = "info1")
public class UserEntity extends ModelEntity{
    @HBaseRowKey
    private String userId;

    private String username;
    private int age;
    private List<String> addresses;
    private Map<String,Object> contactInfo;
    private Double pay;

    @HBaseColumn(name = "is_vip", family = "INFO2", toUpperCase = true)
    private boolean isVip;

    public String getUserId() {
        return userId;
    }

    public void setUserId(String userId) {
        this.userId = userId;
    }

    public String getUsername() {
        return username;
    }

    public void setUsername(String username) {
        this.username = username;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }

    public boolean isVip() {
        return isVip;
    }

    public void setVip(boolean vip) {
        isVip = vip;
    }

    public List<String> getAddresses() {
        return addresses;
    }

    public void setAddresses(List<String> addresses) {
        this.addresses = addresses;
    }

    public Map<String, Object> getContactInfo() {
        return contactInfo;
    }

    public void setContactInfo(Map<String, Object> contactInfo) {
        this.contactInfo = contactInfo;
    }

    public Double getPay() {
        return pay;
    }

    public void setPay(Double pay) {
        this.pay = pay;
    }

    @Override
    public String toString() {
        return "UserEntity{" +
                "userId='" + userId + '\'' +
                ", username='" + username + '\'' +
                ", age=" + age +
                ", addresses=" + addresses +
                ", contactInfo=" + contactInfo +
                ", pay=" + pay +
                ", isVip=" + isVip +
                '}';
    }
}

@HBaseTable(schema = "TEST", name = "LEO_USER", uniqueFamily = "info1")

HBaseTable 娉ㄨВ鐢ㄤ簬瀹氫箟 HBase 鐨勮〃淇℃伅锛宻chema 鐢ㄤ簬瀹氫箟璇ヨ〃鐨勫懡鍚嶇┖闂达紝濡傛灉涓嶆寚瀹氾紝榛樿鏄?default锛?name 鐢ㄤ簬瀹氫箟璇ヨ〃鐨勮〃鍚嶏紝濡傛灉涓嶆寚瀹氾紝琛ㄥ悕鍒欎负绫诲悕鐨勭粍鍚堝崟璇嶆媶鍒嗗姞'_'鎷兼帴锛屽锛歎serEntity 瀵瑰簲鐨勮〃鍚嶄负锛歶ser_entity銆?uniqueFamily 鐢ㄤ簬瀹氫箟濡傛灉鎵€鏈夌殑瀛楁涓嶇壒閰嶇疆鍒楃皣鍚嶏紝鍒欎娇鐢ㄦ澶勯厤缃殑鍒楃皣鍚嶃€?/p>

@HBaseRowKey
private String userId;

璇ユ敞瑙ｈ〃绀?userId 瀛楁涓?rowKey 瀛楁銆?/p>

@HBaseColumn(name = "is_vip", family = "INFO2", toUpperCase = true)
private boolean isVip;

璇ユ敞瑙ｇ敤浜庡畾涔変竴涓瓧娈典俊鎭紝name 鐢ㄤ簬瀹氫箟瀛楁鍚嶏紝濡傛灉涓嶆寚瀹氾紝鍒欓粯璁や娇鐢ㄥ瓧娈靛悕鐨勭粍鍚堝崟璇嶆媶鍒嗗姞'_'鎷兼帴锛屽锛歩sVip锛屽搴旂殑瀛楁鍚嶆槸锛歩s_vip. family 鐢ㄤ簬瀹氫箟璇ュ瓧娈靛睘浜?INFO2 鍒楃皣锛宼oUpperCase 琛ㄧず瀛楁鍚嶆槸鍚﹁浆澶у啓锛岄粯璁?false锛屼笉鍋氭搷浣溿€?/p>

淇濆瓨鏁版嵁

   @Test
    public void testSaveUser() {
        UserEntity userEntity = new UserEntity();
        userEntity.setUserId("10001");
        userEntity.setUsername("leo");
        userEntity.setAge(18);
        userEntity.setVip(true);
        userEntity.setAddresses(Arrays.asList("鍖椾含", "涓婃捣"));
        userEntity.setCreateBy("admin");
        userEntity.setCreateTime(System.currentTimeMillis());

        Map<String, Object> contactInfo = new HashMap<>(2);
        contactInfo.put("email", "[e-mail]");
        contactInfo.put("phone", "18739577988");
        contactInfo.put("address", "娴︿笢鏂板尯");

        userEntity.setContactInfo(contactInfo);
        userEntity.setPay(100000.0d);

        try {
            hBaseTemplate.save(userEntity);
            System.out.println("鐢ㄦ埛鏁版嵁淇濆瓨鎴愬姛锛?);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

闄ゆ涔嬪锛屼繚瀛樻暟鎹椂涔熷彲浠ヤ笉蹇呮瀯閫犳暟鎹ā鍨嬬被锛岃€岀洿鎺ユ瀯閫?map 鏁版嵁妯″瀷銆?/p>

    @Test
    public void testToSave() {
        Map<String, Object> data = new HashMap<>();
        data.put("info1:addresses", Arrays.asList("骞垮窞", "娣卞湷"));
        data.put("info1:username", "leo");
        data.put("info1:age", 18);
        data.put("INFO2:IS_VIP", true);
        data.put("info1:pay", 10000.1d);
        data.put("info1:create_by", "tom");
        data.put("info1:create_time", System.currentTimeMillis());
        Map<String, Object> contactInfo = new HashMap<>(2);
        contactInfo.put("email", "[e-mail]");
        contactInfo.put("phone", "18739577988");
        contactInfo.put("address", "娴︿笢鏂板尯");
        data.put("info1:contact_info", contactInfo);
        hBaseTemplate.save("TEST:LEO_USER", "10002", data);
        System.out.println("鐢ㄦ埛鏁版嵁淇濆瓨鎴愬姛锛?);
    }

鎵归噺淇濆瓨鏁版嵁

    @Test
    public void testToSaveBatch() {
        Map<String, Map<String, Object>> data = new HashMap<>();

        Map<String, Object> data1 = new HashMap<>();
        data1.put("info1:username", "kangkang");
        data1.put("info1:age", 18);
        data1.put("INFO2:IS_VIP", true);

        Map<String, Object> data2 = new HashMap<>();
        data2.put("info1:username", "jane");
        data2.put("info1:age", 18);
        data2.put("INFO2:IS_VIP", false);

        data.put("12003", data1);
        data.put("11004", data2);

        hBaseTemplate.saveBatch("TEST:LEO_USER", data);
        System.out.println("鐢ㄦ埛鏁版嵁鎵归噺淇濆瓨鎴愬姛锛?);
    }

鏌ヨ鏁版嵁

鏍规嵁 RowKey 鏌ヨ

    @Test
    public void testGet() {
        UserEntity userEntity = hBaseTemplate.getByRowKey("10001", UserEntity.class);
        final UserEntity userEntity1 = hBaseTemplate.getByRowKey("10002", UserEntity.class);
        System.out.println("鐢ㄦ埛鏁版嵁鑾峰彇鎴愬姛锛?);
        System.out.println(userEntity);
    }

    @Test
    public void testGetToMap() {
        Map<String, Object> userInfo = hBaseTemplate.getByRowKey("TEST:LEO_USER", "10001");
        System.out.println(Boolean.valueOf(userInfo.get("INFO2:IS_VIP").toString()));
        System.out.println(userInfo);
    }

scan 鏌ヨ

    @Test
    public void testFind() {
        final List<UserEntity> userEntities = hBaseTemplate.findAll(10, UserEntity.class);
        System.out.println(userEntities);
        System.out.println("鐢ㄦ埛鏁版嵁鎵归噺鏌ヨ");
    }

    @Test
    public void testFindByPrefix() {
        final List<UserEntity> userEntities = hBaseTemplate.findByPrefix("11", 10, UserEntity.class);
        System.out.println("鐢ㄦ埛鏁版嵁鎵归噺鏌ヨ");
    }

鍒犻櫎鏁版嵁

    @Test
    public void testDeleteData() {
        hBaseTemplate.delete("TEST:LEO_USER", "12003");
        hBaseTemplate.delete("TEST:LEO_USER", "11004", "INFO2");
        hBaseTemplate.delete("TEST:LEO_USER", "10001", "info1", "addresses");
        System.out.println("鏁版嵁鍒犻櫎瀹屾垚");
    }

    @Test
    public void testDeleteBatch() {
        hBaseTemplate.deleteBatch("TEST:LEO_USER", Arrays.asList("10001", "10002"));
        hBaseTemplate.deleteBatch("TEST:LEO_USER", Collections.singletonList("10003"), "INFO2");
        hBaseTemplate.deleteBatch("TEST:LEO_USER", Collections.singletonList("10004"),
                "info1", "age", "username");
    }

鎬荤粨

鍩轰簬 hbase-sdk 鐨?ORM 鐗规€э紝鍙互寰堟柟渚垮湴瀹炵幇 Java 瀵硅薄涓?HBase 鏁版嵁妯″瀷涔嬮棿鐨勭粦瀹氾紝鐩墠浠ｇ爜鏁翠綋鐨勫疄鐜拌繕鏈変簺绮楃硻锛屾垨璁歌繕瀛樺湪娌℃湁鑰冭檻鍒扮殑鎬ц兘鏂归潰鐨勯棶棰橈紝浣嗚繖浜涚摱棰堜竴瀹氫細鍦ㄥ悗缁殑鍔姏涓竴涓€鏀瑰杽銆?/p>

鎹锛屽叧娉ㄦ垜鐨勪汉閮芥壘鍒颁簡瀵硅薄馃憞