HBase ——Shell操作

Posted 2022-09-05 whoyoung

HBase ——Shell操作

Q：你觉得HBase是什么？

A：一种结构化的分布式数据存储系统，它基于列来存储数据。

基于HBase，可以实现以廉价PC机器集群存储海量数据的分布式数据库的解决方案。

一般用于数据量巨大但查询简单的场景，典型场景包括：账单记录，订单流水，交易记录，数据库的历史记录

 

• 建立表

  HBase中没有数据库的概念，但是存在命名空间的概念，在使用上类似库名，但实际上只是一个目录名。

  另外要记住，HBase中大部分数据都是以HashMap的形式组织的，因此很多时候都能看到类似 key : value的格式。命名空间表的定义也是如此。

  任何表的定义必须有表名和列族，可以没有命名空间

  create ‘ns1:tb1‘,‘cf1‘ #有命名空间的表，ns1是命名空间，cf1是列族
  

  create ‘tb1‘,‘cf1‘ 
  

  list #查看已有表
  

• 删除表

  删除表之前必须了解，HBase作为分布式数据库是没有锁的，更不支持事务。因此，HBase的删除表之前必须先禁用表。

  disable ‘tb1‘
  drop ‘tb1‘
  

• 插入数据

  在定义表的时候我们已经指定了列族，但是光有列族是不足以确定唯一数据的。插入数据时还必须指定行键和列键。

  插入数据，包括创建表都有多种属性可选，在此只介绍最基本的。hbase shell输入命令不加参数即可获得帮助指南。

  put ‘tb1‘,‘rk1‘,‘cf1:c1‘,‘value‘ #rk1是行键，c1是属于cf1族的列健
  

• 查找数据

  在上面已经提到了HBase适用于简单的查询业务场景，但是这种简单是相对于SQL的关联查找而言的。HBase在单表查询是足够强大的。

  HBase中记录分为三部分，ROW、COLUMN和CELL，COLUMN分为family列族和qualifier列键，CELL分为TimeStamp和Value。可以看出，一个唯一的value是由rowkey、family、qualifier、timestamp共同决定的。查找数据也是遵循的也是这个逻辑。

  get ‘tb1‘,‘rk1‘[,‘cf1[:c1]‘] #简单查找
  get ‘tb1‘,‘rk1‘,COLUMN=>[‘C1‘,‘C2‘,‘C3‘],TIMESTAMP=>ts1 #多条件
  

• 扫描数据

  get每次只能查找一个行键，而scan扫描是条件查找整张表

  scan ‘tb1‘,condition #condition可以是列族/键、时间、版本、范围，具体见帮助
  

• 删除数据

  hbase是列模式存储的，因此每次删除只能删除指定列键的row的value

  

  delete ‘tb1‘,‘rk1‘,‘cf1[:c1]‘ #删除的是一条记录
  

   

• 表操作

  Here is some help for this command:
  Alter a table. If the "hbase.online.schema.update.enable" property is set to
  false, then the table must be disabled (see help ‘disable‘). If the 
  "hbase.online.schema.update.enable" property is set to true, tables can be 
  altered without disabling them first. Altering enabled tables has caused problems 
  in the past, so use caution and test it before using in production. 
  
  You can use the alter command to add, 
  modify or delete column families or change table configuration options.
  Column families work in a similar way as the ‘create‘ command. The column family
  specification can either be a name string, or a dictionary with the NAME attribute.
  Dictionaries are described in the output of the ‘help‘ command, with no arguments.
  
  For example, to change or add the ‘f1‘ column family in table ‘t1‘ from 
  current value to keep a maximum of 5 cell VERSIONS, do:
  
    hbase> alter ‘t1‘, NAME => ‘f1‘, VERSIONS => 5
  
  You can operate on several column families:
  
    hbase> alter ‘t1‘, ‘f1‘, NAME => ‘f2‘, IN_MEMORY => true, NAME => ‘f3‘, VERSIONS => 5
  
  To delete the ‘f1‘ column family in table ‘ns1:t1‘, use one of:
  
    hbase> alter ‘ns1:t1‘, NAME => ‘f1‘, METHOD => ‘delete‘
    hbase> alter ‘ns1:t1‘, ‘delete‘ => ‘f1‘
  
  You can also change table-scope attributes like MAX_FILESIZE, READONLY, 
  MEMSTORE_FLUSHSIZE, DURABILITY, etc. These can be put at the end;
  for example, to change the max size of a region to 128MB, do:
  
    hbase> alter ‘t1‘, MAX_FILESIZE => ‘134217728‘
  
  You can add a table coprocessor by setting a table coprocessor attribute:
  
    hbase> alter ‘t1‘,
      ‘coprocessor‘=>‘hdfs:///foo.jar|com.foo.FooRegionObserver|1001|arg1=1,arg2=2‘
  
  Since you can have multiple coprocessors configured for a table, a
  sequence number will be automatically appended to the attribute name
  to uniquely identify it.
  
  The coprocessor attribute must match the pattern below in order for
  the framework to understand how to load the coprocessor classes:
  
    [coprocessor jar file location] | class name | [priority] | [arguments]
  
  You can also set configuration settings specific to this table or column family:
  
    hbase> alter ‘t1‘, CONFIGURATION => ‘hbase.hregion.scan.loadColumnFamiliesOnDemand‘ => ‘true‘
    hbase> alter ‘t1‘, NAME => ‘f2‘, CONFIGURATION => ‘hbase.hstore.blockingStoreFiles‘ => ‘10‘
  
  You can also remove a table-scope attribute:
  
    hbase> alter ‘t1‘, METHOD => ‘table_att_unset‘, NAME => ‘MAX_FILESIZE‘
  
    hbase> alter ‘t1‘, METHOD => ‘table_att_unset‘, NAME => ‘coprocessor$1‘
  
  You can also set REGION_REPLICATION:
  
    hbase> alter ‘t1‘, REGION_REPLICATION => 2
  
  There could be more than one alteration in one command:
  
    hbase> alter ‘t1‘,  NAME => ‘f1‘, VERSIONS => 3 , 
      MAX_FILESIZE => ‘134217728‘ ,  METHOD => ‘delete‘, NAME => ‘f2‘ ,
     OWNER => ‘johndoe‘, METADATA =>  ‘mykey‘ => ‘myvalue‘ 
  

   

数据导入导出

hbase org.apache.hadoop.hbase.mapreduce.ImportTsv \

-Dimporttsv.separator=, \

-Dimporttsv.columns="HBASE_ROW_KEY,order:numb,order:date" \

customer file:///home/vagrant/hbase_import_data.csv

 

s注：customer是要导入数据的表名，order.numb为列键

注2:这是sh命令，运行时如出现hdfs端无jar包文件异常，将本地hbase上lib目录拷贝到hdfs