Phoenix查询测试经验总结

Posted 2023-05-05

参考技术A 适当的索引能够让极大提升查询速度，因此在Phoenix查询的测试用例中包括了对有索引跟无索引的查询性能的比较。测试过程中遇到一些问题，在此记录下来。

</property>

<property>
<name>hbase.regionserver.wal.codec</name>
<value>org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec</value>
</property>
<property>
<name>hbase.region.server.rpc.scheduler.factory.class</name>
<value>org.apache.hadoop.hbase.ipc.PhoenixRpcSchedulerFactory</value>
<description>Factory to create the Phoenix RPC Scheduler that usesseparate queues for index and metadata updates</description>
</property>
<property>
<name>hbase.rpc.controllerfactory.class</name>
<value>org.apache.hadoop.hbase.ipc.controller.ServerRpcControllerFactory</value>
<description>Factory to create the Phoenix RPCScheduler that uses separate queues for index and metadataupdates</description>
</property>
<property>
<name>hbase.coprocessor.regionserver.classes</name>
<value>org.apache.hadoop.hbase.regionserver.LocalIndexMerger</value>
</property>

16/11/29 10:33:50 WARN client.ScannerCallable: Ignore, probably already closed
org.apache.hadoop.hbase.regionserver.LeaseException: org.apache.hadoop.hbase.regionserver.LeaseException: lease '1132' does not exist
at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:221)
at org.apache.hadoop.hbase.regionserver.Leases.cancelLease(Leases.java:206)

...

org.apache.phoenix.exception.PhoenixIOException: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions:
Tue Nov 29 10:33:50 CST 2016, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=60321: row '��s,d' on table 'TEST11' at region=TEST11,\x11\x00\x00\x00\x00\x00\x00\x00\x00,1479985615575.c3adb68acea8d88d223bffd3acc16c2e., hostname=node-20-105,60020,1480385981798, seqNum=1244662

...

Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=18173, waitTime=60001, operationTimeout=60000 expired.
at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1197)
...

<property>
<name>hbase.rpc.timeout</name>
<value>600000</value>
</property>

<property>
<name>hbase.client.operation.timeout</name>
<value>600000</value>
</property>

<property>
<name>hbase.client.scanner.timeout.period</name>
<value>600000</value>
</property>

<property>
<name>hbase.regionserver.lease.period</name>
<value>600000</value>
</property>

<property>
<name>phoenix.query.timeoutMs</name>
<value>600000</value>
</property>

<property>
<name>phoenix.query.keepAliveMs</name>
<value>600000</value>
</property>

<property>
<name>hbase.client.ipc.pool.type</name>
<value>RoundRobinPool</value>
</property>
<property>
<name>hbase.client.ipc.pool.size</name>
<value>10</value>
</property>

CREATE TABLE TABLENAME (pk long PRIMARY KEY,col1 int) IMMUTABLE_ROWS=true;

CREATE INDEX ind_name ON TABLENAME(COLUMN1);
CREATE INDEX ind_name ON TABLENAME(COLUMN1,COLUMN2);
CREATE INDEX ind_name ON TABLENAME(COLUMN1) INCLUDE(COLUMN2);

0: jdbc:phoenix:localhost> explain select ff3,if1 from testinput where ff3 >= 0.7 and ff3 < 0.9 order by if1;
+------------------------------------------+
| PLAN |
+------------------------------------------+
| CLIENT 1-CHUNK PARALLEL 1-WAY RANGE SCAN OVER IND_4 [0.7] - [0.9] |
| SERVER SORTED BY ["IF1"] |
| CLIENT MERGE SORT |
+------------------------------------------+

Phoenix&HBase实践——Phoenix+HBase结合实践操作总结

1、下载上传

下载和hbase版本兼容的tar，上传并解压tar

[hadoop@kb-testhadoop01 phoenix]$ tar -zxvf ../datas/apache-phoenix-4.15.0-HBase-1.4-bin.tar.gz -C ./

2、配置环境变量

环境变量配置
#PHOENIX_HOME
export PHOENIX_HOME=/home/hadoop/phoenix/apache-phoenix-4.15.0-HBase-1.4-bin
export PHOENIX_CLASSPATH=$PHOENIX_HOME
export PATH=$PATH:$PHOENIX_HOME/bin
#所有的都要配HBASE_HOME
#HBASE_HOME
export HBASE_HOME=/home/hadoop/hbase/hbase-1.4.9
export PATH=$PATH:$HBASE_HOME/bin
#配置完成后一定要让环境变量生效
source /etc/profile

3、模式选择

单点模式：

映射server和client这两个jar包拷贝到hbase/lib目录

ln -s /home/hadoop/phoenix/apache-phoenix-4.15.0-HBase-1.4-bin/phoenix-4.15.0-HBase-1.4-client.jar $HBASE_HOME/lib/phoenix-4.15.0-HBase-1.4-client.jar
ln -s /home/hadoop/phoenix/apache-phoenix-4.15.0-HBase-1.4-bin/phoenix-4.15.0-HBase-1.4-server.jar $HBASE_HOME/lib/phoenix-4.15.0-HBase-1.4-server.jar

集群模式：

在phoenix目录下在直接复制分发server.jar和client.jar到各个节点的hbase/lib目录下

cp phoenix-4.14.2-HBase-1.3-server.jar $HBASE_HOME/lib/phoenix-4.15.0-HBase-1.4-server.jar
scp phoenix-4.14.2-HBase-1.3-server.jar kb-testhadoopXX1:$HBASE_HOME/lib/phoenix-4.15.0-HBase-1.4-server.jar
scp phoenix-4.14.2-HBase-1.3-server.jar kb-testhadoopXX2:$HBASE_HOME/lib/phoenix-4.15.0-HBase-1.4-server.jar
...
cp phoenix-4.15.0-HBase-1.4-client.jar $HBASE_HOME/lib/phoenix-4.15.0-HBase-1.4-client.jar
scp phoenix-4.15.0-HBase-1.4-client.jar kb-testhadoopXX1:$HBASE_HOME/lib/phoenix-4.15.0-HBase-1.4-client.jar
scp phoenix-4.15.0-HBase-1.4-client.jar kb-testhadoopXX2:$HBASE_HOME/lib/phoenix-4.15.0-HBase-1.4-client.jar
...

4、修改hbase-site.sh

在hbase-site.sh这添加配置将协调报警关闭，保证Regionserver可以顺利启动

<property>
    <name>hbase.coprocessor.abortonerror</name>
    <value>false</value>
</property>

5、重启hbase

确保Hadoop、Zookeeper已经启动，重启hbase。

stop-habse.sh
start-hbase.sh

6、启动phoenix与退出

启动phoenix进入终端退出终端

单点启动

sqlline.py kb-testhadoop01:2181

集群启动

这里的hostname是指hbase集群配置的zookeeper集群的ip对应的hostname和端口

sqlline.py kb-testhadoop00,kb-testhadoop01:2181,kb-testhadoop02:2181

退出：

!quit