将数据加载到表配置单元中
Posted
技术标签:
【中文标题】将数据加载到表配置单元中【英文标题】:load data into table hive 【发布时间】:2015-01-05 15:56:32 【问题描述】:我正在尝试配置 Hive 以使用 JDBC,我在 eclipse 上使用了这个示例:
public class HiveJdbcClient
private static String driverName = "org.apache.hadoop.hive.jdbc.HiveDriver";
/**
* @param args
* @throws SQLException
**/
public static void main(String[] args) throws SQLException
try
Class.forName(driverName);
catch (ClassNotFoundException e)
// TODO Auto-generated catch block
e.printStackTrace();
System.exit(1);
Connection con = DriverManager.getConnection("jdbc:hive://localhost:10000/default", "", "");
Statement stmt = con.createStatement();
String tableName = "testHiveDriverTable";
stmt.executeQuery("drop table " + tableName);
ResultSet res = stmt.executeQuery("create table " + tableName + " (key int, value string)");
// show tables
String sql = "show tables '" + tableName + "'";
System.out.println("Running: " + sql);
res = stmt.executeQuery(sql);
if (res.next())
System.out.println(res.getString(1));
// describe table
sql = "describe " + tableName;
System.out.println("Running: " + sql);
res = stmt.executeQuery(sql);
while (res.next())
System.out.println(res.getString(1) + "\t" + res.getString(2));
// load data into table
// NOTE: filepath has to be local to the hive server
// NOTE: /tmp/test_hive_server.txt is a ctrl-A separated file with two fields per line
String filepath = "/tmp/test_hive_server.txt";
sql = "load data local inpath '" + filepath + "' into table " + tableName;
System.out.println("Running: " + sql);
res = stmt.executeQuery(sql);
// select * query
sql = "select * from " + tableName;
System.out.println("Running: " + sql);
res = stmt.executeQuery(sql);
while (res.next())
System.out.println(String.valueOf(res.getInt(1)) + "\t" + res.getString(2));
// regular hive query
sql = "select count(1) from " + tableName;
System.out.println("Running: " + sql);
res = stmt.executeQuery(sql);
while (res.next())
System.out.println(res.getString(1));
我能够在 hive 上创建表,但是当我尝试将数据加载到表中时出现错误。所以我的问题是我应该在“test_hive_server.txt”中添加什么来使其工作!因为我尝试了一切,每次都遇到同样的错误。 谢谢!
错误:
Exception in thread "main" java.sql.SQLException: org.apache.thrift.TApplicationException: Internal error processing execute
at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:196)
at com.palmyra.nosql.HiveJdbcClient.main(HiveJdbcClient.java:52)
【问题讨论】:
请同时发布错误信息。 您可以查看此链接***.com/a/31130602 以获取自动配置单元脚本生成工具。 【参考方案1】:其实我自己找到了答案:
ROW FORMAT DELIMITED FIELDS TERMINATED BY ' ';
确定我们要加载到表中的“file.txt”中行的格式,Hive 的默认记录和字段分隔符列表是:
\n
^A
^B
^C
press ^V^A could insert a ^A in Vim.
或者你可以这样做:
create table tableName (key int, value string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ' ';
'file.txt' 应该是这样的:
value1 value2
value3 value4
在这个例子中 value1 和 value3 代表键 而value4和value5代表值
【讨论】:
以上是关于将数据加载到表配置单元中的主要内容,如果未能解决你的问题,请参考以下文章