Oracle Sqoop 检索到 0 条记录

Posted 2023-04-18

【中文标题】Oracle Sqoop 检索到 0 条记录

【英文标题】：Oracle Sqoop Retrieves 0 Records

【发布时间】：2015-03-24 03:02:25

【问题描述】：

我在 Oracle XE 11g 中有一个表

SQL> create table bloblkup (
  2  id NUMBER PRIMARY KEY,
  3  name varchar(28) NOT NULL,
  4  fdata BLOB
  5  );

Table created.

SQL> desc bloblkup
 Name                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 ID                    NOT NULL NUMBER
 NAME                      NOT NULL VARCHAR2(28)
 FDATA                          BLOB

填充

SQL> select * from bloblkup;

    ID NAME
---------- ----------------------------
FDATA
--------------------------------------------------------------------------------
     1 photo.jpg
032135435135

我尝试从集群中将该表 Sqoop 到 HDFS 中

sqoop import --connect jdbc:oracle:thin:@Rhea:1521:xe --username SYSTEM --password oracle --table bloblkup --columns 'name' -m 1

每次都执行完成但提供INFO

15/03/24 09:14:39 INFO mapreduce.ImportJobBase: Retrieved 0 records.

我可以检索数据库和表。

我以创建并拥有该表的系统身份登录。

我还发现我可以查询诸如 all_tables 之类的表并返回行而不是我通过 sqlplus 创建的表>

由于无法找到表的主键，我在第一次尝试失败后添加了 --m 1 标志。我使用 alter table 向表中添加了一个主键，没有任何变化。

想法？

控制台输出：

[root@sandbox ~]# sqoop import --connect jdbc:oracle:thin:@Rhea:1521:xe --username SYSTEM --password oracle --table bloblkup --columns 'name' -m 1
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
15/03/24 09:14:02 INFO sqoop.Sqoop: Running Sqoop version: 1.4.4.2.1.1.0-385
15/03/24 09:14:02 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
15/03/24 09:14:02 INFO manager.SqlManager: Using default fetchSize of 1000
15/03/24 09:14:02 INFO tool.CodeGenTool: Beginning code generation
15/03/24 09:14:04 INFO manager.OracleManager: Time zone has been set to GMT
15/03/24 09:14:05 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM bloblkup t WHERE 1=0
15/03/24 09:14:05 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-root/compile/ce267f99c7e1b14da474c2c395368b67/bloblkup.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
15/03/24 09:14:08 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/ce267f99c7e1b14da474c2c395368b67/bloblkup.jar
15/03/24 09:14:08 INFO manager.OracleManager: Time zone has been set to GMT
15/03/24 09:14:08 INFO mapreduce.ImportJobBase: Beginning import of bloblkup
15/03/24 09:14:09 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
15/03/24 09:14:10 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
15/03/24 09:14:10 INFO client.RMProxy: Connecting to ResourceManager at sandbox.hortonworks.com/192.168.1.91:8050
15/03/24 09:14:12 INFO db.DBInputFormat: Using read commited transaction isolation
15/03/24 09:14:13 INFO mapreduce.JobSubmitter: number of splits:1
15/03/24 09:14:13 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1427151026592_0037
15/03/24 09:14:14 INFO impl.YarnClientImpl: Submitted application application_1427151026592_0037
15/03/24 09:14:14 INFO mapreduce.Job: The url to track the job: http://sandbox.hortonworks.com:8088/proxy/application_1427151026592_0037/
15/03/24 09:14:14 INFO mapreduce.Job: Running job: job_1427151026592_0037
15/03/24 09:14:27 INFO mapreduce.Job: Job job_1427151026592_0037 running in uber mode : false
15/03/24 09:14:27 INFO mapreduce.Job:  map 0% reduce 0%
15/03/24 09:14:38 INFO mapreduce.Job:  map 100% reduce 0%
15/03/24 09:14:39 INFO mapreduce.Job: Job job_1427151026592_0037 completed successfully
15/03/24 09:14:39 INFO mapreduce.Job: Counters: 30
    File System Counters
        FILE: Number of bytes read=0
        FILE: Number of bytes written=107031
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=87
        HDFS: Number of bytes written=0
        HDFS: Number of read operations=4
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=2
    Job Counters
        Launched map tasks=1
        Other local map tasks=1
        Total time spent by all maps in occupied slots (ms)=8553
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=8553
        Total vcore-seconds taken by all map tasks=8553
        Total megabyte-seconds taken by all map tasks=2138250
    Map-Reduce Framework
        Map input records=0
        Map output records=0
        Input split bytes=87
        Spilled Records=0
        Failed Shuffles=0
        Merged Map outputs=0
        GC time elapsed (ms)=76
        CPU time spent (ms)=2170
        Physical memory (bytes) snapshot=145907712
        Virtual memory (bytes) snapshot=897458176
        Total committed heap usage (bytes)=75497472
    File Input Format Counters
        Bytes Read=0
    File Output Format Counters
        Bytes Written=0
15/03/24 09:14:39 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 28.8478 seconds (0 bytes/sec)
15/03/24 09:14:39 INFO mapreduce.ImportJobBase: Retrieved 0 records.

【问题讨论】：

Sqoop 版本：1.4.4.2.1.1.0-385

java版本“1.7.0_45”

添加了 ojdbc7.jar 而不是 ojdbc6.jar 以匹配 java 版本

【参考方案1】：

更新 Oracle 驱动程序版本 ojdbc6.jar 似乎可以工作，但在 JDK 1.7 使用 ojdbc7.jar 时会出现问题

此外，您必须在 SQLPLUS 中“提交”数据库更改才能使其持久化

【参考方案2】：

检查jdbc驱动是否在$SQOOP_HOME/lib下，如果不复制ojdbc6.jar文件到：

/usr/lib/sqoop/lib/目录

从控制台提供更多详细信息。

如果一切正常，则添加 --target-dir 以查看该特定目录的输出。

        /usr/bin/sqoop import --connect jdbc:oracle:thin:system/system@<IP address>:1521:xe  --username <username> -P--table <database name>.<table name> --columns "<column names>" --target-dir  <target directory path> -m 1

【讨论】：

[root@sandbox lib]# ls -l /usr/lib/sqoop/lib 总计 5540 ... -rwxr-xr-x 1 root root 2152051 Mar 23 00:10 ojdbc6.jar

你能分享更多细节吗？

控制台中有消息为 ERROR util.SqlTypeMap: 看起来您正在查找一个没有的列。只需从您的 sqoop 语句中删除 --columns "ID,NAME" 并尝试。

在 Sqoop 语句中你有 --columns 'name'。 sqoop import --connect jdbc:oracle:thin:@Rhea:1521:xe --username SYSTEM --password oracle --table发酵器 --columns 'name' -m 1

原来的表现在可以工作，但是当我尝试新表时问题再次出现。我已经更新了新错误的问题。