oracle如何获取批量数据？

Posted 2023-03-28

【中文标题】oracle如何获取批量数据？

【英文标题】：How to fetch bulk data in oracle?

【发布时间】：2018-05-02 04:08:54

【问题描述】：

有一个名为 PROFIT 的表，其中包含一家公司的所有利润数据。该表包含超过 500 万条数据。表结构如下所示

COMPANYID   YEAR    PROFIT
   100001   2017     50000
   100001   2016     30000
   100001   2015     20000
   100002   2017     80000
   100002   2016     70000
   100002   2015     40000
   100003   2016    100000
   100003   2015     75000
   100004   2017     50000
   100004   2016     40000
   100004   2015     30000
   100004   2014     10000
   100004   2013      5000

我有一个包含 75000 companyid 的 Excel 列表，我需要获取所有这些公司的数据。我不能使用IN，因为它一次只允许列表中有 999 个值。

请帮助我。如何使用单个查询获取所有这些记录？目前我正在使用 Oracle 11g。

【问题讨论】：

在你之前的问题中，你说你可以通过sqlloader加载你的excel数据。为什么你不能在这里做同样的事情？将其加载到另一个临时表后，您可以执行 select *... in ( select from your_temp_table

以前我在开发环境中工作，但在这种情况下是生产环境。我只有读取权限。 :(

在这里你可以使用堆栈上的 TYPO 的这个答案。 link

如果是这种情况，我们几乎无能为力。您可能必须要求相关管理员为您临时创建生产表并让您运行查询，或者可能创建从 Prod 到 DEV 的数据库链接，并使用 db 链接从 Prod 连接 dev 表。外部表要求您创建一个目录，这在您的情况下似乎不太可能。

在 Oracle SQL developer 中创建与您的文件结构相同的表。将 Excel 数据加载到该表中。如有必要，您可以创建表约束和/或索引。使用简单的选择从新创建的表中插入到目标表中，或者合并，或者 PL/SQL 批量收集。

【参考方案1】：

假设 ...

您只想运行一条 SQL 语句 电子表格只是一个 ID 列表

...您可以编写这样的脚本。

将 Excel 电子表格另存为 .csv 在支持正则表达式的文本编辑器中打开.csv文件 使用正则表达式模式将每个数字变成一个选择语句：select \1 as co_id from dual union all 顶部和尾部输出以创建with 子句子查询 将其加入您的利润表

所以现在你有这样的东西：

with cte as (
    select 1 as co_id from dual union all
    select 2 as co_id from dual union all
    ....
    select 75000 as co_id from dual 
) 
select p.companyid
       , p.year
       , sum(p.profit) as annual_profit -- or whatever
from cte
     join profit p
       on cte.co_id = p.companyid
group by p.companyid
         , p.year

这不是动态解决方案，但静态脚本应该可以用于一次性练习。如果这是一份定期报告，您应该协商在生产中进行更改的权利，以便支持更好的方法。

【讨论】：

非常感谢。我检查了少量数据并且它正在工作。是的，这不是一份定期报告。我每月都会收到这些类型的要求。大多数时候，我得到两千或三千条记录。所以大多数时候我使用“IN”运算符并多次运行查询。但是这次记录数很大。非常感谢

【参考方案2】：

希望您可以在生产环境中运行 PL/SQL 代码。然后你可以试试下面的代码，它接受最多 4GB 的字符串，并在应用拆分器后以行的形式返回。

由于您在excel中有行，因此您可以复制所有后缀为“，”的行，然后删除新行，使其成为您的输入参数。并将所有这些值传递到单个配额或字符串分隔符中，就像我在下面的示例代码中传递的那样。

然后您可以将您的表传递给 IN 子句，就像我在“test_2”中传递的那样，一个虚拟表。

declare
v_input_str clob := '100002,100003,100004,100005,100006,100007,100008,100009,100010,100011,100012,100013,100014,100015,100016,100017,100018,100019,100020,100021,100022,100023,100024,100025,100026,100027,100028,100029,100030,100031,100032,100033,100034,100035,100036,100037,100038,100039,100040,100041,100042,100043,100044,100045,100046,100047,100048,100049,100050,100051,100052,100053,100054,100055,100056,100057,100058,100059,100060,100061,100062,100063,100064,100065,100066,100067,100068,100069,100070,100071,100072,100073,100074,100075,100076,100077,100078,100079,100080,100081,100082,100083,100084,100085,100086,100087,100088,100089,100090,100091,100092,100093,100094,100095,100096,100097,100098,100099,100100,100101,100102,100103,100104,100105,100106,100107,100108,100109,100110,100111,100112,100113,100114,100115,100116,100117,100118,100119,100120,100121,100122,100123,100124,100125,100126,100127,100128,100129,100130,100131,100132,100133,100134,100135,100136,100137,100138,100139,100140,100141,100142,100143,100144,100145,100146,100147,100148,100149,100150,100151,100152,100153,100154,100155,100156,100157,100158,100159,100160,100161,100162,100163,100164,100165,100166,100167,100168,100169,100170,100171,100172,100173,100174,100175,100176,100177,100178,100179,100180,100181,100182,100183,100184,100185,100186,100187,100188,100189,100190,100191,100192,100193,100194,100195,100196,100197,100198,100199,100200,100201,100202,100203,100204,100205,100206,100207,100208,100209,100210,100211,100212,100213,100214,100215,100216,100217,100218,100219,100220,100221,100222,100223,100224,100225,100226,100227,100228,100229,100230,100231,100232,100233,100234,100235,100236,100237,100238,100239,100240,100241,100242,100243,100244,100245,100246,100247,100248,100249,100250,100251,100252,100253,100254,100255,100256,100257,100258,100259,100260,100261,100262,100263,100264,100265,100266,100267,100268,100269,100270,100271,100272,100273,100274,100275,100276,100277,100278,100279,100280,100281,100282,100283,100284,100285,100286,100287,100288,100289,100290,100291,100292,100293,100294,100295,100296,100297,100298,100299,100300,100301,100302,100303,100304,100305,100306,100307,100308,100309,100310,100311,100312,100313,100314,100315,100316,100317,100318,100319,100320,100321,100322,100323,100324,100325,100326,100327,100328,100329,100330,100331,100332,100333,100334,100335,100336,100337,100338,100339,100340,100341,100342,100343,100344,100345,100346,100347,100348,100349,100350,100351,100352,100353,100354,100355,100356,100357,100358,100359,100360,100361,100362,100363,100364,100365,100366,100367,100368,100369,100370,100371,100372,100373,100374,100375,100376,100377,100378,100379,100380,100381,100382,100383,100384,100385,100386,100387,100388,100389,100390,100391,100392,100393,100394,100395,100396,100397,100398,100399,100400,100401,100402,100403,100404,100405,100406,100407,100408,100409,100410,100411,100412,100413,100414,100415,100416,100417,100418,100419,100420,100421,100422,100423,100424,100425,100426,100427,100428,100429,100430,100431,100432,100433,100434,100435,100436,100437,100438,100439,100440,100441,100442,100443,100444,100445,100446,100447,100448,100449,100450,100451,100452,100453,100454,100455,100456,100457,100458,100459,100460,100461,100462,100463,100464,100465,100466,100467,100468,100469,100470,100471,100472,100473,100474,100475,100476,100477,100478,100479,100480,100481,100482,100483,100484,100485,100486,100487,100488,100489,100490,100491,100492,100493,100494,100495,100496,100497,100498,100499,100500,100501,100502,100503,100504,100505,100506,100507,100508,100509,100510,100511,100512,100513,100514,100515,100516,100517,100518,100519,100520,100521,100522,100523,100524,100525,100526,100527,100528,100529,100530,100531,100532,100533,100534,100535,100536,100537,100538,100539,100540,100541,100542,100543,100544,100545,100546,100547,100548,100549,100550,100551,100552,100553,100554,100555,100556,100557,100558,100559,100560,100561,100562,100563,100564,100565,100566,100567,100568,100569,100570,100571,100572,100573';
v_count number;
begin

select count(1) into v_count
from test_2 
where id in (with i as (select v_input_str as str from dual)
                select to_number(regexp_substr(str, '[^,]+', 1, level)) in_clause_values
                from i connect by level <= length(regexp_replace(str, '[^,]+')) + 1
            );

dbms_output.put_line('No of elements : '||v_count);

end;

【讨论】：

删除 WITH 子句，因为它不再需要，从 test_2 中选择 count(1) 到 v_count where id in (select to_number(regexp_substr(v_input_str, '[^,]+', 1, level) ) 来自双重连接的 in_clause_values 按级别