当我运行命令计划laravel时，如何解决“致命错误：内存不足...”？

Posted 2021-04-12

我的命令是这样的：

<?php
namespace AppConsoleCommands;
use IlluminateConsoleCommand;
use AppModelsItemDetail;
class ImportItemDetail extends Command
{
    protected $signature = 'sync:item-detail';
    protected $description = 'sync item detail';
    public function __construct()
    {
        parent::__construct();
        ini_set('max_execution_time', 0);
        ini_set('memory_limit', '-1');
    }
    public function handle()
    {
        $path = storage_path('json/ItemDetail.json');
        $json = json_decode(file_get_contents($path), true);
        foreach ($json['value'] as $value) {
            ItemDetail::updateOrCreate(
                [ 'entry_number' => $value['Entry_Number'] ],
                [ 
                    'item_number' => $value['Item_Number'],
                    'description' => $value['Description'],
                    ....
                ]
            );
        }
    }
}

如果我运行该命令，则存在如下错误：

Fatal error: Out of memory (allocated 255852544) (tried to allocate 8192 bytes)

虽然我在构造中设置了ini_set('memory_limit', '-1');

我该如何解决错误？

注意 ：

我在json文件中的记录包含数十万个数据。因此，检查该字段需要很长时间

更新

我的ItemDetail.json是这样的：

{
  "@odata.context":"http://myapp-svr01.www.myapp.com:1224/DynamicsNAV100/ODataV4/$metadata#Collection(NAV.ODATAITEM)","value":[
    {
      "@odata.etag":"W/"JzE2O0lBQUFBQUNIbXdrQ0FBQAD5OzE4ODQ1MDQ4NjA7Jw=="","Entry_Number":123,"Item_Number":"9805010101","Variant_Code":"039","Posting_Date":"2018-01-02T00:00:00Z","Entry_Type":"Sale","Description":"MISC BISBAN POLOS 11MM A847","Quantity":-7200,"Source_Type":"Customer","Order_Type":" ","Sales_Amount_Actual":1800000,"ETag":"16;IAAAAACHmwkCAAAA9;1884504860;"
    },{
      "@odata.etag":"W/"JzE2O0lBQUFBQUNIbkFrQ0FBQAD5OzE4ODQ1MDQ5MTA7Jw=="","Entry_Number":124,"Item_Number":"9805010102","Variant_Code":"100","Posting_Date":"2018-01-02T00:00:00Z","Entry_Type":"Sale","Description":"MISC BISBAN POLOS 11MM A915","Quantity":-7200,"Source_Type":"Customer","Order_Type":" ","Sales_Amount_Actual":1800000,"ETag":"16;IAAAAACHnAkCAAAA9;1884504910;"
    }
  ]
}

我只在json上面给了2条记录。实际上有大约150,000条记录

答案

问题是你是在一次读取整个文件，而是应该以块或逐行读取它。例如：

$items = [];
$file = fopen(storage_path('json/ItemDetail.json'),'r');
while (!feof($file)){
    $line = fgets($file);
    $obj = json_decode($line);
    // $obj is type stdClass
    // transform $obj to fit the update or create method
    ItemDetail::updateOrCreate( ... )
}

更新

在处理大文件时，您应该查看流式解析器。

jsonstreamingparser将是一个不错的选择。