从db创建CSV的PHP内存限制 - 如何减少PHP使用的内存？

Question

我在这里运行db查询，并使用mysqli和MYSQLI_USE_RESULT以更快地执行查询。但是这里需要对数据进行php操作，因此它并不像在返回的每一行上写入CSV文件那么简单。我知道数据库查询正在完成（即使它花了很长时间），但db查询确实完成了，所以问题必须在PHP内存端，因为我得到504网关超时错误。我的代码如下：

global $wpdb, $root_dir;

if (!defined('ABSPATH'))
    $root_dir = dirname(__FILE__) . '/';
else
    $root_dir = ABSPATH;

require_once($root_dir . 'wp-config.php');
$wp->init();
$wp->parse_request();
$wp->query_posts();
$wp->register_globals();

$start_date = !empty($_GET['start']) ? DateTime::createFromFormat('Y-m-d', $_GET['start']) : '';
$end_date = !empty($_GET['end']) ? DateTime::createFromFormat('Y-m-d', $_GET['end']) : '';

$append = array();

if (!empty($start_date))
    $append[] = $start_date->format('Y-m-d');

if (!empty($end_date))
    $append[] = $end_date->format('Y-m-d');

$filename = 'user-export' . (!empty($append) ? '_' . implode('_', $append) : '');

header("Cache-Control: must-revalidate, post-check=0, pre-check=0");
header('Content-Description: File Transfer');
header("Content-type: text/csv");
header("Content-Disposition: attachment; filename="" . $filename . ".csv"");
header("Expires: 0");
header("Pragma: public");


ini_set('memory_limit', '-1');
ini_set('max_execution_time', '-1');
set_time_limit(0);

$headers = array('Order Date/Time', 'Name', 'Billing Address', 'Shipping Address', 'Phone', 'Email Address');
$meta_keys = array('_billing_first_name', '_billing_last_name', '_billing_address_1', '_billing_address_2', '_billing_city', '_billing_state', '_billing_postcode', '_billing_email', '_billing_country', '_billing_phone', '_shipping_address_1', '_shipping_address_2', '_shipping_city', '_shipping_state', '_shipping_postcode', '_shipping_country');
$csv_data = $emails = $order_ids_skipped = $meta = array();

// Using an unbuffered query...
$mysqli  = new mysqli(DB_HOST, DB_USER, DB_PASSWORD, DB_NAME);

if (mysqli_connect_errno()) {
    printf("Connect failed: %s
", mysqli_connect_error());
    exit();
}

$uresult = $mysqli->query("
    SELECT p.ID, p.post_date, pm.meta_key, pm.meta_value
    FROM {$wpdb->posts} AS p 
    INNER JOIN {$wpdb->postmeta} AS pm ON (pm.post_id = p.ID AND pm.meta_key IN ('" . implode("','", $meta_keys) . "'))
    WHERE p.post_type = 'shop_order' AND p.post_status IN ('wc-sent-delivery', 'wc-awaiting-delivery', 'wc-completed')" . (!empty($start_date) && !empty($end_date) ? " AND (p.post_date >= '" . $start_date->format('Y-m-d') . " 00:00:00' AND p.post_date <= '" . $end_date->format('Y-m-d') . " 23:59:59')" : "") . "
    ORDER BY p.post_date DESC", MYSQLI_USE_RESULT);

if ($uresult) 
{
    while ($order_query = $uresult->fetch_assoc())
    {
        // If the email already exists, continue...
        if ($order_query['meta_key'] == '_billing_email' && !empty($emails) && in_array($order_query['meta_value'], $emails))
        {
            if (isset($csv_data[$order_query['ID']]))
                unset($csv_data[$order_query['ID']]);

            if (isset($meta[$order_query['ID']]))
                unset($meta[$order_query['ID']]);

            $order_ids_skipped[] = $order_query['ID'];

            continue;
        }

        if (in_array($order_query['ID'], $order_ids_skipped)) continue;

        if (!isset($csv_data[$order_query['ID']]))
            $csv_data[$order_query['ID']] = array(
                'order_date' => $order_query['post_date']
            );

        $meta[$order_query['ID']][$order_query['meta_key']] = $order_query['meta_value'];

        if ($order_query['meta_key'] == '_billing_email')
            $emails[] = $order_query['meta_value'];
    }
}
$uresult->close();

if (!empty($meta))
{
    foreach($meta as $order_id => $meta_data)
    {
        $billing = array(
            'addr1' => !empty($meta_data['_billing_address_1']) ? $meta_data['_billing_address_1'] : '',
            'addr2' => !empty($meta_data['_billing_address_2']) ? $meta_data['_billing_address_2'] : '',
            'city' => !empty($meta_data['_billing_city']) ? $meta_data['_billing_city'] : '',
            'state' => !empty($meta_data['_billing_state']) ? $meta_data['_billing_state'] : '',
            'zip' => !empty($meta_data['_billing_postcode']) ? $meta_data['_billing_postcode'] : '',
            'country' => !empty($meta_data['_billing_country']) ? $meta_data['_billing_country'] : ''
        );
        $shipping = array(
            'addr1' => !empty($meta_data['_shipping_address_1']) ? $meta_data['_shipping_address_1'] : '',
            'addr2' => !empty($meta_data['_shipping_address_2']) ? $meta_data['_shipping_address_2'] : '',
            'city' => !empty($meta_data['_shipping_city']) ? $meta_data['_shipping_city'] : '',
            'state' => !empty($meta_data['_shipping_state']) ? $meta_data['_shipping_state'] : '',
            'zip' => !empty($meta_data['_shipping_postcode']) ? $meta_data['_shipping_postcode'] : '',
            'country' => !empty($meta_data['_shipping_country']) ? $meta_data['_shipping_country'] : ''
        );

        $csv_data[$order_id]['name'] = !empty($meta_data['_billing_last_name']) ? trim($meta_data['_billing_first_name']) . ' ' . trim($meta_data['_billing_last_name']) : trim($meta_data['_billing_first_name']);
        $csv_data[$order_id]['billing'] = implode(', ', array_filter($billing));
        $csv_data[$order_id]['shipping'] = implode(', ', array_filter($shipping));
        $csv_data[$order_id]['phone'] = !empty($meta_data['_billing_phone']) ? $meta_data['_billing_phone'] : '';
        $csv_data[$order_id]['email'] = !empty($meta_data['_billing_email']) ? $meta_data['_billing_email'] : '';
    }


    if (!empty($csv_data))
        build_csv_file($csv_data, $headers);
}

// No need to continue here.
exit;

function str_putcsv($input, $delimiter = ',', $enclosure = '"')
{
    // $fp = fopen('php://temp', 'r+b');
    $fp = fopen('php://output', 'r+b');
    fputcsv($fp, $input, $delimiter, $enclosure);
    rewind($fp);
    rtrim(stream_get_contents($fp), "
");
    fclose($fp);
    // return $data;
}

function build_csv_file($data, $headers)
{
    global $root_dir;

    $output = '';

    if (empty($data))
        return array();

    if (!empty($headers))
        str_putcsv($headers, ',', '"') . PHP_EOL;

    foreach($data as $part)
        str_putcsv($part, ',', '"') . PHP_EOL;
}

将此文件保存为testing.php并从wordpress的根目录执行以下url：

https://example.com/testing.php?start=2010-03-05&end=2018-03-05

只返回504网关错误。这绝对是一个PHP内存限制问题，但是，我认为可以流式传输CSV文件，甚至可以在这里使用php://output。不知道如何更好地处理这个问题，以减少正在使用的PHP内存，或者如果有一种方法可以正确地执行此操作，同时仍然正确地维护数据。

有人可以帮我吗？非常感谢。我相信我使用的是PHP 7.2版。你认为这些天php可以更好地处理内存。我相信我的PHP内存最大可达1024MB。我似乎一直在超过PHP内存限制。正在成为一个真正的麻烦。