• php导出大数据scv


    1.我系统要导出30w的数据,刚开始我使用的是:

    ini_set('memory_limit', '1024M');
    set_time_limit(0);
    header("Content-type:text/csv");
    header("Content-Disposition:attachment;filename=收款明细.csv");
    header('Cache-Control:must-revalidate,post-check=0,pre-check=0');
    header('Expires:0');
    header('Pragma:public');
    (1) $str= "订单编号,来源店铺,付款ID,付款日期,付款金额,付款方式,付款币种,单据类型,记账币种,所属法人,凭证编码
    ";
    (2) $result = 查询30w的数据。
    foreach ($result as $item) {
    $str .= $item['order_no'] . ",";
    $str .= $item['order_from'] . ",";
    $str .= $item['payment_id'] . ",";
    $str .= date('Y-m-d ', strtotime($item['payment_time'])) . ",";
    $str .= $item['payment_amount'] . ",";
    $str .= $item['payment_method'] . ",";
    $str .= $item['payment_currency'] . ",";
    $str .= $documentType[$item['document_type']] . ",";
    $str .= $item['account_currency'] . ",";
    $str .= $item['corporate_sn'] . ",";
    $str .= $item['certificate_code'] . ",";
    $str .= " ";
    }
    echo mb_convert_encoding($str, "CP936", "UTF-8");
    unset($str);
    return ;
    描述:我要导出30w的大数据,(2) 一下子查询30w的数据可能会导致查询连接数据库超时 (1) 如果查询30w的数据查出来了,然后拼接到一个变量上,变量也可能会导致字符串太长而报错。

    所以我们想到查询的时候进行切割,分页去查询。在字符串拼接的时候,可以转换字符串然后echo,优化代码如下:


    ini_set('memory_limit', '1024M');
    set_time_limit(0);
    header("Content-type:text/csv");
    header("Content-Disposition:attachment;filename=收款明细.csv");
    header('Cache-Control:must-revalidate,post-check=0,pre-check=0');
    header('Expires:0');
    header('Pragma:public');
    $top= "订单编号,来源店铺,付款ID,付款日期,付款金额,付款方式,付款币种,单据类型,记账币种,所属法人,凭证编码
    ";
    echo mb_convert_encoding($top, "CP936", "UTF-8");
    $count = 查询总数的sql语句。

    $pre_count=50000;
    //循环获取
    for($i=0;$i<intval($count/$pre_count)+1;$i++){

    $result = 分页查询返回结果。
    foreach ($result as $item) {
    $str .= $item['order_no'] . ",";
    $str .= $item['order_from'] . ",";
    $str .= $item['payment_id'] . ",";
    $str .= date('Y-m-d ', strtotime($item['payment_time'])) . ",";
    $str .= $item['payment_amount'] . ",";
    $str .= $item['payment_method'] . ",";
    $str .= $item['payment_currency'] . ",";
    $str .= $documentType[$item['document_type']] . ",";
    $str .= $item['account_currency'] . ",";
    $str .= $item['corporate_sn'] . ",";
    $str .= $item['certificate_code'] . ",";
    $str .= " ";
    echo mb_convert_encoding($str, "CP936", "UTF-8");
    unset($str);
    }
    }
    return ;



  • 相关阅读:
    js数组的基本用法及数组根据下标(数值或字符)移除元素
    Oracle备份一张表
    linux中常见的文件操作命令
    java图片二进制相互转换
    getParameterMap的使用
    前端常用
    Oracle 常用
    JAVA中int、String的类型转换
    MySQL 5.7 新特性大全和未来展望
    你有自己的Web缓存知识体系吗?
  • 原文地址:https://www.cnblogs.com/kobigood/p/5584140.html
Copyright © 2020-2023  润新知