• python利用eval方法提升dataframe运算性能


     eval方法可以直接利用c语言的速度,而不用分配中间数组,不需要中间内存的占用.

    如果包含多个步骤,每个步骤都要分配一块内存

    import numpy as np
    import pandas as pd
    import timeit


    df = pd.DataFrame({'a': np.random.randn(10000000),
    'b': np.random.randn(10000000),
    'c': np.random.randn(10000000),
    'x': 'x'})
    # print df
    start_time = timeit.default_timer()
    df['a']/( df['b']+0.1)-df['c']
    end_time = timeit.default_timer()
    print (end_time - start_time)
    print "___________________"
    start_time = timeit.default_timer()
    pd.eval("df['a']/( df['b']+0.1)-df['c']")
    end_time = timeit.default_timer(http://www.my516.com)
    print (end_time - start_time)
    运行时间对比 

    0.136633455546
    ___________________
    0.087637596342
    As of version 0.13 (released January 2014), Pandas includes some experimental tools that allow you to directly access C-speed operations without costly allocation of intermediate arrays.
    ---------------------

  • 相关阅读:
    学习进度条
    阅读《实例化需求》10-12章有感
    学习进度条
    阅读《实例化需求》7–9章有感
    学习进度条
    软件需求与分析课堂讨论一
    课程引言课后作业1
    MVC实例应用模式
    MVC模式介绍
    二十三种设计模式
  • 原文地址:https://www.cnblogs.com/ly570/p/11198599.html
Copyright © 2020-2023  润新知