• 后台运行python程序 遇到缓冲区问题


    From: http://www.iteye.com/topic/867446

    环境:linux

     

    一段执行时间很长的程序(用python做hive客户端执行mapreduce) 在linux后台执行,把结果输出到某文件:

     

     

    Python代码  收藏代码
    1. python xxx.py > log.log&   

     

     遇到的问题,程序没报错,文件里却什么都没有,没有输入进去。为什么呢?

    于是我首先尝试用:

     

     

    Java代码  收藏代码
    1. nohup python xxx.py > log.log &  

     

    预料之中 还是不行。

     

    于是我做实验:

     

    写了个test.py:

     

    Java代码  收藏代码
    1. import sys,time  
    2. from threading import Thread  
    3. class worker(Thread):  
    4.      def run(self):  
    5.          for x in xrange(0,111):  
    6.              print x  
    7.              time.sleep(1)  
    8. def run():  
    9.      worker().start()  
    10. if __name__ == '__main__':  
    11.     run()  

     每秒打印一次

    我直接用python text.py 执行  没问题  每秒输出一个数

    但我在后台执行:

     

    Python代码  收藏代码
    1. python test.py > log.log&   

     

    还是不行。开始不输出  直到程序执行完,一次性的写到log.log文件了。

    为什么呢? 

    原因可能是python 的print 先写到缓冲区了,还没flush到文件。

    于是加了一行“ sys.stdout.flush()” --在每次print后都flush一次。:

     

    Java代码  收藏代码
    1. import sys,time  
    2. from threading import Thread  
    3. class worker(Thread):  
    4.      def run(self):  
    5.          for x in xrange(0,111):  
    6.              print x  
    7.              sys.stdout.flush()    
    8.              time.sleep(1)  
    9. def run():  
    10.      worker().start()  
    11. if __name__ == '__main__':  
    12.     run()  
     

    问题解决。


    ===============================================================================

    还可以:python-u xxx.py > log.log & ,再细看下帮助文档:man python

    PYTHON(1)                                                            PYTHON(1)


    NAME
           python - an interpreted, interactive, object-oriented programming language


    SYNOPSIS
           python [ -d ] [ -E ] [ -h ] [ -i ] [ -m module-name ] [ -O ]
                  [ -Q argument ] [ -S ] [ -t ] [ -u ]
                  [ -v ] [ -V ] [ -W argument ] [ -x ] [ -3 ]
                  [ -c command | script | - ] [ arguments ]


    DESCRIPTION
           Python  is  an  interpreted,  interactive, object-oriented programming language that combines remarkable power with very clear syntax.
           For an introduction to programming in Python you are referred to the Python Tutorial.  The Python Library Reference documents built-in
           and  standard types, constants, functions and modules.  Finally, the Python Reference Manual describes the syntax and semantics of the
           core language in (perhaps too) much detail.  (These documents may be located via the INTERNET RESOURCES below; they may  be  installed
           on your system as well.)


           Python’s  basic  power  can  be  extended  with your own modules written in C or C++.  On most systems such modules may be dynamically
           loaded.  Python is also adaptable as an extension language for existing applications.  See the internal documentation for hints.


           Documentation for installed Python modules and packages can be viewed by running the pydoc program.


    COMMAND LINE OPTIONS
           -c command
                  Specify the command to execute (see next section).  This terminates the option list (following options are passed as  arguments
                  to the command).


           -d     Turn on parser debugging output (for wizards only, depending on compilation options).


           -E     Ignore environment variables like PYTHONPATH and PYTHONHOME that modify the behavior of the interpreter.


           -h     Prints the usage for the interpreter executable and exits.


           -i     When  a  script  is passed as first argument or the -c option is used, enter interactive mode after executing the script or the
                  command.  It does not read the $PYTHONSTARTUP file.  This can be useful to inspect global variables or a  stack  trace  when  a
                  script raises an exception.


           -m module-name
                  Searches sys.path for the named module and runs the corresponding .py file as a script.

          -u    Force stdin, stdout and stderr to be totallyunbuffered.  On systems where it matters, also put stdin,  stdout  and  stderr  in

                  binary  mode.   Note  that  there  is  internal  buffering in xreadlines(), readlines() and file-object iterators ("for line in
                  sys.stdin") which is not influenced by this option.  To work around this, you will want to use "sys.stdin.readline()" inside  a
                  "while 1:" loop.


           -v     Print  a  message  each  time a module is initialized, showing the place (filename or built-in module) from which it is loaded.
                  When given twice, print a message for each file that is checked for when searching for a module.  Also provides information  on
                  module cleanup at exit.


           -V     Prints the Python version number of the executable and exits.



  • 相关阅读:
    结对第一次—原型设计(文献摘要热词统计)
    第一次作业-准备篇
    软件工程实践总结
    团队作业第二次—项目选题报告
    软工实践|结对第二次—文献摘要热词统计
    软工实践|结对第一次—原型设计(文献摘要热词统计)
    第一次作业-准备篇
    个人作业——软件工程实践总结作业
    团队作业第二次——项目选题报告
    结对第二次—文献摘要热词统计
  • 原文地址:https://www.cnblogs.com/chenjianhong/p/4144326.html
Copyright © 2020-2023  润新知