前两天被问到常用的python lib和module有哪些？最常用的那几个，其他的一下子竟然回答不上。想想也是，一般情况下，遇到一个问题，在网上一搜，顺着线索找到可用的例子，然后基本没有怎么深究。结果就导致了现在没有网络基本写不出代码的状况，搜索是程序员必须的技能，但是只依靠搜索例子是无法成长的。
今天抽出1个小时时间把一些重要的module的简要整理出来，逐步一个个熟悉。
0. Date Structures and Algorithms
abc
The abc module defines a metaclass and a pair of decorators for defining new abstract
base classes.
array
This module defines an object type which can compactly represent an array of basic values: characters, integers, floating point numbers
The array module defines a new object type, array, that works almost exactly like a list, except that its contents are constrained to a single type.The type of an array is determined at the time of creation.
bisect
The bisect module provides support for keeping lists in sorted order. It uses a bisection algorithm to do most of its work.
This module provides support for maintaining a list in sorted order without having to sort the list after each insertion. For long lists of items with expensive comparison operations, this can be an improvement over the more common approach. The module is called
bisect because it uses a basic bisection algorithm to do its work.
collections
The collections module contains high-performance implementations of a few useful container types, abstract base classes for various kinds of containers, and a utility function for creating name-tuple objects.
This module implements specialized container datatypes providing alternatives to Python’s general purpose built-in containers, dict, list, set, and tuple.
| namedtuple() | factory function for creating tuple subclasses with named fields |
New in version 2.6. |
| deque | list-like container with fast appends and pops on either end |
New in version 2.4. |
| Counter | dict subclass for counting hashable objects |
New in version 2.7. |
| OrderedDict | dict subclass that remembers the order entries were added |
New in version 2.7. |
| defaultdict | dict subclass that calls a factory function to supply missing values |
New in version 2.5. |
contextlib
The contextlib module provides a decorator and utility functions for creating context managers used in conjunction with the
with statement.
functools
The functools module contains functions and decorators that are useful for creating higher-order functions, functional programming, and decorators.
The
functools module is for higher-order functions: functions that act on or return other functions. In general, any callable object can be treated as a function for the purposes of this module.
heapq
The heapq module implements a priority queue using a heap.
This module provides an implementation of the heap queue algorithm, also known as the priority queue algorithm.
itertools
The itertools module contains functions for creating efficient iterators, useful for looping over data in various ways.
This module implements a number of
iterator building blocks inspired by constructs from APL, Haskell, and SML. Each has been recast in a form suitable for Python.
operator
The operator module provides functions that access the built-in operators
The
operator module exports a set of efficient functions corresponding to the intrinsic operators of Python.
1. Files and Dictionary
bz2
The bz2 module is used to read and write data compressed according to the bzip2 compression algorithm.
filecmp
The filecmp module provides the functions, which can be used to compare files and directories.
fnmathch
The fnmatch module provides support for matching filenames using UNIX shell-style wildcard characters.This module only performs filename matching, whereas the glob module can be used to actually obtain file listings.
glob
The glob module returns all filenames in a directory that match a pattern specified using the rules of the UNIX shell (as described in the fnmatch module).
gzip
The gzip module provides a class, GzipFile, that can be used to read and write files compatible with the GNU gzip program. GzipFile objects work like ordinary files except that data is automatically compressed or decompressed.
shutil
The shutil module is used to perform high-level file operations such as copying, removing, and renaming.The functions in this module should only be used for proper files and directories. In particular, they do not work for special kinds of files on the file
system such as named pipes, block devices, etc. Also, be aware that these functions don't always correctly deal with advanced kinds of file metadata (e.g., resource forks, creator codes, etc.).
tarfile
The tarfile module is used to manipulate tar archive files. Using this module, it is possible to read and write tar files, with or without compression.
tempfile
The tempfile module is used to generate temporary filenames and files.
zipfile
The zipfile module is used to manipulate files encoded in the popular zip format
(originally known as PKZIP, although now supported by a wide variety of programs).
Zip files are widely used by Python, mainly for the purpose of packaging.
zlib
The zlib module supports data compression by providing access to the zlib library.
The zlib library is available at
http://www.zlib.net.
2. Operating System Services
commands
The commands module is used to execute simple system commands specified as a string and return their output as a string. It only works on UNIX systems.The functionality provided by this module is somewhat similar to using backquotes (`) in a UNIX shell script. For example, typing x = commands.getoutput('ls –l') is similar to saying x=`ls –l`.
ConfigParser
The ConfigParser module (called configparser in Python 3) is used to read .ini format configuration files based on the Windows INI format.These files consist of named sections, each with its own variable assignments such as the following:
# A comment
; A comment
[section1]
name1 = value1
name2 = value2
[section2]
; Alternative syntax for assigning values
name1: value1
name2: value2
datetime
The datetime module provides a variety of classes for representing and manipulating dates and times. Large parts of this module are simply related to different ways of creating and outputting date and time information. Other major features include mathematical operations such as comparisons and calculations of time deltas. Date manipulation is a complex subject, and readers would be strongly advised to consult Python's online documentation for an introductory background concerning the design of this module.
A
date object represents a simple date consisting of a year, month, and day.The following four functions are used to create dates:
date(year, month, day)
time
The time module provides various time-related functions. In Python, time is measured as the number of seconds since the epoch.The epoch is the beginning of time (the point at which time = 0 seconds).The epoch is January 1, 1970, on UNIX and can be determined by calling time.gmtime(0) on other systems.
errno
The errno module defines symbolic names for the integer error codes returned by various
operating system calls, especially those found in the os and socket modules.These
codes are typically found in the errno attribute of an OSError or IOError exception.
fcntl
The fcntl module performs file and I/O control on UNIX file descriptors. File descriptors can be obtained using the fileno() method of a file or socket object.
io
The io module implements classes for various forms of I/O as well as the built-in open() function that is used in Python 3.The module is also available for use in Python 2.6.
logging
The logging module provides a flexible facility for applications to log events, errors,
warnings, and debugging information.This information can be collected, filtered, written
to files, sent to the system log, and even sent over the network to remote machines.
This section covers the essential details of using this module for most common cases.
mmap
The mmap module provides support for a memory-mapped file object.This object behaves both like a file and a byte string and can be used in most places where an ordinary file or byte string is expected. Furthermore, the contents of a memory-mapped file are mutable.This means that modifications can be made using index-assignment and slice-assignment operators. Unless a private mapping of the file has been made, such changes directly alter the contents of the underlying file.
msvcrt
The msvcrt module provides access to a number of useful functions in the Microsoft Visual C runtime library.This module is available only on Windows.
optparse
The optparse module provides high-level support for processing UNIX-style command-line options supplied in sys.argv.
os
The os module provides a portable interface to common operating-system services. It does this by searching for an OS-dependent built-in module such as nt or posix and exporting the functions and data as found there. Unless otherwise noted, functions are available on Windows and UNIX. UNIX systems include both Linux and Mac OS X.
os.path
The os.path module is used to manipulate pathnames in a portable manner. It's imported by the os module.
signal
The signal module is used to write signal handlers in Python. Signals usually correspond to asynchronous events that are sent to a program due to the expiration of a timer, arrival of incoming data, or some action performed by a user.The signal interface emulates that of UNIX, although parts of the module are supported on other platforms.
subprocess
The subprocess module contains functions and objects that generalize the task of creating new processes, controlling input and output streams, and handling return codes.
The module centralizes functionality contained in a variety of other modules such as os, popen2, and commands.
winreg
The winreg module (_winreg in Python 2) provides a low-level interface to the Windows registry.The registry is a large hierarchical tree in which each node is called a key.The children of a particular key are known as subkeys and may contain additional subkeys or values.
3. Thread and Concurrency
multiprocessing
The multiprocessing module provides support for launching tasks in a subprocess, communicating and sharing data, and performing various forms of synchronization.The programming interface is meant to mimic the programming interface for threads in the threading module. However, unlike threads, it is important to emphasize that processes do not have any shared state.Thus, if a process modifies data, that change is local only to that process.
The features of the multiprocessing module are vast, making it one of the larger and most advanced built-in libraries.
processes
All of the features of the multiprocessing module are focused on processes.They are described by the following class
Process([group [, target [, name [, args [, kwargs]]]]])
A class that represents a task running in a subprocess.The arguments in the constructor should always been specified using keyword arguments. target is a callable object that will execute when the process starts, args is a tuple of positional arguments passed to target, and kwargs is a dictionary of keyword arguments passed to target. If args and kwargs are omitted, target is called with no arguments. name is a string that gives a descriptive name to the process. group is unused and is always set to None. Its presence here is simply to make the construction of a Process mimic the creation of a thread in the threading module.
threading
The threading module provides a Thread class and a variety of synchronization primitives for writing multithreaded programs.
Thread Objects
The Thread class is used to represent a separate thread of control. A new thread can be created as follows:
Thread(group=None, target=None, name=None, args=(), kwargs={})
This creates a new Thread instance. group is None and is reserved for future extensions. target is a callable object invoked by the run() method when the thread starts. By default, it’s None, meaning that nothing is called. name is the thread name.
queue
The queue module (named Queue in Python 2) implements various multiproducer, multiconsumer queues that can be used to safely exchange information between multiple threads of execution.
The queue module defines three different queue classes:
Queue([maxsize])
Creates a FIFO (first-in first-out) queue. maxsize is the maximum number of items that can be placed in the queue. If maxsize omitted or 0, the queue size is infinite.
LifoQueue([maxsize])
Creates a LIFO (last-in, first-out) queue (also known as a stack).
PriorityQueue([maxsize])
Creates a priority queue in which items are ordered from lowest to highest priority.
When working with this queue, items should be tuples of the form (priority, data) where priority is a number.
4. Net Programming and Socket
asynchat
The asynchat module simplifies the implementation of applications that implement asynchronous networking using the asyncore module. It does this by wrapping the low-level I/O functionality of asyncore with a higher-level programming interface that is designed for network protocols based on simple request/response mechanisms (for example, HTTP).
asyncore
The
asyncore module is used to build network applications in which network activity is handled asynchronously as a series of events dispatched by an event loop, built using the
select() system call. Such an approach is useful in network programs that want to provide concurrency, but without the use of threads or processes.This method can also provide high performance for short transactions. All the functionality of this module is provided by the dispatcher class, which is a thin wrapper around an ordinary socket object.
select
The select module provides access to the select() and poll() system calls.
socket
The socket module provides access to the standard BSD socket interface. Although it's based on UNIX, this module is available on all platforms.The socket interface is designed to be generic and is capable of supporting a wide variety of networking protocols (Internet,TIPC, Bluetooth, and so on). However, the most common protocol is the Internet Protocol (IP), which includes both TCP and UDP. Python supports both IPv4 and IPv6, although IPv4 is far more common.
ssl
The ssl module is used to wrap socket objects with the Secure Sockets Layer (SSL), which provides data encryption and peer authentication. Python uses the OpenSSL library (http://www.openssl.org) to implement this module.
socketserver
This module is called socketserver in Python 3.The SocketServer module provides classes that simplify the implementation of TCP, UDP, and UNIX domain socket servers.
5. NETWORK
telnetlib
The
telnetlib module provides a
Telnet class that implements the Telnet protocol.
imaplib
(NNTP protocol)
This module defines three classes,
IMAP4,
IMAP4_SSL and
IMAP4_stream, which encapsulate a connection to an IMAP4 server and implement a large subset of the IMAP4rev1 client protocol as defined in
RFC 2060. It is backward compatible with IMAP4 (
RFC 1730) servers, but note that the
STATUS command is not supported in IMAP4.
nntplib
(NNTP protocol)
This module defines the class
NNTP which implements the client side of the NNTP protocol. It can be used to implement a news reader or poster, or automated news processors. For more information on NNTP (Network News Transfer Protocol), see Internet
RFC 977.
poplib
This module defines a class,
POP3, which encapsulates a connection to a POP3 server and implements the protocol as defined in
RFC 1725.
smtpd
This module offers several classes to implement SMTP servers. One is a generic do-nothing implementation, which can be overridden, while the other two offer specific mail-sending strategies.