• 第十章 同步设备I/O与异步设备I/O


    //1.
    CreateFile 用于打开和创建磁盘文件,也可以打开许多其他设备(目录、磁盘驱动器、串口、并口、邮件槽、管道等)
    https://msdn.microsoft.com/en-us/library/windows/desktop/aa363858(v=vs.85).aspx
    HANDLE WINAPI CreateFileW(
    __in     LPCWSTR lpFileName,							//文件名或者设备
    __in     DWORD dwDesiredAccess,							//访问方式
    __in     DWORD dwShareMode,								//共享方式
    __in_opt LPSECURITY_ATTRIBUTES lpSecurityAttributes,	//安全属性,一般为NULL
    __in     DWORD dwCreationDisposition,					//打开方式
    __in     DWORD dwFlagsAndAttributes,
    __in_opt HANDLE hTemplateFile
    );
    
    (A):lpFileName:
    	The name of the file or device to be created or opened. You may use either forward slashes (/) or backslashes () in this name.
    (B):dwDesiredAccess:
    	0:不读也不写,如果只想改变设备的配置(比如修改文件的时间戳),那么可以传0
    	GENERIC_READ:Read access
    	GENERIC_WRITE:Write access
    	GENERIC_EXECUTE:Execute access
    	GENERIC_ALL:All possible access rights
    (C):dwShareMode:
    	0:Prevents other processes from opening a file or device if they request delete, read, or write access
    	FILE_SHARE_DELETE:感觉没什么鸟用,暂不研究
    	FILE_SHARE_READ:Enables subsequent open operations on a file or device to request read access.
    					Otherwise, other processes cannot open the file or device if they request read access.
    					If this flag is not specified, but the file or device has been opened for read access, the function fails.
    	FILE_SHARE_WRITE:Enables subsequent open operations on a file or device to request write access.
    					Otherwise, other processes cannot open the file or device if they request write access.
    					If this flag is not specified, but the file or device has been opened for write access or has a file mapping with write access, 
    					the function fails.
    (D):dwCreationDisposition:
    	An action to take on a file or device that exists or does not exist.
    	For devices other than files, this parameter is usually set to OPEN_EXISTING.
    
    	CREATE_NEW:		Creates a new file, only if it does not already exist.
    					If the specified file exists, the function fails and the last-error code is set to ERROR_FILE_EXISTS (80).
    					If the specified file does not exist and is a valid path to a writable location, a new file is created.
    	CREATE_ALWAYS:	Creates a new file, always.
    					If the specified file exists and is writable, the function overwrites the file, the function succeeds, 
    					and last-error code is set to ERROR_ALREADY_EXISTS (183).
    					If the specified file does not exist and is a valid path, a new file is created, the function succeeds, 
    					and the last-error code is set to zero.
    	OPEN_EXISTING:	Opens a file or device, only if it exists.
    					If the specified file or device does not exist, the function fails and the last-error code is set to ERROR_FILE_NOT_FOUND (2).
    	OPEN_ALWAYS:	Opens a file, always.
    					If the specified file exists, the function succeeds and the last-error code is set to ERROR_ALREADY_EXISTS (183).
    					If the specified file does not exist and is a valid path to a writable location, the function creates a file and the last-error code is set to zero.
    	TRUNCATE_EXISTING:Opens a file and truncates it so that its size is zero bytes, only if it exists.
    					If the specified file does not exist, the function fails and the last-error code is set to ERROR_FILE_NOT_FOUND (2).
    					The calling process must open the file with the GENERIC_WRITE bit set as part of the dwDesiredAccess parameter
    (E):dwFlagsAndAttributes:
    	The file or device attributes and flags, FILE_ATTRIBUTE_NORMAL (0x80) being the most common default value for files.
    	FILE_FLAG_OVERLAPPED:The file or device is being opened or created for asynchronous I/O.
    	FILE_ATTRIBUTE_NORMAL:The file does not have other attributes set. This attribute is valid only if used alone.
    	此标志位的其他功能十分丰富,暂不详解
    (F):hTemplateFile:
    	A valid handle to a template file with the GENERIC_READ access right. The template file supplies file attributes and extended attributes for the file that is being created.
    	This parameter can be NULL.
    	When opening an existing file, CreateFile ignores this parameter.
    	When opening a new encrypted file, the file inherits the discretionary access control list from its parent directory. For additional information, see File Encryption.
    (G):函数调用失败返回值为 INVALID_HANDLE_VALUE
    
    //2.
    (A):BOOL WINAPI GetFileSizeEx(__in HANDLE hFile, __out PLARGE_INTEGER lpFileSize); 用于获取文件大小
    	hFile:
    		A handle to the file. The handle must have been created with the FILE_READ_ATTRIBUTES access right or equivalent, 
    		or the caller must have sufficient permission on the directory that contains the file.
    	lpFileSize:
    	A pointer to a LARGE_INTEGER structure that receives the file size, in bytes.
    (B):BOOL WINAPI SetFilePointerEx(__in HANDLE hFile, __in LARGE_INTEGER liDistanceToMove, __out_opt PLARGE_INTEGER lpNewFilePointer, __in DWORD dwMoveMethod); 设置文件指针位置
    	hFile:
    		A handle to the file. The file handle must have been created with the GENERIC_READ or GENERIC_WRITE access right. 
    	liDistanceToMove:
    		The number of bytes to move the file pointer. A positive value moves the pointer forward in the file and a negative value moves the file pointer backward.
    	lpNewFilePointer:
    		A pointer to a variable to receive the new file pointer. If this parameter is NULL, the new file pointer is not returned.
    	dwMoveMethod:
    		The starting point for the file pointer move. This parameter can be one of the following values.
    		FILE_BEGIN:
    			The starting point is zero or the beginning of the file. If this flag is specified, then the liDistanceToMove parameter is interpreted as an unsigned value.
    		FILE_CURRENT:
    			The start point is the current value of the file pointer.
    		FILE_END:
    			The starting point is the current end-of-file position.
    	Note that it is not an error to set the file pointer to a position beyond the end of the file. The size of the file does not increase until you call the SetEndOfFile, 
    	WriteFile, or WriteFileEx function. A write operation increases the size of the file to the file pointer position plus the size of the buffer written,
    	leaving the intervening bytes uninitialized.You can use SetFilePointerEx to determine the length of a file. To do this, use FILE_END for dwMoveMethod and seek to location zero. 
    	The file offset returned is the length of the file. However, this practice can have unintended side effects, 
    	such as failure to save the current file pointer so that the program can return to that location. It is simpler and safer to use the GetFileSizeEx function instead.
    	You can also use SetFilePointerEx to query the current file pointer position. To do this, specify a move method of FILE_CURRENT and a distance of zero.
    (C):BOOL WINAPI SetEndOfFile(__in HANDLE hFile);根据当前文件指针位置来设置文件结束
    (D):执行同步文件IO:以下两个函数的最后一个参数传NULL, CreateFile 中不要指定 FILE_FLAG_OVERLAPPED 标志
    	BOOL WINAPI ReadFile(_In_ HANDLE hFile, _Out_ LPVOID lpBuffer, _In_ DWORD nNumberOfBytesToRead, _Out_opt_ LPDWORD lpNumberOfBytesRead, _Inout_opt_ LPOVERLAPPED lpOverlapped);
    	BOOL WINAPI WriteFile(_In_ HANDLE hFile,_In_ LPCVOID lpBuffer, _In_ DWORD nNumberOfBytesToWrite, _Inout_opt_ LPOVERLAPPED lpOverlapped);
    (E):FlushFileBuffers:刷新缓存中的数据入设备
    
    //3.异步I/O基础
    (A):为了以异步方式来访问设备,必须先调用 CreateFile 并传入 FILE_FLAG_OVERLAPPED
    (B):为了将I/O请求加入设备驱动队列,必须使用 ReadFile 和 WriteFile
    (C):在执行异步设备I/O的时候,必须在 ReadFile 和 WriteFile 的最后一个参数传入一个已初始化的 OVERLAPPED 结构
    typedef struct _OVERLAPPED {
    	ULONG_PTR Internal;			//用于保存错误码
    	ULONG_PTR InternalHigh;		//异步I/O完成时已传输字节数
    	union {		
    		struct {
    			DWORD Offset;		//偏移量的低32位
    			DWORD OffsetHigh;	//偏移量的高32位	
    		} DUMMYSTRUCTNAME;		//64位偏移量代表当前操作的文件位置(系统会忽略文件的文件指针),非文件设备此值必须为0,否则I/O请求会失败
    		PVOID Pointer;			
    	} DUMMYUNIONNAME;
    
    	HANDLE  hEvent;				//事件内核对象句柄
    } OVERLAPPED, *LPOVERLAPPED;
    (D):设备驱动程序不会以先入先出的方式来处理队列中的I/O请求
    (E):如果I/O请求是以同步方式执行的,则 ReadFile WriteFile 会返回非零值,如果I/O请求是以异步方式执行的或者调用时候发生了错误,则这两个函数会返回 FALSE 。 GetLastError 返回 ERROR_IO_PENDING 则表示
    	异步I/O请求已成功加入了队列
    (F):在异步I/O请求完成之前,一定不能销毁发出I/O请求时所使用的数据缓存和 OVERLAPPED 结构,否则内存会遭受破坏。(特别是当 OVERLAPPED 结构被定义为局部变量时候,特别容易会发生这种错误)
    
    //4.取消队列中的设备I/O请求
    (A):BOOL WINAPI CancelIo(__in HANDLE hFile);
    	hFile:A handle to the file.The function cancels all pending I/O operations for this file handle.
    	
    	Cancels all pending input and output (I/O) operations that are issued by the calling thread for the specified file. 
    	The function does not cancel I/O operations that other threads issue for a file handle.
    (B):可以关闭设备句柄来取消已添加的异步I/O请求,而不管他们是哪个线程添加的
    (C):当线程终止时,系统会自动取消该线程发出的所有I/O请求,如果请求被发往的设备具有与之关联的I/O完成端口,则其不会被取消
    (D):BOOL WINAPI CancelIoEx(__in HANDLE hFile, __in_opt LPOVERLAPPED lpOverlapped); 
    	可以取消任意线程的给定文件句柄的一个指定I/O请求
    	若第二个参数为NULL,则会取消该设备的所有I/O请求
    (E):被取消的I/O请求会返回错误码 ERROR_OPERATION_ABORTED
    
    //5.接收I/O请求完成通知
    (1):触发设备内核对象
    (A):ReadFile WriteFile 会在I/O请求添加到队列之前,先将设备内核对象置为未触发状态,当设备驱动完成了请求之后,驱动程序会将设备内核对象设为触发状态
    (B):当向一个设备同时发出多个I/O请求的时候,这种方法没什么用,因为一旦一个操作完成,该内核对象就会被触发
    
    (2):触发事件内核对象
    	OVERLAPPED 最后一个成员 hEvent 用来标识一个事件内核对象(由我们自己创建),当一个异步I/O请求完成的时候,设备驱动会检查此成员是否为NULL,
    	若不为NULL,则设备驱动会调用 SetEvent 来触发事件。如果想同时执行多个异步设备I/O请求,我们必须为每个I/O请求创建不同的事件对象
    
    (3):可提醒I/O
    (A):当系统创建一个线程的时候,会同时创建一个与线程相关联的队列,称为异步调用(APC)队列。为了将I/O请求添加到线程的APC队列中,我们应该使用 ReadFileEx 和 WriteFileEx,
    	当我们使用上述两个函数发出I/O请求的时候,这两个函数会将其回调函数地址传给设备驱动程序。当设备驱动程序完成I/O请求的时候,会在发出I/O请求的线程APC队列中添加一项,
    	该项包含了完成函数的地址、发出I/O请求时所使用的 OVERLAPPED 结构
    (B):当一个可提醒I/O完成时,设备驱动程序不会试图去触发一个事件内核对象。事实上,设备根本不会用到 OVERLAPPED 结构中的 hEvent 成员。
    (C):当线程处于可提醒状态时,系统会检查他的APC队列,对队列中的每一项,系统会调用其完成函数,并传入I/O错误码、已传输字节数以及 OVERLAPPED 结构地址
    (D):APC队列是系统维护的,系统会以任意的顺序来执行我们添加到队列中的I/O请求
    (E):SleepEx 
    	WaitForSingleObjectEx 
    	WaitForMultipleObjectsEx
    	SignalObjectAndWait
    	GetQueuedCompletionStatusEx
    	MsgWaitForMultipleObjectsEx
    	以上六个函数可以将线程置为可提醒状态
    (F):
    	QueueUserAPC:允许我们手动的将一项添加到APC队列中,可以实现高效的线程间通信
    
    	void WINAPI FunApc(ULONG_PTR pLong)
    	{
    		Sleep(100);
    		printf("%d
    ", pLong);
    	}
    
    	unsigned __stdcall FunThread(void* pVoid)
    	{
    		auto nRe = SleepEx(100, TRUE);
    		return 0;
    	}
    
    	int _tmain(int argc, _TCHAR* argv[])
    	{
    		HANDLE hThread = reinterpret_cast<HANDLE>(_beginthreadex(nullptr, 0, FunThread, nullptr, 0, nullptr));
    
    		Sleep(10);	
    		/*
    			If an application queues an APC before the thread begins running, the thread begins by calling the APC function. After the thread calls an APC function, 
    			it calls the APC functions for all APCs in its APC queue
    		*/
    
    		long aValue[10] = {};
    		for (int i = 0; i < 10; ++i)
    		{
    			aValue[i] = i;
    			QueueUserAPC(FunApc, hThread, aValue[i]);
    		}
    
    		system("pause
    ");
    		return 0;
    	}
    	/*
    		结果:顺序输出0-9
    		After the thread is in an alertable state, the thread handles all pending APCs in first in, first out (FIFO) order, 
    		and the wait operation returns WAIT_IO_COMPLETION. A thread enters an alertable state by using 
    	*/
    (G):
    	The APC support provided in the operating system allows an application to queue an APC object to a thread. To ensure successful execution of functions used by the APC,
    	APCs should be queued only to threads in the caller's process.
    	Note  Queuing APCs to threads outside the caller's process is not recommended for a number of reasons. 
    	DLL rebasing can cause the addresses of functions used by the APC to be incorrect when the functions are executed outside the caller's process. Similarly, 
    	if a 64-bit process queues an APC to a 32-bit process or vice versa, addresses will be incorrect and the application will crash. Other factors can prevent successful function execution, 
    	even if the address is known.
    
    	Each thread has its own APC queue. The queuing of an APC is a request for the thread to call the APC function. 
    	The operating system issues a software interrupt to direct the thread to call the APC function.
    	When a user-mode APC is queued, the thread is not directed to call the APC function unless it is in an alertable state. 
    	After the thread is in an alertable state, the thread handles all pending APCs in first in, first out (FIFO) order, 
    	and the wait operation returns WAIT_IO_COMPLETION. A thread enters an alertable state by using 
    	SleepEx, SignalObjectAndWait, WaitForSingleObjectEx, WaitForMultipleObjectsEx, or MsgWaitForMultipleObjectsEx to perform an alertable wait operation.
    	If an application queues an APC before the thread begins running, the thread begins by calling the APC function. After the thread calls an APC function, 
    	it calls the APC functions for all APCs in its APC queue.
    	It is possible to sleep or wait for an object within the APC. If you perform an alertable wait inside an APC, it will recursively dispatch the APCs. 
    	This can cause a stack overflow.
    	When the thread is terminated using the ExitThread or TerminateThread function, the APCs in its APC queue are lost. The APC functions are not called.
    	When the thread is in the process of being terminated, calling QueueUserAPC to add to the thread's APC queue will fail with (31) ERROR_GEN_FAILURE.
    	Note that the ReadFileEx, SetWaitableTimer, and WriteFileEx functions are implemented using an APC as the completion notification callback mechanism.
    
    	https://msdn.microsoft.com/zh-cn/library/windows/desktop/ms684954.aspx
    (H):可提醒I/O的劣势:
    	(1):可提醒I/O要求我们必须创建回调函数,这使得代码变的复杂
    	(2):可提醒I/O最大的问题是:发出I/0请求的线程必须同时完成对通知的处理,这将导致线程负载不均衡,导致程序伸缩性不太好
    
    (4)I/O完成端口
    (A):在并发服务器下,会存在大量处于可运行状态的线程,所以系统在切换这些线程时会浪费很多时间,为了解决这个问题,就是使用I/O完成端口内核对象
    (B):HANDLE WINAPI CreateIoCompletionPort(__in HANDLE FileHandle, __in_opt HANDLE ExistingCompletionPort, __in ULONG_PTR CompletionKey, __in DWORD NumberOfConcurrentThreads);
    	https://msdn.microsoft.com/en-us/library/windows/desktop/aa363862(v=vs.85).aspx	此网站很详细,这里不赘述
    (C):当我们创建一个I/O完成端口的时候,系统内核实际上会创建5个不同的数据结构:
    	(1):设备列表:
    		(A):表示与该端口相关联的设备,可以使用 CreateIoCompletionPort 来将设备关联到完成端口,每个设备对应一个传入的完成键,我们可以通过此完成键来区分不同的设备
    		(B):当设备句柄被关闭的时候,其对应的项会被删除
    	(2):I/O完成队列(先入先出):
    		(A):当一个与I/O完成端口关联的设备的一个异步I/O请求完成时,系统会将已完成的I/O请求添加到I/O完成端口的I/O完成队列的末尾,队列中的每一项包含的信息有:
    			已传输的字节数、完成键的值、 OVERLAPPED 结构指针、错误码
    		(B):PostQueuedCompletionStatus 可以模拟已完成的I/O请求,在I/O完成队列中添加一项,此函数可以与线程池中所有的线程进行方便的通信
    		(C):当完成端口从等待队列中删除一项时,I/O完成队列会删除被处理掉的项
    		(D):为了发出一个在完成时不被添加到队列中的I/O请求,我们必须在 OVERLAPPED 结构的 hEvent 成员中保存一个有效的事件句柄,并将它与1按位或起来
    	(3):等待线程队列(后入先出):
    		(A):一般等待线程所构成的线程池的个数是主机CPU个数的2倍
    		(B):当线程调用 GetQueuedCompletionStatus 时,线程会进入睡眠,调用线程的线程标识符会添加到等待线程队列
    		(C):I/O完成队列不为空,而且正在运行的线程数小于最大并发数, GetQueuedCompletionStatus 会先从I/O完成队列删除对应的项,
    			接着将等待线程队列中的项(GetQueuedCompletionStatus 的主调线程)转移到已释放线程列表,然后函数返回
    		(D):GetQueuedCompletionStatusEx 可以一次处理多个I/O请求
    	(4):已释放线程列表:
    		(A):完成端口在等待线程队列唤醒一个线程或已暂停的线程被唤醒,此时会将对应项添加到已释放线程列表中
    		(B):线程再次调用 GetQueuedCompletionStatus 或 调用一个函数将自己挂起时,此时对应项会从已释放列表中删除
    	(5):已暂停线程列表
    		(A):已释放的线程调用一个函数将自己挂起,会添加对应项入此列表
    		(B):已挂起的线程被唤醒,则对应线程会添从此列表删除并添加入已释放线程列表中
    (D):完成端口的目标是根据在创建完成端口时指定并发线程数量,将尽可能多的线程保持在已释放列表中。如果一个已释放线程进入等待状态,那么已释放线程列表就会缩减,完成端口就可以释放另一个正在等待的线程,
    	如果一个已暂停线程被唤醒,那么他就会离开已暂停线程列表并重新进入已释放线程列表,意味着此时已释放线程列表中的线程数量将大于最大允许的并发线程数量,这也是为什么线程池中线程个数需要比最大并发线程数多的原因
    

      

  • 相关阅读:
    zookeeper基础
    4. Zookeeper单机版安装
    3. 服务中间件Dubbo
    2. Maven工程的搭建
    Maven的优点
    应用架构的演进历史 MVC、 RPC、SOA 和 微服务架构
    js 正则表达式验证
    后台正则表达式验证部分部分总结
    Linux下记录登录用户历史操作
    Django使用Ace实现在线编辑器
  • 原文地址:https://www.cnblogs.com/szn409/p/8476171.html
Copyright © 2020-2023  润新知