• MMF I/O


    What Do Memory-Mapped Files Have to Offer?

    One advantage to using MMF I/O is that the system performs all data transfers for it in 4K pages of data. Internally all pages of memory are managed by the virtual-memory manager (VMM). It decides when a page should be paged to disk, which pages are to be freed for use by other applications, and how many pages each application can have out of the entire allotment of physical memory. Since the VMM performs all disk I/O in the same manner—reading or writing memory one page at a time—it has been optimized to make it as fast as possible. Limiting the disk read and write instructions to sequences of 4K pages means that several smaller reads or writes are effectively cached into one larger operation, reducing the number of times the hard disk read/write head moves. Reading and writing pages of memory at a time is sometimes referred to as paging and is common to virtual-memory management operating systems.

    使用MMF I/O的一个优势是,系统在一个4K的page中执行所有的数据传输。在内部,memory的所有page都是由VMM来管理的。 VMM决定什么时候一个page应该写到disk中,哪个page要被释放以供其他应用使用,每个应用在整个屋里内存的配额中可以有多少个page。既然VMM使用同样的方式执行Disk I/O- 一次读写一个page -这已经是被优化过的了。限制对disk的读写为4K 大小page的序列意味着小的读写操作被cach在一个大操作中,减少了磁盘读写的次数。每次读写一个page又被称为paging,并且它对VMM的OS是非常重要的。

    Another advantage to using MMF I/O is that all of the actual I/O interaction now occurs in RAM in the form of standard memory addressing. Meanwhile, disk paging occurs periodically in the background, transparent to the application. While no gain in performance is observed when using MMFs for simply reading a file into RAM, other disk transactions can benefit immensely. Say, for example, an application implements a flat-file database file structure, where the database consists of hundreds of sequential records. Accessing a record within the file is simply a matter of determining the record's location (a byte offset within the file) and reading the data from the file. Then, for every update, the record must be written to the file in order to save the change. For larger records, it may be advantageous to read only part of the record into memory at a time as needed. Unfortunately, though, each time a new part of the record is needed, another file read is required. The MMF approach works a little differently. When the record is first accessed, the entire 4K page(s) of memory containing the record is read into memory. All subsequent accesses to that record deal directly with the page(s) of memory in RAM. No disk I/O is required or enforced until the file is later closed or flushed.

    使用MMF I/O的另外一个好处是所有真实的IO操作都是以标准内存寻址的方式发生在RAM中的。同时,disk paging在后台周期性的发生,对应用时透明的。使用MMF简单地将文件的内存读到RAM中是没有什么性能优势的,但其他的disk操作事务可以受益良多。例如一个程序实现了一个扁平文件的数据库文件结构,在文件中包含了上百条的顺序记录。访问文件中的一个记录就是确定记录在文件中的偏移、从文件中读取数据。之后,有任何更新,记录必须被保存中文件中来保存这些变化。对于比较大的记录,只读取一部分数据是有益处的。但是不幸的是,每次记录的新部分被请求时,就会再次发生文件读写。MMF方法有些不一样。当记录第一次被访问时,包含这条记录的4K大的page被读进内存。所有后续对记录的访问就可以直接跟RAM中的page发生。没有任何磁盘读写,直到文件被关闭或者被flush。

    Note

    During normal system paging operations, memory-mapped files can be updated periodically. If the system needs a page of memory that is occupied by a page representing a memory-mapped file, it may free the page for use by another application. If the page was dirty at the time it was needed, the act of writing the data to disk will automatically update the file at that time. (A dirty page is a page of data that has been written to, but not saved to, disk; for more information on types of virtual-memory pages, see "The Virtual-Memory Manager in Windows NT" on the Developer Network CD.)

    The flat-file database application example is useful in pointing out another advantage of using memory-mapped files. MMFs provide a mechanism to map portions of a file into memory as needed. This means that applications now have a way of getting to a small segment of data in an extremely large file without having to read the entire file into memory first. Using the above example of a large flat-file database, consider a database file housing 1,000,000 records of 125 bytes each. The file size necessary to store this database would be 1,000,000 * 125 = 125,000,000 bytes. To read a file that large would require an extremely large amount of memory. With MMFs, the entire file can be opened (but at this point no memory is required for reading the file) and a view (portion) of the file can be mapped to a range of addresses. Then, as mentioned above, each page in the view is read into memory only when addresses within the page are accessed.

    How Are They Implemented?

    Since Windows NT is a page-based virtual-memory system, memory-mapped files represent little more than an extension of an existing, internal memory management component. Essentially all applications in Windows NT are represented in their entirety by one or more files on disk and a subset of those files resident in random access memory (RAM) at any given time. For example, each application has an executable file that represents pages of executable code and resources for the application. These pages are swapped into and out of RAM, as they are needed, by the operating system. When a page of memory is no longer needed, the operating system relinquishes control over the page on behalf of the application that owns it and frees it for use by another. When that page becomes needed again, it is re-read from the executable file on disk. This is called backing the memory with a file, in this case, the executable file. Similarly, when a process starts, pages of memory are used to store static and dynamic data for that application. Once committed, these pages are backed by the system pagefile, similar to the way the executable file is used to back the pages of code. Figure 2 is a graphical representation of how pages of code and data are backed on the hard disk.

    既然Windows NT是一个基于page的VM系统,MMF代表的不仅仅是现存的内部内存关系组件的一个扩展。本质上来说,在Windows NT中所有的应用其总体都可以表示为一个磁盘上的文件,同时其一部分驻留在RAM中。这些page被OS交换进或者出RAM。当一个page不再被需要时,OS替拥有它的应用交出对page的控制权。当这个page又被需要时,它又被从disk中读出来。

    ms810613.maname02(en-us,MSDN.10).gif

    Figure 2. Memory used to represent pages of code in processes for Windows NT are backed directly by the application's executable module while memory used for pages of data are backed by the system pagefile.

    Treating both code and data in the same manner paves the way for propagating this functionality to a level where applications can use it, too—which is what Windows does through memory-mapped files.

    Shared Memory in Windows NT

    Both code and data are treated the same way in Windows NT—both are represented by pages of memory and both have their pages backed by a file on disk. The only real difference is the file by which they are backed—code by the executable image and data by the system pagefile. Because of this, memory-mapped files are also able to provide a mechanism for sharing data between processes. By extending the memory-mapped file capability to include portions of the system pagefile, applications are able to share data that is backed by the pagefile. Shown in Figure 3, each application simply maps a view of the same portion of the pagefile, making the same pages of memory available to each application.

    ms810613.maname03(en-us,MSDN.10).gif

    Figure 3. Processes share memory by mapping independent views of a common region in the system pagefile.

    Windows NT's tight security system prevents processes from directly sharing information among each other, but MMFs provide a mechanism that works with the security system. In order for one process to share data with another via MMFs, each process must have common access to the file. This is achieved by giving the MMF object a name that both processes use to open the file.

    Internally, a shared section of the pagefile translates into pages of memory that are addressable by more than one process. To do this, Windows NT uses an internal resource called a prototype page-table entry (PPTE). PPTEs enable more than one process to address the same physical page of memory. A PPTE is a system resource, so their availability and security is controlled by the system alone. This way processes can share data and still exist on a secure operating system. Figure 4 indicates how PPTEs are used in Windows NT's virtual addressing scheme.

    ms810613.maname04(en-us,MSDN.10).gif

    Figure 4. Prototype page-table entries are the mechanism that permits pages of memory to be shared among processes.

    One of the best ways to use an MMF for sharing data is to use it in a DLL (dynamic-link library). The PortTool application serves as a useful illustration. PortTool uses a DLL to provide its porting functionality and relies on the main application for the user interface. The reason for this is simple: Other applications can then also use the DLL functionality. That is, other editors that are programmable can import the porting functionality. Because it is entirely feasible for PortTool to be running while another editor that imports the PortTool DLL is also running, it is best to economize system resources as much as possible between the applications. PortTool does this by using an MMF for sharing the porting information with both processes. Otherwise, both applications would be required to load their own set of porting information while running at the same time, a waste of system resources. The PortTool code demonstrates sharing memory via an MMF in a DLL.

  • 相关阅读:
    django 的操作
    numpy 初步学习
    django orm 多对多操作
    selenium 安装chrome 以及chromedriver
    配置uwsgi + nginx
    ubuntu安装 anaconda
    django ---手动删除数据库
    前端(六)之盒模型显隐、定位与流式布局思想
    前端(五)之display 总结与浮动
    前端(四)之精灵图与盒模型布局
  • 原文地址:https://www.cnblogs.com/whyandinside/p/1776547.html
Copyright © 2020-2023  润新知