`
izuoyan
  • 浏览: 8879220 次
  • 性别: Icon_minigender_1
  • 来自: 上海
社区版块
存档分类
最新评论

Virtual Memory I: the problem

阅读更多

Virtual Memory I: the problem

This article serves mostly as background to help understand why the kernel developers are considering making fundamental virtual memory changes at this point in the development cycle. It can probably be skipped by readers who understand how high and low memory work on 32-bit systems.

A 32-bit processor can address a maximum of 4GB of memory. One could, in theory, extend the instruction set to allow for larger pointers, but, in practice, nobody does that; the effects on performance and compatibility would be too strong. So the limitation remains: no process on a 32-bit system can have an address space larger than 4GB, and the kernel cannot directly address more than 4GB.

In fact, the limitations are more severe than that. Linux kernels split the 4GB address space between user processes and the kernel; under the most common configuration, the first 3GB of the 32-bit range are given over to user space, and the kernel gets the final 1GB starting at0xc0000000. Sharing the address space gives a number of performance benefits; in particular, the hardware's address translation buffer can be shared between the kernel and user space.

If the kernel wishes to be able to access the system's physical memory directly, however, it must set up page tables which map that memory into the kernel's part of the address space. With the default 3GB/1GB mapping, the amount of physical memory which can be addressed in this way is somewhat less than 1GB - part of the kernel's space must be set aside for the kernel itself, for memory allocated with vmalloc(), and various other purposes. That is why, until a few years ago, Linux could not even fully handle 1GB of memory on 32-bit systems. In fact, back in 1999, Linus decreedthat 32-bit Linux would never, ever support more than 2GB of memory. "This is not negotiable."

Linus's views notwithstanding, the rest of the world continued on with the strange notion that 32-bit systems should be able to support massive amounts of memory. The processor vendors added paging modes which could use physical addresses which exceed 32 bits in length, thus ending the 4GB limit for physical memory. The internal addressing limitations in the Linux kernel remained, however. Happily for users of large systems, Linus can acknowledge an error and change his mind; he did eventually allow large memory support into the 2.3 kernel. That support came with its own costs and limitations, however.

On 32-bit systems, memory is now divided into "high" and "low" memory. Low memory continues to be mapped directly into the kernel's address space, and is thus always reachable via a kernel-space pointer. High memory, instead, has no direct kernel mapping. When the kernel needs to work with a page in high memory, it must explicitly set up a special page table to map it into the kernel's address space first. This operation can be expensive, and there are limits on the number of high-memory pages which can be mapped at any particular time.

For the most part, the kernel's own data structures must live in low memory. Memory which is not permanently mapped cannot appear in linked lists (because its virtual address is transient and variable), and the performance costs of mapping and unmapping kernel memory are too high. High memory is useful for process pages and some kernel tasks (I/O buffers, for example), but the core of the kernel stays in low memory.

Some 32-bit processors can now address 64GB of physical memory, but the Linux kernel is still not able to deal effectively with that much; the current limit is around 8GB to 16GB, depending on the load. The problem now is that larger systems simply run out of low memory. As the system gets larger, it requires more kernel data structures to manage, and eventually room for those structures can run out. On a very large system, the system memory map (an array ofstruct pagestructures which represents physical memory) alone can occupy half of the available low memory.

There are users out there wanting to scale 32-bit Linux systems up to 32GB or more of main memory, so the enterprise-oriented Linux distributors have been scrambling to make that possible. One approach is the4G/4G patchwritten by Ingo Molnar. This patch separates the kernel and user address spaces, allowing user processes to have 4GB of virtual memory while simultaneously expanding the kernel's low memory to 4GB. There is a cost, however: the translation buffer is no longer shared and must be flushed for every transition between kernel and user space. Estimates of the magnitude of the performance hit vary greatly, but numbers as high as 30% have been thrown around. This option makes some systems work, however, so Red Hat ships a 4G/4G kernel with its enterprise offerings.

The 4G/4G patch extends the capabilities of the Linux kernel, but it remains unpopular. It is widely seen as an ugly solution, and nobody likes the performance cost. So there are efforts afoot to extend the scalability of the Linux kernel via other means. Some of these efforts will likely go forward - in 2.6, even - but the kernel developers seem increasingly unwilling to distort the kernel's memory management systems to meet the needs of a small number of users who are trying to stretch 32-bit systems far beyond where they should go. There will come a time where they will all answer as Linus did back in 1999: go get a 64-bit system.

原文地址: Virtual Memory I: the problem

分享到:
评论

相关推荐

    C++ Programming: From Problem Analysis to Program Design, 8th Edition

    C++ Programming: From Problem Analysis to Program Design By 作者: D. S. Malik ISBN-10 书号: 1337102083 ISBN-13 书号: 9781337102087 Edition 版本: 8 出版日期: 2017-02-13 pages 页数: 1491 Contents ...

    Memory Coherence in Shared Virtual Memory Systems.pdf

    This paper studies the memory coherence problem in designing and implementing a shared virtual memory on loosely-coupled multiprocessors. Two classes of algorithms for solving the problem are ...

    Making C++ Objects Persistent: the Hidden Pointers

    C11 objects of types that have virtual functions or virtual base classes contain volatile (‘memory’) pointers. We call such pointers ‘hidden pointers’ because they were not specified by the user. ...

    C++.Programming.From.Problem.Analysis.to.Program.Design.7th.Edition

    C++ PROGRAMMING: FROM PROBLEM ANALYSIS TO PROGRAM DESIGN, Seventh Edition remains the definitive text for a first programming language course. D.S. Malik's time-tested, student-centered methodology ...

    微软内部资料-SQL性能优化2

    Application memory tuning provides more of the computer's virtual memory to applications by providing less virtual memory to the operating system. Although a system having less than 2 GB of physical ...

    Oracle WebLogic Server 10gR3: Troubleshooting Methodologies

    Working knowledge of the Java Virtual Machine Working knowledge with Linux Operating System Course Objectives List the uses of software patterns when troubleshooting Identify common system ...

    Principles of Computer Architeture

    INTRODUCTION 1 VERVIEW 1 1.1 O RIEF ISTORY 1.2 A B H 1 V N M 4 HE ON EUMANN ODEL ...B.3.1 The State Assignment Problem 550 B.3.2 Excitation Tables 554 SOLUTIONS TO PROBLEMS 569 INDEX 623

    FlexGraphics_V_1.79_D4-XE10.2_Downloadly.ir

    - FIX: The TFlexPanel.FindControlAtPoint method maked virtual to realize RealTime-capability when on mouse cursor moving the flex-object search not occurs. - FIX: After deleting the selected points ...

    Windows Azure Succinctly(Syncfusion,2014)

    By that I mean developers should focus on the code in the application that is most focused on solving the specific business problem, instead of trying to reinvent frameworks for solving problems that...

    Senfore_DragDrop_v4.1

    work around will fix the problem: In the project options of *all* projects which uses these components, add the following conditional define: NO_WIN32_LEAN_AND_MEAN The define *must* be made in ...

    Modern Operating System

    spaces (virtual memory). Chapter 4 is about the abstraction of the disk into files. Together, processes, virtual address spaces, and files are the key concepts that operating systems provide, so these...

    BobBuilder_app

    I deferred from testing the get test over 100 million record as it would require a huge array in memory to store the Guid keys for finding later, that is why there is a NT (not tested) in the table. ...

    uhdd.sys源码

    It uses only 656 to 752 bytes of upper memory depending on the system and it can also load in 640K DOS memory RDISK is a simple and small RAMdisk driver for use when resizing or other features are...

    Google C++ Style Guide(Google C++编程规范)高清PDF

    The main reason for making a virtual function inline is to place its definition in the class, either for convenience or to document its behavior, e.g., for accessors and mutators. The -inl.h Files...

    BCGControlBarPro.v12.00

    CBCGPGridCtrl::EnsureVisible is now virtual. Added navigation by TAB (Shift+TAB) key. Added "Ctrl+Z" (Undo) handler for in-place edit of the grid item. Changes in CBCGPGridCtrl::SetCurSel for SM_...

    Cisco Press - OSPF Network Design Solutions, 2nd Edition

    Determining Router Memory Requirements 197 Router CPU Requirements 199 Bandwidth Usage 199 OSPF Security 199 Area Design Considerations 200 Area Design Overview 200 Considering Physical Proximity 201 ...

    Bochs - The cross platform IA-32 (x86) emulator

    - Fixed Bochs crash when accessing the first byte above emulated memory size - Internal Debugger - Introduced range read/write physical watchpoints - Allow reloading of segment registers from ...

    EurekaLog_7.5.0.0_Enterprise

    12)..Added: Fatal memory problem now change thread name (to simplify debugging) 13)..Added: boPauseELThreads and boDoNotPauseELServiceThread options (currently not visible in UI) 14)..Added: Support ...

    simulated-virtual-memory

    塔布拉·德·莫莫里亚 Esterepositóriocontémocódigorelativo ao trabalhopráticodo curso de Sistemas ...O Problema 问题包括虚拟的实现器,虚拟的实现器,质量的虚化器,合格的生产者(páginas)。 签证,签

    HxD HeX Editor 英文绿色版

    - RAM-Editor: can read and write virtual memory of other processes - Data-folding for better overview in RAM-Editor - Splitting and joining files - Safe deletion of files (shredder) - File compare ...

Global site tag (gtag.js) - Google Analytics