Recently, I’m conducting experimental simulations for my thesis, using NS-2.
It’s about mobile ad hoc networks (MANET).

In some scenario, the number of nodes in a network grows, the demand of memory also increases tremendously. Since NS-2 is a memory hungry network simulator, it’s a good idea to add more RAM for better performance.

Now, I’m putting four sticks of DDR2 1G memory module on the motherboard of my server, all memory slots are full!!

What is PAE?

Large memory configuration machines require access to more than the 4 gigabyte limit. Due to this limitation, Intel introduced IA-32 processors with 36-bit physical address space access, which allows memory configurations of up to 64 GB.

FreeBSD provides support for this capability via the Physical Address Extension (PAE) kernel configuration option, available in all current release versions of FreeBSD.

Due to the limitations of the Intel memory architecture, no distinction is made for memory above or below 4 gigabytes. Memory allocated above 4 GB is simply added to the pool of available memory.

PAE HOWTO

To enable PAE support in the kernel, simply add the following line to the kernel configuration file:

options PAE


For simplicity and flexibility, you can also “include” the original kernel configuration file in a new PAE-ed one. There is a good example among stock configuration files in the source tree, you can find it in /usr/src/sys/i386/conf/PAE.

Take my server, VENUS, as an example, configuration file can be included hierarchically. The configuration file,VENUS (the same name as machines’), is included by VENUS-SMP for SMP. VENUS-SMP is included by VENUS-PAE for PAE configuration.

VENUS-PAE
    |
    +--VENUS-SMP
            |
            +--VENUS

PAE Limitations

PAE support in FreeBSD has a few limitations:

• A process is not able to access more than 4 gigabytes of VM space.
• KLD modules cannot be loaded into a PAE enabled kernel, due to the differences in the build framework of a module and the kernel.
• More details are shown in [1]

Non-PAE Migration Guide

For people who migrated from non-PAE configuration files, simply adding PAE may cause devices malfunctioned. Most of them are network related devices, including ppp, ipfw, netgraph, etc.
Solution are given in next section.

Loadable Kernel Modules are NOT supported

Since KLD modules cannot be loaded into a PAE enabled kernel, these options must be pre-compiled into kernel, e.g. IPFIREWALL series for ipfw, NETGRAPH series for ng devices and COMPAT_LINUX for Linux compatability.

Details are shown below:

options       IPFIREWALL
options       IPFIREWALL_VERBOSE
options       IPFIREWALL_DEFAULT_TO_ACCEPT

options       NETGRAPH
options       NETGRAPH_ETHER
options       NETGRAPH_PPPOE
options       NETGRAPH_SOCKET
options       COMPAT_LINUX

Reference

1. FreeBSD Handbook: 8.4.1 Large Memory Configurations (PAE)
2. FreeBSD Manual: pae(4)