Kernel 2.6 On Fedora-based Systems

Since this has come up regularly, I've decided to finally blog it.

- The "good 'ole days" of the kernel-source-ver.i386.rpm package

Up through all Red Hat Linux releases and Fedora Core 1, which subsequently means through Red Hat Enterprise Linux 3, Red Hat included a package entitled kernel-source-ver.i386.rpm. The purpose of this "binary" RPM was to create a /usr/src/linux-ver directory (and associated /usr/src/linux[-2.x] symbolic link). This was the standard kernel location for not only building custom kernels, but where any 3rd party kernel modules (with their own Makefile) could expect to find kernel headers, build-time configuration information, etc...

There were many issues with this approach.

First off, until kernel 2.6, if you wanted to build any 3rd party kernel module -- even one that included it's own Makefile and built systems -- you had to "prep" a kernel source tree that was exactly as your running kernel. This meant applying the correct configuration, the exact Makefile tags, etc... that were often daunting for newer users -- let alone even experienced users. It was more than just the common "build from source" routine of ./configure and make install. And the order was often important as well.

Secondly, the system headers, including C library and other kernel headers, were commonly used by applications as well as kernel building -- especially when different C compilers were being used for the system. Without going into a lot of specifics, it was not long before kernel headers and system headers became separate entities. A major reason being is that some headers and other build-time configuration information might need to be re-created in the kernel build process. In fact, early in Red Hat Linux 7, changes to kernel 2.2's mrproper ("Mr. Proper" is a non-US marketing name for "Mr. Clean") Makefile target resulted in the blowing away many of the system headers.

- Red Hat does not built kernels from kernel-source-ver.i386.rpm

RPM is a build system and designed as such for a reason. Red Hat Linux and Fedora-based distros (collectively referred to Fedora-based) as well as even other package systems and distros, such as Debian and Debian-based, have used a dedicated, "protected" build environment for kernels for a long time. Starting in Red Hat Linux 6 (well over 6 years ago), Red Hat kernels, modules and other components that have required the kernel source tree have been built from the kernel-ver.src.rpm, and not via a source tree located in /usr/src/linux-ver.

This addresses many issues, including:

1. Include all components to complete a build
2. Ensure exact steps to complete a build
3. Avoid negative affects to files in the production system (chroot)
4. Force documented changes with source, SPEC, etc...

As of Red Hat Linux 7, I personally and professionally stopped building kernels from kernel-source (now kernel-sourcecode as we'll talk about in a bit). The exact order and requirements in not only building but even "prepping" the kernel source tree for 3rd party kernel modules is extremely important. But beyond that, the "mrproper" target can do some particularly nasty things to your systems' /usr/include and other directories. In the kernel.src.rpm build under the chroot, the system's /usr/include and other directories are not touched, and they have their own, self-contained support.

- From kernel-source-ver.i386.rpm to kernel-sourcecode-ver.noarch.rpm to no more "binary" kernel source

Starting with Fedora Core 2 based on the new kernel 2.6, Red Hat decided to deprecate the "binary" kernel-source-ver.i386.rpm.

At first, they included a new architecture-independent "binary" package name called kernel-sourcecode-ver.noarch.rpm to avoid confusion with the old approach. Mid Fedora Core 2, they considered it redundant to continue doing so and, subsequently, it is not included with Red Hat Enterprise Linux 4 onward either. There is still a "binary" RPM target defined as kernel-sourcecode in the kernel-ver.src.rpm (Source RPM), but Red Hat no loner builds it by default. Redundancy, safety, etc... are major reasons.

To build a "prep'd" kernel 2.6 source tree in FC2+/RHEL4+, you should follow these steps:

1. Download the kernel-ver.src.rpm source RPM (SRPM, .src.rpm) package for your kernel version. If you don't know your kernel version, run `uname -r`. Alternatively, and preferrably, You should use the automated methods of YUM, UP2DATE, etc... to get the correct Source RPM (.src.rpm) of the currently running version of the kernel without any manual checking.

2. Install this package, e.g., rpm -ihv kernel-`uname -r`.src.rpm

3. Prep the kernel source for your architecture, e.g., rpmbuild -bp --target=i686 /usr/src/redhat/kernel-2.6.spec

That will safely prep the kernel in the RPM chroot environment into a mode that is "ready to use" under /usr/src/redhat/BUILD/kernel-2.6*/linux-2.6*/, although you should consult the kernel-2.6.spec file to see what Makefile targets have and have not been run. You may wish to create a symbolic linke to /usr/src/linux-`uname -r` but be careful when executing any Makefile targets in the unprotected, root filesystem of the system.

- Non-Red Hat/Fedora Distribution Source Code Rebuilds

Even though Red Hat does not build and distribute the kernel-sourcecode-ver.noarch.rpm, any distribution that rebuilds from Fedora Development, Core, Legacy or Red Hat Enterprise Linux kernel-ver.src.rpm may or may not choose to do the same. Some 3rd party rebuilds have decided to include it. Others agree with Red Hat, that it should be deprecated and the chroot RPM system build process be used.

As I eluded to earlier, you can still create a kernel-sourcecode-ver.noarch.rpm package that creates the /usr/src/linux-ver directory for any FC2+/RHEL4+ kernel 2.6 release. The procedure is as follows:

1. and 2., as the previous section.

3. Modify the SPEC file, e.g.,
# vi /usr/src/redhat/SPECS/kernel-2.6.spec
as follows ...

%ifarch noarch
%define builddoc 0
%define buildsource 1
%define buildup 0
...

4. Build the noarch target, which builds kernel-sourcecode

# rpmbuild -bb --target=noarch /usr/src/redhat/SPECS/kernel-2.6.spec

5. Install the "binary" kernel-sourcecode RPM

# rpm -ihv /usr/src/redhat/RPMS/noarch/kernel-sourcecode-`uname -r`.noarch.rpm

Now you have your /usr/src/linux-2.6 directory. I highly recommend against building your kernels in the "unprotected" and "root anchored" environment from an arbitrary set of commands (instead of the chroot of RPM with a pre-tested sequence of commands), but that is your choice.

Yes, before kernel 2.6, the location of /usr/src/linux-ver was expected so you could build 3rd party kernel modules. But it was typically safe to to do so, since you're not running the kernel's Makefile. You still typically wanted to "prep" the kernel tree itself in the RPM chroot environment, even if you were only going to symlink it to /usr/src/linux-`uname -r` after doing so. Kernel 2.6 addresses this, which brings me to my next point.

- Kernel 2.6 build subdirectory and the kernel-devel-ver.*.rpm package

In the kernel modules directory (e.g., /lib/modules/`uname -r`/), there is a subdirectory called build. It points to the headers and other configuration, dependencies, etc... of the running kernel so 3rd party drivers can be built. Before kernel 2.6, this was a symlink to /usr/src/linux-`uname -r`. As of kernel 2.6, this is now static in the kernel itself (taking up a few MBs). The idea here is that you should be able to build 3rd party kernel modules against the current kernel without having to prep a full kernel tree.

Since most newer, 3rd party module Makefiles were already looking to /lib/modules/`uname -r`/build/ to locate the kernel headers, build-time configuration information, etc... As such, the change in kernel 2.6 should be transparent for most newer 3rd party kernel modules with their own Makefile, with exception of the fact that you no longer have to . At the same time, and in the worst case for kernel 2.6, it's often merely just a matter of:
ln -s /lib/modules/`uname -r`/build /usr/src/linux-`uname -r`

You almost never need to "prep" a kernel source tree to build 3rd party kernel modules against it.

Now as of Fedora Core 4 (and accidentally one Fedora Core 3 kernel release), as well as Red Hat Enterprise Linux 4, Red Hat decided to separate kernel 2.6's /lib/modules/`uname -r`/build/ subdirectory from the rest of the kernel modules in /lib/modules/`uname -r`. This new package is known as kernel-devel-ver.*.rpm (where * is the binary architecture, i386, i586, i686, athlon, x86_64, etc...).

So as of Fedora Core 4+ and Red Hat Enterprise Linux 4+, if you want to build 3rd party kernel modules, you need to to install the kernel-devel-ver.*.rpm package so the /lib/modules/`uname -r`/build/ subdirectory is present. A simple yum install kernel-devel or up2date kernel-devel takes care of that for an Internet connected system. By not installing it, you save several MBs of disk space (which may be useful in embedded/read-only systems, etc...), which I'm sure was Red Hat's rationale (since just the binary kernel modules can total less than 10MB in some cases).

6 Things to Know About Linux

I am subscribed to enough user groups that everyday Linux (and other Free Software Foundation's, FSF's, GNU's Not UNIX systems, among other UNIX systems) is oversold to people, people foolishly try to explain Linux or, far worse yet, start using some already misguided discussion of Linux to market one Linux distribution against another. Most completely miss the concept of community developed software, which is where much of the differences lie, and not in some sort of ideology of "free" or "less evil" or "anything but brand X." In fact, many of these arguments are as far away from community developed software as you can get.

With that said, there are six (6) things about Linux to know, and I don't care if you're Richard Stallman, you should know these (and people like Eric S. Raymond, Bruce Parens and Linux Torvalds clearly do). I mean, some people have already beaten the "Linux is a UNIX-like OS, not a Windows-like OS" like a dead horse. Let's go even beyond that ...

1. What the Pricetag on Windows Buys You

When you buy Windows XP, you are not buying a $50-200 product. And when you buy MS Office, you are not buying a $200-600 product. You are buying licenses, and it costs Microsoft virtually next to nothing to reproduce them in comparison. Therefore, you are funding a multi-billion dollar software bundling and marketing service. Now before you think I'm demonizing Microsoft, hear me out.

Those billions of dollars go to sign licensing agreements to ensure Windows is bundled with multimedia codecs, license agreements to intellectual property (IP) rights to patents such as on 3D gaming, and countless other, 3rd party solutions that would not and could not be used without their permission. You pay for the right to get to use those in your everyday computing -- from when you receive the computer system from the Tier-1 PC Original Equipment Manufacturer (OEM) who pre-installed your Windows version to a specific and exacting form. Or when you get other software that is explicitly designed for your system, or the hardware you buy at the computer superstore who are largely Microsoft partners -- all guaranteed to work together!

Now, of course, this money goes to market those partner products, licenses and solutions. Which means you are not only gaining access to them, but you are using a system that actively promotes them. A system that holds a 90+% share of the computing market, a market of exclusion and inclusion. Products designed to work today and for a good 3 years, as not only do most consumers upgrade their computers every 3 years, but most businesses, business accounting/depreciation and business management terms have 3 year cycles. But more on that in a minute.

2. What Is Community Developed Software All About?

Community developed software, poorly called Free Software (aka Freeware), more appropriately termed Freedom Software -- or Freedomware for short -- is about a community bonding together to share information. That means people pushing aside most intellectual property (IP) rights, helping one another solve problems and otherwise coming up with good, solid and -- most importantly -- repeatable approaches and universal interfaces aka "open standards" to how systems and software should work. But the real key to ensuring that community developed software works on the premise of both individual choice (I choose to work for a public good, but I am not forced to) and non-exclusionary participation (I don't discriminate against others, even ones I do not like), without bias.

E.g., even the SCO Group, currently disliked for much of its more recent legal actions (outside of its initial contract dispute with IBM) and more recent anti-Linux marketing approach, has full rights to use and distribute various, mission critical Freedomware in its UNIX products.

Freedomware development and use thrives on open standards, as anything freely redistributable without written licensing agreements, negotiations and other inhibiting, non-technical aspects. At the same time, Freedomware survives on the protections of IP itself, as it could not exist in a market of IP if it did not hold its own. It is just that those who choose to join community developments choose to share their IP, instead of horde it, so all may benefit. The result is that innovation in software occur much faster -- and have resulted in nearly all solutions you use today, even on Windows (which we'll get to in a minute). And open source and open standard systems, software and formats do work with each other without fuss, as long as people work together.

The downside is that Freedomware does not gain access to proprietary IP. That means unless an open standard has taken hold in the public, Freedomware cannot likely support it. Although "clean room" compatible software and "reverse engineering" efforts (which are always the "last resort") attempt to make Freedomware compatible with proprietary systems, by their very nature, community software is a legal mindfield for proprietary IP (and sometimes vice-versa). The community cannot simply license proprietary IP to be freely shared like a private entity could.

3. Commercial Companies and Community Developed Software

Now some will quickly point to commercial companies now selling community developed software. For the most part, many of these organizations are small. So while you may pay $50-100 for their software, which seems "reasonable" to the cost of Microsoft's -- that funding only amounts to maybe $50,000/year, maybe $500,000/year for a common Linux distribution. Now what does that go towards? It barely pays enough for the Internet update servers, a few management individuals and other administrative details. The rest of the support is done by volunteers and other individuals outside the organization. And it most definitely does not leave room for such private organizations to license proprietary IP, which might have legal issues in merging with community developed software for the public to freely use.

Now there are other companies, ones selling a $150-3,000 OS, with $500-15,000 add-ons. Such companies are Red Hat. Red Hat, like some of its predecessors including Cygnus who Red Hat has since absorbed, built its entire commercial model on giving away all of its IP, and giving away all of its software, and then still turning around and charging money for what you could get for free. So what does Red Hat with its $50+M/year software funding through subscriptions? It spends that money procuring more software and making it legally redistributable, largely those software components that fund its repeat licensees -- businesses and server software. Such software components like filesystems, directory services and the like. Red Hat relies on more community developments, even those it helps fund indirectly with its massive number of in-house developers, in a non-supporting role. And under no circumstances does Red Hat license proprietary IP that it cannot give away at some point.

Which leaves only a few other companies. Hewlett-Packard has many focuses, and has more recently started to share some of its IP and work with Linux, one of the greatest of notables. And while Sun Microsystems has largely been one of the poster-corporation for open standards, many of which Linux benefits from today, it has only more recently discovered what sharing its IP and source can do. IBM has largely treated Linux as a product for expanding its proprietary solutions and services line, although IP has donated many patents (IBM being the world's foremost patent holder) that were troubling Linux developers in more recent months.

So these companies, among others, have been sharing to some extent, helping bring some proprietary IP to be freely used. But that still does not solve the "inclusion" of what 90% of users have in Windows. Unless we look at that "other UNIX-like" platform.

4. Why MacOS X is the "Other" UNIX-Like Platform That People Like

Apple MacOS X is the UNIX-like platform that people say is "better." By "better," let's put it back in terms of #1 above. Apple is a company with significant funding to sign licensing agreements, guarantee a good amount of hardware compatibility, work with larger companies and -- most importantly -- has the necessary cash to spend on marketing. Apple didn't invent MP3, they got recording studios on-board to make money distributing their works in MP3. Apple didn't invent the MP3 player, they were just the first company, and with enough money, to market a single line of products. And most of all, Apple has the most experience in understand its consumers, bridging the leading-tech to the closet-geek consumer.

That's why when you go to the Superstore, even those that don't cater to Apple, hardware often works with Apple, and some software is available for Apple. Apple has the money to deal with both the licensing and marketing aspects -- not as far as Microsoft, but enough so "it just (typically) works when you get home." No messy codecs to download, no video drivers to obtain, no legally questionable software that is not included, etc... And they have the money to support their efforts.

5. UNIX Invented It, Microsoft Bought the 3rd Best (Leftover) and Marketed It

Everything was invented on UNIX or UNIX-like platforms. The graphical user interface (GUI), the Internet, the office suite, the web browser, the media player (let alone "eye candy" / "skinned" media player), etc..., etc..., etc... Most people assume otherwise because they are part of the 90+%. Even newer Linux users continue to assume, because they did not grow up with UNIX/Linux. But the reality is that in the free sharing of ideals, open standards and open source have always been where things have been invented. Not only that, but if legally allowed, Microsoft uses open source, and when not, it buys the 3rd Best product (usually the minority "left over") and markets the hell out of it.

Who's to say what is better? If you read most any review on an office suite, the comparison is often made based on how MS Office works, let alone MS Office compatibility -- something that MS Office itself fails against older versions of its. It does not take into consideration components that MS Office does not have, and in many cases, did not implement long ago, let alone how well other office suites maintain compatibility with themselves and their old documents over time. When it comes to internet browsers, it's not about what features are supported (let alone cross-platform standards), but how well it renders Microsoft Internet Explorer (IE) on Windows-only sites (let alone how well some browsers "fool" sites into thinking they are Microsoft browsers, because some of their code degrades features -- even MS IE on MacOS X!).

People find it surprising when they use Corel products and find features that not only surpass MS Office in capabilities, not only read their old formats far better than MS Office reads its own, but reads even older MS Office documents better than newer versions of MS Office can. That's one of the major reasons organizations that need 3+ years of document longevity -- such as those in law and medicine -- still run Corel, although quite modern and powerful versions (and not 16-year old WP 5.1). Or those who adopted StarOffice in the early '90s that had features that MS Office still does not have, let alone still reads its old file formats as well. Corel and StarOffice today, along with IBM SuiteSuite and others, are moving to support or already support fully documented XML standards. Microsoft's incomplete 2003 XML disclosure which is not even used as the standard in MS Office 11 (2003), is not even being used for the new, supposive XML standard in MS Office 12 (2005), and full disclosure is still forthcoming.

Again, open standards and open source are open for all to see. The OpenOffice, now OpenDocument, standard has been fully published and open to extensions -- even from Microsoft -- for over a half-decade. Microsoft is legally free to use the exact code in its products, yet it will not. At the same time, open source is not allowed to use proprietary IP from Microsoft. And since proprietary hold with an ever-changing format -- using control of a distribution channel such as the Tier-1 PC OEM and computer superstore retail channel -- is the exclusionary/inclusionary factor, this is not a technical issue.

6. The Distribution Channel is Not a Technical Problem

Linux will continue to have compatibility issues with proprietary IP, no matter how much reverse engineering or even some outside donations occur. The core company, and its core partners, that control 90+% of all computer distribution -- the 4 components: system, peripherals, OS and applications -- have a very good profit model. To support Linux is to break that model.

The peripheral vendor who wants to sell you a cheap printer, then make a lot of money on supplies, then force you to upgrade the next time you change your system or OS. The system vendor who wants to sell you a new system when you want a new OS. The OS that you require when you get a new application that doesn't run on the old. It makes no sense for the peripheral vendor to expend money to write drivers for an older, unsupported OS when they won't get any more money for it. There is no sense for the OS vendor to maintain application and system support for long term as it gains them no further revenue.

Especially when 90+% of American consumers go into a computer superstore, or to a Tier-1 PC OEM, and expect to get a new computer with new peripherals with a new OS and new applications every 2-3 years, or businesses that have a 3 year accounting cycle, or managers who can justify budgets every 3 years (or their predecessor was so fired). And in each and every case, who cares if it doesn't work, old files aren't accessible and otherwise useless in 3 years?

So to prevent others from writing compatible drivers, compatible software, compatible OSes, is to not share that information. And without that information, Linux cannot have drivers, compatible software and compatible OS interfaces. Furthemore, to keep costs of hardware down, today's storage devices, printers, scanners and almost everything in a system is software-based anyway, with little hardware. Even if generic Linux software drivers are written (and in many cases they are to support just about any feature), the vendors make little changes in the software licensed from 3rd parties that Linux, in the best case, only supports 500+ variants of a product of literally thousands of OEMs that made changes.

Conclusion

Which is why if you are seriously considering any type of community software -- such as Linux -- you have to recognize that you won't always be able to buy the hardware on-sale at the superstore, or be compatible with the latest, proprietary IP application software or be able to view the latest, MS IE-only web sites with Windows-only media formats. Only vendors who have funding to both license and market have clout to work with proprietary IP, with one of the few, UNIX-like platforms being Apple's MacOS X -- but even it has some limitations in comparison to Windows. If you need that proprietary IP, you're probably far better paying the $50-150 for Windows, another $50-100 for system utilities, and locking down Windows tight, security-wise (although it will probably break a lot of your Windows applications) than run Linux.

At the same time, Linux and even community developed software (some of which is available for Windows) works "out of the box" with open standards, other open source software, well designed hardware (especially so and better than Windows in many cases) and, most importantly, drivers, interfaces, terminology and -- especially -- data formats that are virtually eternal. That's why UNIX-like systems have a heafty following, especially by established users, even more and more non-technical, application-focused these days. Linux also caters to enterprise management more fluidly with enterprise packaging and roll-outs -- which can be a catch-22 and curse for more "home consumers" who "just want it to install." Especially when it comes to well designed, multiuser, "I don't trust you to control the system at a 'superuser' at any time" defaults in UNIX-like systems.

Which is why Linux is slowly but surely moving from the Internet platform to the data center to the corporate desktop -- for control, security and managability, many of which are of not only not user focused, but an intentional hinderance at times (at least compared to "how Windows works"). Which is why the superstores still don't cater to Linux. Which is why no Linux companies have spent the significant resources to gain proprietary IP and roll out a partially proprietary desktop version of Linux (which will probably prove to be a legal impossibility with Linux for the most part -- unlike a BSD-based, legal Apple MacOS X, long story). But give corporate adoption time.

Because it might be the corporation that starts to force you to run StarOffice at home to be compatible with the office.

Final Thoughts
How far that can Freedomware on Windows turn into and make in-roads into the proprietary IP clique and distribution channel that opens up to Linux in general? No one knows yet. I'm doubtful it can break into a majority, because it's hard to fight licenses and marketing that costs real money to obtain. Only when everyone agrees to share will it.

But as a staunch man of Libertarian ideals, I'm not about to tell people that must use Linux. In fact, the most I'd ever regulate is open standards, and to only a limited extent. Forcing people to use something against their will, no matter how wrong, misguided and self-inflicting I think they are, is taking away their choice. And when a community moves from choosing to work together and accepting those who do not to the point of forcing everyone to work with them, that's when that other "commu" prefix word becomes reality. Choice is freedom, and why community developed software is very American, because it's all about choice -- including the right to choose a proprietary piece of software.

In fact, if it delivers values, which much proprietary software from other companies (than the one everyone likes to hate), there is nothing wrong with it. Companies like Corel have done this for decades. But when you mandate a private corporation and monopoly through legislative requirements, it's no longer choice, but the facist economic model. Which is why open standards can be very American as well, because it forces both community organizations and private corporations to win on their merits -- of which, many private corporations can and do better than community organizations, when it comes to open standards.

The Nation's #1 Running Back ...

Where he spent most of his time last Saturday ...


Take away his reversal at the end of the 2nd quarter, and his rushes against our 3rd team late in the 4th quarter, and you have a man who didn't even gain 40 yards on the overwhelming majority of his carries.

What is x86-64? "Long Mode" memory model ...

Since I'm getting tired of explaining this over and over, it's definitely time for a Blog post!

Looking beyond the hardware changes in the current AMD64 (NUMA/HyperTransport) architecture from the previous AMD (EV6) architecture, x86-64 comes down to a new "memory model" for programmers. Even if you're not a programmer, this affects you as a user.

- OSes and Memory Models

"Software compatibility" is all about memory models -- the way the CPU handles address registers, address translation (typically with page or page translation). Your OS' "kernel" (or main, controlling program) must be able to provide the "memory models" that programs need. What the programs do with those memory models is up to them -- and the source of compatibility issues.

The OS's kernel can provide any memory model that is compatible with its own. In most cases, this means any memory model that is the same or "smaller" although that can be a simplification. It all depends on what the memory model entails. E.g., some memory models that allow a program direct access to processor registers that are "protected" may not be allowed.

- History: DOS Memory Models

DOS is known for 3 main memory models. NOTE: There were actually more than just a few DOS Memory Models, but I don't want to spend my time talking about that.

- 16-bit DOS (16-bit offset) could provide 64KiB (COM)
- 20-bit DOS (16-bit segment + 16-bit offset) could provide newer 640KiB/1MiB (EXE)
- 32-bit DOS (16-bit segment + 32-bit offset) Protected Memory Interface (DPMI) could provide 16-64+MiB of memory, as well as legacy COM and EXE compatibility.

Early OS/2, Novell NetWare (DOS-based) as well as all MS-DOS 7.x (Windows 95/98/Me) versions _use_ DPMI. NT and Linux can provide DOS/Win16 Emulation via DPMI (that can run COM and EXE as well). In NT, this is the NT Virtual DOS Machine (NTVDM). In Linux, this is typically DOSEmu, DOSBox, etc... (although some software emulate a full 8086/8088, etc...).

[ NOTE: There were a few 24-bit (286/16MiB, 24-bit offset) and 32-bit (386/4GiB, 32-bit offset) Extenders that were not so compatible -- long story. ]

Programs and libraries of different memory models can_not_ use each other though. E.g., Win16 could not use Win32 libraries and vice-versa. Microsoft did, however, come up with Win16-on-Win32 so some shared objects, etc... could be used and run on the same OS.

Now let's talk the 32-bit space and higher.

- 32-bit, 4GiB "Flat" Addressing (i386)

In Intel i386, there is a 16-bit segment and a 32-bit offset register. The offset register is capable of addressing 32-bit (4GiB) "flat." The segment register is often used to designate a location in memory as the "start". Adding the 16-bit segment to the 32-bit off-set is known as "normalization" (also used in 20-bit DOS addressing, only with a 16-bit offset register).

-36-bit, Processor Address Extensions (i686)

In the Intel i686 (Pentium Pro on-ward) processors can now map above 32-bit. In coordination with the 16-bit segment register with Processor Address Extensions (PAE), which is a bit on the CPU. They use a 3-level Translation Table, hence 36-bit PAE mode. The offset still allows programs to use the same 32-bit flat space and work together, the OS is doing the translation and paging above 4GiB. Some programs can be written to PAE36, although interacting with 32-bit libraries
may or may not be an issue (depending).

- NT and Linux PAE support

Before I can continue, I have to explain kernel (including I/O, memory mapped I/O and other reservations) and userspace.

In NT (including NT4.0, 5.0/2000 and 5.1/XP/2003) use a split 2GiB/2GiB kernel/user model. The consumer and even entry Server versions (even XP Pro???) only allow an absolute of 4GiB -- possibly only 2GiB of "user" memory.Advanced Server versions still use the same model, but they support PAE and can map in 512MiB pages in the 2GiB user portion, up to 64GiB.

Yeah, forget more memory for running Battlefield 2 on XP. But even if you ran a Server version which did support more, it would be slower because of the paging overhead.

Linux kernels can use a variety of modes. In fact, if you have 1GiB or less of memory, it is recommend you use the 32-bit 3GiB/1GiB kernel/user model, which gives you 960MiB (just under 1GiB) usable, but uses _no_ page translations (10-20% faster). The other mode is to use "HIMEM" or the 3-level PAE mode. The most popular is now the 4GiB/4GiB model. The kernel is always in memory, but is virtualized to a 4GiB area. It then maps in different 4GiB user pages as
necessary.

But in all cases, there is still only a 32-bit offset register, and the 16-bit segment register and "normalized" 32-bit address is often used alongside page tables.

-52-bit PAE and 48-bit "Long Mode" (x86-64)

x86-64 now introduces a new model called "Long Mode." On the page table side, it is a 4-level, 52-bit PAE approach that is completely compatible with 36-bit PAE. In other words, a 52-bit PAE kernel can also provide 36-bit PAE and 32-bit memory model compatibility (probably DPMI as well, to a point).

"Long Mode" also offers a 48-bit (256TiB) "Flat" address model by using the 16-bit segment register as bits 32-47 to the 32-bit offset register of bits 0-31. This is the new "memory model" that programs and libraries may use.

48-bit "flat" programs cannot call 32-bit "flat" libraries and vice-versa. It's not the kernel, but the way programs talk to each other. Outside of the kernel, if an OS ships with 48-bit libraries, then they cannot be used by 32-bit programs. Therefore, some x86-64 OSes only ship with a PAE52 kernel, but still offer largely 32-bit libraries and programs, with a few 48-bit exceptions.

NOTE: FYI, there is a 40-bit (1TiB) _physical_ platform limitation to current AMD64 designs. It's due to the Athlon64/Opteron's core design being shared with the older, 40-bit EV6 Althon. Yes, the original "32-bit" Athlon was capable, on the physical platform, of addressing 40-bit. Hint: EV6 was designed for 64-bit Alpha 264, not Athlon.

- Windows 64-bit v. GNU/Linux x86-64

Windows XP 64-bit edition is still largely a 32-bit OS. It ships with a PAE52 capable kernel, but at the heart of Windows is Win32. Win32 is heavily x86 (data alignment ignorant, long story), and while Win64 is slowly developing set of replacements, the main component is Win32-on-Win64 (WoW). It's a way so Win64 programs can use Win32 objects, which are at the heart of the supposed "64-bit" OS still.

Independent Software Developers (ISVs) must come up with most of their own libraries in the absence of complete Win64 support. Unfortunately, most core OS librares are still 32-bit, which is a major issue. As such, many 32-bit versions of applications run faster on the 64-bit version (let alone even the 32-bit version) than 64-bit versions of applications -- depending on if they applications use 32-bit OS interfaces, or 64-bit ones.

GNU platforms have been "64-bit clean" for a long time, and GNU/Linux for Alpha pioneered this over a decade ago. The result is that Red Hat and SuSE have shipping, true 64-bit versions with 48-bit applications and libraries. The Linux Filesystem Hierarchy Standard (FHS) and newer IEEE POSIX 2001+ and X/Open Single UNIX Specification (SUS) v3 standards define that on 64-bit platforms that offer both 64-bit and 32-bit memory models, /lib contain the legacy 32-bit libraries and /lib64 contain the new 64-bit libraries.

So on a GNU/Linux platform, such as Fedora Core, Red Hat Enterprise Linux, SuSE Linux, etc... built for x86-64, all OS appliations are 64-bit (48-bit memory model) and their libraries are located in /lib64 (48-bit memory model), while very few applications are 32-bit, and a good set of common libraries are available in 32-bit under /lib. This includes multimedia (DirectMedia, DSL), 3D (OpenGL), etc... Even the X-Window version 11 (X11) graphical system is truly 64-bit, but can display 32-bit X11 programs, even those that use OpenGL over X11 (GLX).

There are a few issues. One is the fact that web browser plug-ins are still typically 32-bit. Red Hat ships both a 32-bit and 64-bit (48-bit memory model) Mozilla with its products, but the more popular Firefox is only 64-bit and doesn't work with most plug-ins that are only available in 32-bit form (e.g., Sun only recently introduced a Linux/x86-64 version of its Java Runtime Engine, JRE).

But for the most part, GNU has been 64-bit for so long with portability tools (e.g., Autoconf, Automake, etc...) for builting on Alpha (always 64-bit), SPARCv9 (64-bit), MIPS4000+ (64-bit), etc... so almost all major Freedomware applications and libraries build without any modification. And most enterprise applications have been running on them from before Linux/x86-64 was an option.

This is wholly untrue of the Windows platform, which has always run as 32-bit even on the 64-bit Digital Alpha processor/platform.

USF Deserves Top-25 Votes (Withdrawn)

As a UCF Alumni who has seen extreme, repeated arrogance from USF fans in the past (don't get me started on who didn't want to play whom, even the USF's local paper and the Florida State Legislature knows that story), understand this is not something I'm stating out of any "love" for USF. But from a pure reality standpoint, USF is clearly being ignored in the AP and ESPN polls, and clearly deserving of some top 25 votes.

New Sagarin Rankings

It's far from atypical that by week 6 or so in the college football season that various rankings start to "settle out" as enough teams have played top 25, or previously alleged top 25 teams, and 2005 is not any different. 6-0 Penn State has clearly showed their worth as a Top-10 team, and it's time to stop the speculation of how good or bad they are.

Now while people focus on the computer ratings/rankings of the Bowl Championship Series (BCS), I like to focus on one far more accurate and respected. The USA Today Jeff Sagarin ratings/rankings are respected because of their completeness. In American college football, there is a major reason why the BCS is a joke, and the Sagarins are far more accurate -- how division I-AA teams are handled.

In the BCS system, beating up on a division I-AA team will net you a LOT of points. In fact, this was one of my biggest issues with many USF fans when USF used to play 2-3 I-AA teams a year, and UCF played none (or 1 in the worst case back in 2003). Not only did they incorrectly state they, USF, qualified for a bowl (sorry, your 2-3 wins against I-AA teams in various years don't count, and you didn't get 6 wins against I-A teams to qualify -- simple math USF fans!), but the BCS statistics used to get quoted to me. I always relied on the Sagarins because Jeff rates and ranks all I-A and I-AA teams against each other, which currently results in no I-AA team being higher than #63 against all I-A teams as well. This factors well into how I-A team wins against I-AA teams are calculated, whereas the BCS rewards beating up on I-AA teams.

Well, the new Sagarins are out and the Penn State record has clearly shifted a lot ...
- NCAFF Sagarin by Conference
- NCAAF Sagarin by Team

USF Is Clearly a Top-25 Team Right Now

The new rankings put UCF at #47 in the Sagarins. No UCF fan, coach or player is boasting right now, and UCF knows it could easily lose many games the Sagarins say we should win in the future. But for now, the #47 ranking is fairly accurage. South Carolina is #46, which beat UCF who was on the road. #72 is Marshall, who lost to UCF while they were on the road, Sagarins showing a 6 point differential.

Should UCF play to their ability, we should lose at Southern Miss (4.5 differential), but win all our others, with a close one against Houston at home (5 point differential). No UCF fan, coach or player is going to predict such, we know where we were not long ago. In fact, we barely beat #142 UL Lafayette, so that's clearly an indicator that the top-100 may not be where we deserve in the end. But for now, we have to hope we play to our ability -- and as O'Leary has said, a bowl game is our goal (of which C-USA has 5 tie-ins).

Which brings me to USF. Other than getting kicked by Miami, of which USF hurt themselves far too often, USF has played well. They played Penn State tough, who are now ranked #5. And they trashed Louisville, although they had home field advantage and Louisville self-destructed. Which is why USF has a very accurate #16 ranking in the Sagarins, most notably the #7 hardest schedule to date. Considering they played Miami (again, lost handily), Penn State (lost only by a little) and Louisville (beat handily), they are 3-2 not because they are not a good football team.

Now does this mean they deserve to be in the Top 25? There are several teams that deserve to be in the Top 25 of either the AP or ESPN polls, and they get votes, but they don't gather enough to be in the Top 25. Even UCF got a few 24th place votes back in 1998 (as well as a "25th place moral vote" in 1997). But what has USF seen this year?

Nothing! Not even a single sportswriter or coach has given South Florida a single vote in either poll. As much as I cannot stand the overwhelming majority of trash talking USF fans who do not bother to extend the same courtesy to us UCF fans (yes, we're older, yes we didn't put money into our football program -- but no, your AD/coach didn't want to play us, and it took a legislative bill in process to get your AD to finally agree to play us -- ALL FACT!), USF isn't being fairly considered.

They have played tough teams, and with exception of Miami, they have either won or played enough to prove to be a Top-25 contender. It's just wrong and plain ignorant for not a single voter in either poll to give them at least a 20-something vote. They deserved a vote, especially after Penn State's convincing win this past week.

USF Should Know What To Do (But Will They Do It)?

So if I'm USF, I'm looking to trash Pitt this coming week, and then a message by beating poll-bordering West Virginia the week after. South Florida can do it. They have to do it. Take the frustration out on your next two opponents and there will be no question. UCF had to do this too, and we came up short far too often.

Unfortunately, after seeing what happened at Miami, there's one thing that UCF can continue to say that USF hasn't done yet. UCF knows how to play against huge crowds since we've been doing it for the last decade. USF's 2nd largest crowd was home against UCF, almost 50,000, as other trips have been sub-50,000 with the exception of Penn State. USF has to grow up quickly if they want to cancel out that factor -- although their play at Penn State was a good sign.

Again, it's up to USF to get that respect. I will, of course, be rooting against them. But as of right now, they have played well enough to get at least some Top 25 votes, and I have to admit that.

Low-Cost LPI Exams: LinuxWorld Netherlands

The Linux Professional Institute (LPI) is offering low-cost exams at the LinuxWorld in Utrecht (Netherhlands) on Wednesday and Thursday, 2005 October 9th and 10th. The cost will be eu35, compared to the normal Vue/Prometric price of eu110.

For more information on this testing offer, see the official LPI press release.

For more on the non-profit, distribution-neutral, training-neutral LPI program, see the LPI Certification Program pages.

Flipped ATX Cases Gone Mainstream ...

I've been signing the weight, form-factor and, yes, even cooling praises of a 11" x 9" x 14" "Small Enough Form Factor" MicroATX (with full ATX power supply) Cube enclosure, but not everyone goes for the MicroATX form-factor which limits you to 4 slots and only 5 drives (only 2 x 5.25"). As such, many are still looking for a bit more expansion.

- Cooling the Slots (and CPUs) Better

One thing I really dislike about ATX is the "heat trapping" of the AGP, PCI, PCI-X and even the new PCIe slots. The "top" of the board with components is facing "downward" in the typical ATX tower. The aforementioned MicroATX cubes solve this (like any desktop), most ATX systems still ignore it, or try to introduce workarounds. But the reality is that the typical ATX tower traps heat and still leaves the CPU in an area where it gets a lot of heat.

Intel introduced the BTX form-factor, which finally flipped the slot orientation, which I've been calling for a long time now. But the still puts the CPU in the front-middle, making it incompatible. Not only that, but it limits expansion and doesn't offer a dual-slot option. In other words, it makes sacrifices many positive things about ATX (and MicroATX), and also introduces a radical change in compatibility. So it has caught on with little other than Tier-1 OEMs feed directly from Intel.

- Flipped ATX Enclosures

Thankfully a few vendors took note last year and introduced a few options. Unfortunately, most were very, very expensive. I went after the first one, the Lian Li PC-V series. I opted for the Extended ATX (12" x 13") version, the Lian Li PC-V1200 (black). It's about the same price as the Standard ATX (12" x 9.6") version, the Lian Li PC-V1000, although many complain about the lack of space for many power supplies in the PC-V1000 (so the PC-V1200 might be a better choice). They are the sturdiest alumnium cases you will ever find (very, very thick panels, good, internal structure), and you'll pay for that cost ($200+). They have sinced added the "Plus" model, which adds a large 120mm radial fan over the slots for a total of 3 fans in the case. I added it as a $25 add-on for my original model, but the new "Plus" model is typically only $10 more and well worth it.

The very thick and sturdy, all-alumnium Lian Li PC-V1200 fits a server-sized EATX (12" x 13") mainboard, just about any sized PS (including EPS12V) in the lower bay, the intake 120mm keeps my five (5) 7200rpm drives cool, and the CPU and "above slot" radial 120mm outtake fans keep the CPUs and cards cool.

But the Lian Li PC-V series is definitely overkill for most people. I had also been following the introduction of the steel structure, alumnium front SilverStone TJ-06 with its "CPU wind tunnel", but it showed up at distributors and resellers months after the Lian Li PC-V. And it wasn't much of a savings, while being taller and heavier than the Lian Li PC-V.

Within 6 months, a few other clones started to arrive and were highlighted in Tom's Hardware Guide's 2005 April Midi ATX Towers review. In that review, no less than three (3) new versions of the same OEM design as found in the SilverStone TJ-06 appeared. Today, NewEgg not only features two (2) of those designs, plus the SilverStone TJ-06 itself:

• Enermax CS-718
  Tom's 2005 April Review -- NewEgg Product Page
• Sharktoon Silverton
  Tom's 2005 April Review
• SilverStone TJ-06
  Tom's 2005 April Review -- NewEgg Product Page (1/3)
• Thermalrock Circle RH-F030
  Tom's 2005 April Review -- NewEgg Product Page
  

But another, much cheaper $85 alternative.

The $85 Aerocool Sprial Galaxies features the flipped ATX design with a full 120mm in to 120mm out CPU wind tunnel for, a bargain compared to comparable designs that are $50+ more, while still fitting even a full EATX (12" x 13") mainboard and EPS12V power supply.

Cheap/Local Doesn't Have To Be Too Bad

Of course, if don't want to pay a fortune for most of the above products (let alone to ship them -- although the $85 Aerocool with $15.99 3-day shipping from NewEgg is hard not to consider), or want to find something local, things don't have to be too bad. A good 120mm outtake fan does wonders, and even a side-mounted, CPU intake duct works very well -- without the flip slot orientation. There are many, many variants of the common 10-slot design these days -- whether the stright ten (10) 5.25" bays, or maybe six (6) 5.25" and six (6) 3.5" internal, side-facing HD bays -- for well under $50 with a power supply.

Even Antec has a good presence at Best Buy and CompUSA now. Not only does the more costly alumnium Antec Super LANBoy come with 120mm fans, but even their cheaper, shorter (and often on-sale for under $50 after rebate) Antec SLK2650-BQE comes with a 120mm outake fan and a 80mm side intake duct. Short of needing a case for a dual-Opteron EATX mainboard with 600W EPS12V power supply, even just the local sub-$50 case is all that is needed for most cooling requirements these days -- possibly even an SLI (with a heafty ATX 2.0 PS).

Of course, if you're going to put in only a few 5.25" and 3.5" drives and not going SLI for video, I can't help but suggesting maybe MicroATX is all you need. Especially with the new nVidia C51 (GeForce61x0+nForce4x0) mainboards out now. Now only if someone made an SLI version.