Theoretically, specialized operating systems (such as Mac OS or Sun's OS) should have a performance advantage over a generalized system such as Windows. This is because the specialized systems can be optimized for the hardware, whereas a generalized system must be designed to work on as wide an array of hardware as possible.
Open source software should have the benefits of both types of operating system. Since it is distributed with source code, it can be compiled with optimizations particular to a specific hardware, while mainstream distrubtions can be compiled without optimizations to allow the greatest possible compatibility.
To test this theory, I took a computer that was being recycled back into production use and made some tests of it during setup. The computer was installed with FreeBSD 4.9. It is a 466mhz Pentium II.
To test my theory, I took a large (150M) file and used the bzip2 and bunzip2 programs to alternatly compress and decompress it, and the md5 program to generate a checksum of the file. Each process was run 3 times and timed. The entire system (world and kernel) was then rebuilt with compiler flags appropriate to the hardware (CFLAGS= -O -pipe and CPUTYPE=p2). This recompile includes the bzip2, bunzip2, and md5 programs themselves. The following table displays the results.
| Process | Unoptimized average | Optimized average | % improvement |
|---|---|---|---|
| bzip2 | 189.77s | 189.68s | 0.05% |
| bunzip2 | 48.14s | 47.05s | 2.26% |
| md5 | 4.85s | 4.70s | 6.19% |
Each test was run three times and the results averaged. Dummy runs were done on each test to ensure that the processor was the bottleneck and not I/O or some other factor.
Now that I've run the tests, I can see the need for more testing (see caveats). But it definately seems as if compiling from source is worth the effort. This gives all open-source software a performance advantage over any closed-source software that can not be compiled from source.
Obviously, the results show that not all applications are going to improve the same amount due to optimization. All of the programs tested are very processor intensive and should yield noticable improvements due to optimizations, but it is possible that other applications could see a larger improvement.
A 5% gain is not going to overcome all performance problems, but on a system designed for high-throughput, that 5% advantage can be significant.
I'd like to find some other programs to test, to get a wider idea of how much variance there is.
The processor in use was only a PII. It's possible (even likely) that more modern processors will see an even larger improvement, since they have more optimizations available.
Since the kernel was rebuilt along with the programs tested, it's likely that some of the improvment is due to a faster kernel.