An article like this requires a lot of caveats and many large grains of salt. But we were interested to see what would happen if we compared the performance of a very early version of Microsoft's upcoming desktop OS with Apple's released one, and we thought you would be, too. Windows 8 is far from finished, but its creators have made bold claims about performance. I've already compared the Windows 8 Developer Preview with Windows 7, which presented its own challenges.
Comparing with Mac OS X introduces even more hazards. On one hand the version of Windows 8 we test with here, the Developer Preview is far from finished—it's not even ready to be called a "beta"—and code optimization is usually among the last tasks in any software project. On the other hand, we have Apple's fully baked and optimized Mac OS X Lion, which has been a released product for over six months. Add to this that Lion is finely tuned to mesh with the Apple hardware it runs on, compared with Windows, which must run on a huge array of different hardware combinations from many vendors.
Despite all the warnings, as you'll see, Windows 8 was surprisingly up to the challenge. And indeed, some of Windows 8's developers' big claims are that it will take up less memory, run fewer CPU processes, and boot faster, all of which should add up to better performance.
| Mac OS X Lion | Windows 8 Developer Preview | |
| Startup (seconds) | 32 | 26 |
| Shutdown (seconds) | 2.7 | 16 |
| CD Ripping in iTunes (min:sec – lower is better) | 3:44 | 3:52 |
| Geekbench 2.2 64-bit score (higher is better) | 10212 | 11920 |
| SunSpider in Firefox 10 (ms, lower is better) | 181 | 195 |
| Mozilla Kraken 1.1 (ms, lower is better) | 2967 | 2789 |
| Psychedelic Browsing (RPM, higher is better) | 2760 | 7159 |
| *Green cells denote the winner. | ||
To put those claims to the test, I installed Windows 8 Developer Preview on a MacBook using Boot Camp, and ran a few comparison tests. Since most standard benchmark suites, such as Futuremark's PCMark and 3DMark, don't run on Macs, I had to make do with whatever measures would run on both. Startup and shutdown times, of course, along with some browser based benchmarks (Mozilla Kraken, Sunspider, Psychedelic Browsing). The only standard performance benchmark I use for this comparison is Primate Labs' Geekbench 2.2.
Setup
I ran the tests below on a late 2011 15-inch MacBook Pro with a 2.2GHz Intel Core i7 and 4GB DDR3 RAM. I installed 64-bit Windows 8 Developer Preview in a Boot Camp partition. I ran all tests run three times and averaged the results.
I ran the tests below on a late 2011 15-inch MacBook Pro with a 2.2GHz Intel Core i7 and 4GB DDR3 RAM. I installed 64-bit Windows 8 Developer Preview in a Boot Camp partition. I ran all tests run three times and averaged the results.
Startup and Shutdown Times
The time it takes to get your computer up and running can make the difference between getting that idea down or information retrieved in time or not. To determine the startup time for each OS, I started the stopwatch (actually an iPhone) at the point of choosing the OS from the multi-boot menu (arrived at by holding the Option key after pressing the start button) and stopped it when the desktop displayed and was functional. My results here were quite shocking: Windows 8 booted faster on the MacBook than OS X Lion did! Of course, Lion is a bigger OS with more tools and features at this point, so there's probably a good reason for this. The difference was 31 seconds for Lion, and 26 seconds for Windows 8 Developer Preview.
The time it takes to get your computer up and running can make the difference between getting that idea down or information retrieved in time or not. To determine the startup time for each OS, I started the stopwatch (actually an iPhone) at the point of choosing the OS from the multi-boot menu (arrived at by holding the Option key after pressing the start button) and stopped it when the desktop displayed and was functional. My results here were quite shocking: Windows 8 booted faster on the MacBook than OS X Lion did! Of course, Lion is a bigger OS with more tools and features at this point, so there's probably a good reason for this. The difference was 31 seconds for Lion, and 26 seconds for Windows 8 Developer Preview.
For shutdown times, the opposite state of affairs prevailed: OS X Lion shut down completely in under 3 seconds, while Windows 8 took 16 seconds. I ran the stopwatch from the time of choosing the equivalent of the "really shut down" choice in each OS to the time the laptop's motors all went quiet. It would seem that if Lion is loading more code into memory, it would need more time to save state and so on for shutdown. And Windows 8's developers have made claims about much more efficient state-saving for that OS's shutdown, but the numbers don't lie: Mac OS X Lion has a far more efficient shutdown procedure.
iTunes Ripping Test
A popular app used in both OSes is iTunes, and I measured how long ripping a CD took in each (Buena Vista Social Club, to be exact.) This test didn't show much difference between the two OSes, with Lion coming in a scant 8 seconds quicker. It took Windows 8 3:52 to rip the 60 minute disc to 256Kbps M4A tracks, while Lion took 3:44. So the newcomer at least held its own against the finished OS here.
A popular app used in both OSes is iTunes, and I measured how long ripping a CD took in each (Buena Vista Social Club, to be exact.) This test didn't show much difference between the two OSes, with Lion coming in a scant 8 seconds quicker. It took Windows 8 3:52 to rip the 60 minute disc to 256Kbps M4A tracks, while Lion took 3:44. So the newcomer at least held its own against the finished OS here.
Geekbench
Geekbench, from Primate Labs, runs a series of geeky tests like prime number, Mandelbrot, blowfish encryption, text compression, image sharpen and blur, and memory stream test. The subtests comprise both single- and multithreaded applications. The results are normalized so that a score of 1000 is the score a Power Mac G5 1.6GHz, so a higher number is better.
Geekbench, from Primate Labs, runs a series of geeky tests like prime number, Mandelbrot, blowfish encryption, text compression, image sharpen and blur, and memory stream test. The subtests comprise both single- and multithreaded applications. The results are normalized so that a score of 1000 is the score a Power Mac G5 1.6GHz, so a higher number is better.
In another surprise, Windows 8 bested Lion slightly on this test. Since the benchmark is mostly designed to rate the hardware, any slight improvement is a feather in the OS software's cap. Windows 8 showed a nearly 17 percent edge over Lion. The test maker even encourages comparing the competing operating systems: Its page's Cross Platform section challenges, "Compare apples and oranges. Or Macs and PCs."
JavaScript Browser benchmarks
Browser benchmarks offer a good cross-platform test. I compared JavaScript performance using two standard benchmarks: WebKit's SunSpider and Mozilla's Kraken. I used the same browser for each OS Firefox 10, since IE10 doesn't run on Mac OS X. But I was met with mixed results on these tests. The simpler SunSpider had Lion ahead, while on the more thorough Kraken, Windows 8 showed to advantage.
Browser benchmarks offer a good cross-platform test. I compared JavaScript performance using two standard benchmarks: WebKit's SunSpider and Mozilla's Kraken. I used the same browser for each OS Firefox 10, since IE10 doesn't run on Mac OS X. But I was met with mixed results on these tests. The simpler SunSpider had Lion ahead, while on the more thorough Kraken, Windows 8 showed to advantage.
Psychedelic Browsing
This test from the Internet Explorer development team aims to show off a browser's ability to accelerate Web content using the system's graphics hardware. It spins a rainbow-colored circle and reports a result in RPMs. For this one, I originally tried Firefox in both systems, but that browser didn't show hardware-acceleration on the Mac version. I then tried Google Chrome, with the same result. The answer was to simply use each OS's native Web browser—IE10 for Windows 8 and Safari 5.1.3 for Lion, which actually showed signs of hardware acceleration support, where Firefox and Chrome hadn't, yielding results in the double digits, rather than in four figures. My final result on native browsers was Windows 8, 7159, and Lion, 2760. Both compare well with results like 65 for non-hardware-accelerating browsers.
This test from the Internet Explorer development team aims to show off a browser's ability to accelerate Web content using the system's graphics hardware. It spins a rainbow-colored circle and reports a result in RPMs. For this one, I originally tried Firefox in both systems, but that browser didn't show hardware-acceleration on the Mac version. I then tried Google Chrome, with the same result. The answer was to simply use each OS's native Web browser—IE10 for Windows 8 and Safari 5.1.3 for Lion, which actually showed signs of hardware acceleration support, where Firefox and Chrome hadn't, yielding results in the double digits, rather than in four figures. My final result on native browsers was Windows 8, 7159, and Lion, 2760. Both compare well with results like 65 for non-hardware-accelerating browsers.
Conclusion
These tests confirm what many, including us at PCMag.com have experienced: Windows 8 is no slouch—so far. It feels snappy on pretty much any hardware you throw it at. And here we are at a very early stage in its pre-release. It speaks well that the nascent OS can hold its own against a mature, fully optimized, hardware specific OS like Lion. It's worth mentioning again that it's early days yet, and we can't possibly know how much more code Microsoft is going to add to Windows 8 in the next few months, but we can hope that they don't add too much to bog it down before release. It certainly seems that their strategy this time is in the opposite direction, towards honing it down.
These tests confirm what many, including us at PCMag.com have experienced: Windows 8 is no slouch—so far. It feels snappy on pretty much any hardware you throw it at. And here we are at a very early stage in its pre-release. It speaks well that the nascent OS can hold its own against a mature, fully optimized, hardware specific OS like Lion. It's worth mentioning again that it's early days yet, and we can't possibly know how much more code Microsoft is going to add to Windows 8 in the next few months, but we can hope that they don't add too much to bog it down before release. It certainly seems that their strategy this time is in the opposite direction, towards honing it down.
Comments
Post a Comment