Browsers - Stress Tested

The other day I came across a huge (6.49 MB) .htm file generated as output by SYSTAT (a powerful, yet surprisingly user friendly desktop application for statistical analysis. See www.systat.com). Opening this file in a browser can really stress-test the latter.

I tested opening the file in different browsers – IE-7, IE-8, Firefox 3.5.1, Safari 4.0, Opera 9.64 and Chrome 2.0. I also observed the memory usage and quality and speed of scrolling when this big file is open. The results are as below.

First I explain the methodology.

Rendering

I open the browser with blank page. Drag n drop the file in it. The browser window does not show the content of the file immediately as you drop it. For big files, it takes even more time. Till that time, the blank page remains blank. From the point of dropping it in the browser, till something appears in the window, the time is measured. This is called partial rendering time. But, still the full page is not loaded. The browser shows top part of the page, while still loading incrementally the remaining lower part of the page. When the full page is loaded and indicated by the browser in some way, like 'Done' appearing in the status bar in case of IE, the time is measured. This is called completion time.

Memory Usage

Once the full page is loaded, I look at the task manager so as to how much memory is being used by the browser. This is 'memory usage maximized'. Then the browser window is minimized and memory usage is noted again, to record 'memory usage minimized'.

Scrolling

With window maximized and page at the top end, 'Page Down' key is kept pressed till the end of the page is reached. The time recorded is called 'time taken' for scrolling. The quality is recorded as per user experience; if it is smooth or jerky scroll.

Now the results!

Safari 4.0 performs best at rendering. It opens the file in 5 seconds. All those 5 seconds the page shows blank. Then the whole page is visible at once after 5 seconds. Scrolling the page is absolutely smooth. It takes 38 seconds to scroll down the page using 'Page Down' key. Yes the page is so big! If we count the number of times 'Page Down' is pressed, perhaps we can get the 'length' of the page in physical length units like inches or centimeters. Safari shows to be worst in memory usage. It hogs 348
MB of memory when in maximized state, which reduces to 3.8
MB when minimized.

IE-7 appears blank for 6.2 seconds. Complete rendering takes 16.4 seconds. Scrolling the page is jerky. Scrolling down the page takes a whopping 9 minutes 37 seconds (577 seconds). It was a pain testing this! Better than Safari, IE-7 uses 297 MB of memory when in maximized state, which reduces to 3.8 MB when minimized.

IE-8 appears to be worse than IE-7 for these tests. The page shows blank for 15.1 seconds. Then after a few seconds rendering finishes at 16.9 seconds. Scrolling the page is jerky. Scrolling down the page takes 9 minutes 15 seconds (565 seconds). Worse that IE-7, IE-8 hogs 296 MB of memory when in maximized state, which DOES NOT reduce when minimized. It continues to hog 296 MB of memory!

Firefox 3.5.1 appears to be the slowest in HTML rendering! Though it shows content after 1.5 seconds, the whole page takes 24.2 seconds to load. Scrolling the page is absolutely smooth. Scrolling down takes 48 seconds. It uses 278 MB of memory when in maximized state, which DOES NOT reduce when minimized.

Chrome
2.0 shows content in 1.5 seconds, loads the full page in 10.5 seconds. Scrolling the page is jerky like that in IE and takes 38 seconds. Chrome uses 270 MB of memory when in maximized state which reduces to 135 MB when minimized.

Opera
9.64 shows content in 1.6 seconds and loads the full page in 11.2 seconds. Scrolling the page is absolutely smooth and takes 40 seconds to the bottom of the page. Opera uses least 219 MB of memory when in maximized state, which reduces to least 2.5 MB when minimized. Opera is best with memory usage.

Summary

Properties	Rendering		Memory Usage [MB]		Scrolling
Browsers	Partial Time [seconds]	Completion Time [seconds]	Maximized	Minimized	Smoothness	Time Taken [seconds]
Chrome 2.0	1.5	10.5	270	135	not smooth, jerky	38
Firefox 3.5.1	1.5	24.2	278	276	absolutely smooth	48
IE 7	6.2	16.4	297	3.8	not smooth, jerky	9 min 37 s
IE 8	15.1	16.9	296	296	not smooth, jerky	9 min 25 s
Opera 9.64	1.5	11.2	219	2.5	absolutely smooth	40
Safari 4.0	5	5	348	3.8	absolutely smooth	38

Bright Green	Best
Green	< Best
Purple	< Worst
Red	Worst

Opera turns out to be the all round balanced browser with no Purple or Red areas, with rendering performance next to Safari and at par with Chrome. Opera is the best with memory usage. IE-8 turns out to be the worst with two Purple and four red areas. IE-7 is better that IE-8 with memory usage and partial rendering time. Except for hogging large memory which is a thumb down, Safari is the fastest browser. 5 seconds for loading 6.49 MB file is a feat! Firefox, the most widely used after IE; maybe we can say the most popular browser by merit (not by forceful use) disappoints in two areas - page loading time and memory usage. I was surprised by these results as I primarily use Firefox.

Note that those that show the content quickly, Chrome, Firefox and Opera, deliver better user experience; maybe because we do not like to wait, more so when the wait is still not over. Safari, though makes you wait for 3.5 seconds more, there is no more waiting as it loads the full page by then!

So, with respect to these tests, the rankings are:

Safari > Opera > Chrome > Firefox > IE-7 > IE-8

Please note that the ranking will change if we bring in more parameters for testing like availability of quality add-ons or plug-ins, security, tab handling etc.