Home

Section 2: Measuring Progress


Setting A Baseline

It is a common mistake to make performance oriented modifications to a website before measuring existing site performance. By doing this, it can then prove impossible to determine whether your changes have resulted in real performance improvements, or if instead they have resulted in reduced performance. For this reason, the first thing you should do is to set up proper monitoring of your website, and to quantify your current performance.

What To Monitor

There are many useful measurements that can be regularly monitored on your website, and a large number of tools that can help with taking these measurements. Each website will some unique monitoring needs, however there are some basic measurements that most all websites will want to regularly monitor. The following list will give you some ideas of what you should monitor. Thought the list is numbered, the numbers do not indicate the level of importance of each item. Instead, we number items so we can provide specific examples when we discuss monitoring tools.

  1. The time it takes to load the front page when not logged in and with nothing cached by your browser.
  2. The time it takes to load the front page again, when not logged in but when you already have the CSS, JavaScript and images cached by your.
  3. The time it takes to load the front page page when you're logged in.
  4. The time it takes to load each of your 25 most popular types of web pages.
  5. The time it takes to load the above pages from different areas of the world.
  6. The popularity of the various types of web pages on your website. (Example types of pages include the front page, forum pages, RSS feeds, and custom pages generated by modules.)
  7. Server resource utilization such as CPU, load average, free memory, cached memory, swap, disk IO, and network traffic.
  8. The number of pages being served by your web server(s).
  9. Your database, including including the number of queries per second, the efficiency of your query cache, how much memory is being used, and how often temporary tables are being created.
  10. Database queries taking more than 1 second to complete.
  11. Database queries not using indexes.
  12. The number of searches being performed per hour.
  13. Memcache, including how much memory is being used, how many queries are being made per second, and your hit versus miss rates.

Monitoring Tools

You will need to use multiple tools to fully monitor your website. Some of these tools can run on your existing infrastructure, while other tools may need to live outside of your network.

ps

Ps displays information about all processes that are currently running on your server. The command line utility supports a large number of optional flags that control which processes are displayed, and what information is displayed about each process. Information that can be displayed includes CPU usage, memory usage, how much CPU time the process has used, and much more. Common invocations of ps include ps -ef and ps aux. Learn more about ps by typing man ps on most Unix servers.

top

Top provides an automatically updating view of the processes running on a server. It offers a quick summary of a server's health, showing CPU utilization, as well as memory and swap usage. Processes can be sorted in many ways, such as listing the processes that are consuming the most CPU, or the processes that are using the most memory.

vmstat

Vmstat offers a useful report on several areas of system health, including the number of processes waiting to run, memory usage, swap activity, CPU utilization, and Disk IO. A common invocation of vmstat is vmstat 1 10. Learn more about vmstat by typing man vmstat on most Unix servers.

Sar

Sar is part of the Sysstat collection of Unix performance monitoring tools. Sar can be configured to collect regular comprehensive snapshots of a system's health without putting any noticeable load on the system. It is a very good idea to enable Sar on any server that you are managing, as the historical information this utility collects can prove invaluable when tuning a server, or when performing damage control on a failed server.

Cacti

Cacti is a PHP front-end for RRDTool, displaying useful graphs based on historical data collected from your servers. By default it tracks useful system information such as CPU and memory utilization, however it can also be integrated with programs such as MySQL, Apache, and memcache, displaying useful historical graphs of their performance.

YSlow

YSlow is a FireFox add-on that enhances Firebug to analyze how quickly your web pages load, highlighting areas that can be improved. This tool is discussed in depth in chapter 13.

AWStats

AWStats is a log analyzer that can be used to create graphical reports from web server and proxy log files. When scaling a Drupal website, you can achieve better performance by disabling Drupal's core statistics module, and instead using AWStats to generate regular reports from Apache's own access logs.

devel module

The devel module is one of a suite of development oriented Drupal modules. Among its many useful features, it can display a list of all queries used to build each page served by a Drupal powered website, highlighting slow queries and queries that are run multiple times. The devel module is discussed in depth in chapter 6.

mysqlreport

Mysqlreport is a perl script that generates reports based on numerous internal "status variables" maintained by MySQL. With this script, you can quickly interpret what these variables mean, helping you to tune your server for better performance. Mysqlreport is discussed in depth in chapter 22.

mysqlsla

Mysqlsla, the MySQL Statement Log Analyzer, is a perl script that helps you analyze MySQL logs. This script will be discussed in depth in chapter 23, detailing how it can be used to review MySQL's slow query logs.

mytop

Mytop is a useful tool for monitoring a MySQL database from the command line. It offers a summary of database threads in a format similar to how top lists running server processes.

innotop

Innotop was originally written to monitor MySQL's InnoDB storage engine, but it has long since evolved into a very powerful tool for monitoring all aspects of MySQL. Inspired by mytop, it takes MySQL monitoring to a new level.

MySQL Enterprise Monitor

The MySQL Enterprise Monitor is a commercial offering by Sun Microsystems for monitoring one or more MySQL servers. The comprehensive tool provides useful charts and graphs, makes tuning suggestions, and can send alerts when your MySQL servers need attention.

Online Services

There are many online services that can help you with monitoring your website. It is beyond the scope of this book to list and review all of these services, but popular examples include Google Analytics, IndexTools, ClickTracks and Omniture. Other online services can help you to understand how quickly your web pages are loading from various locations around the world, including Keynote, Webmetrics, Alert Bot, and host-tracker.com.