The HTTP Website Monitoring Blog

UTF-8 is your friend – Part 3

httpmon — Thu, 30 Dec 2010 00:34:27 +0000

This is the third and last part of the UTF-8 is your friend blog posts, which aims to help you integrate UTF-8 support in the server-side code (covered in part 1), in the database (covered in part 2) and finally in the browser output, which we will cover here.

Luckily, enabling UTF-8 output on the browser level, using HTML, is the easiest part. For this, we are going to use the META HTML tag that provides metadata information for the HTML document. Usually, metadata are used for the keywords or description of the page for search engine optimization purposes, but can also be used to define the character set to be used. As a matter of fact, the one and only single line displayed below is all what you need to add to your HTML files.

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

If you have followed the three parts of these blog posts, you shouldn’t have any problem using any language in your web application thanks to UTF-8. That’s why UTF-8 is your friend and it’s worth learning how to tame it.

The hundredth monitored website

httpmon — Fri, 17 Dec 2010 17:51:13 +0000

Last month, we proudly celebrated our hundredth monitored website. It might sound like a small milestone, nevertheless, for us at HTTPMon, it is great to see people actually using this website monitoring service.

We are even prouder to say that the company who added the 100th monitored website is, in our opinion, one of the most serious and renowned PHP web development agency in Switzerland. In fact, to thank them for trusting us on monitoring one of their amazing projects, which by the way is a flash website event game developed for a public Swiss company, we are going to mention their name here. You might have already recognised the company I am speaking about; if not, it is Liip AG. Thanks guys and continue your great work!

We hope to see in the future many other companies or private persons trusting HTTPMon for monitoring the uptime and status of their websites.

The cron daemon was on strike

httpmon — Fri, 08 Oct 2010 20:04:17 +0000

The cron daemon, a job scheduler on Linux (and other UNIX operating systems), was for some unknown yet reasons not running for nearly a week. This means that unfortunately the prober was not checking the websites’ availability and therefore you will not see any probes for the period between the 30th of September and the 4th of October.

We are sorry about that, usually the cron daemon is very reliable but we will be checking closely to make sure it is running correctly again. We might also evaluate another method in the future not to be dependent on cron.

UTF-8 is your friend – Part 2

httpmon — Sat, 25 Sep 2010 22:43:52 +0000

In part 1 of this blog post series we have spoken about integrating the UTF-8 character set in your server side scripts. In this second part, we will introduce how to integrate UTF-8 in your database using MySQL as an example.

After having collected your data using PERL (or any other programming language) and successfully transformed the foreign character set encoded data to UTF-8, you will most probably want to store them into a backend, such as a database. In our case, we store this data in a MySQL database. For the data to be stored correctly it is of utmost importance that your database or the table that will contain those data is configured for the UTF-8 character set. Unfortunately, MySQL uses by default ISO-8859-1 (a.k.a. latin1), so you will definitely need to change the encoding to UTF-8 if you don’t want to end up with scrambled and weird characters in your tables.

If you want to enable UTF-8 for your whole database, which is by the way the simplest and safest method, you will have to append “CHARACTER SET UTF8” to your “CREATE DATABASE” SQL statement. An example is listed below for a fictive database named “mydb”.

CREATE DATABASE mydb CHARACTER SET UTF8;

It is also possible to configure UTF-8 on a finer level enabling this character set for specific tables of your database by adding “DEFAULT CHARSET=UTF8” to the “CREATE TABLE” statement as shown below for the fictive “mytable” table.

CREATE TABLE mytable (
        col1 VARCHAR(255) NOT NULL,
        col2 VARCHAR(255) NOT NULL
) DEFAULT CHARSET=UTF8;

If you have already created your database you are out of luck, because there is no way to modify the character encoding of the database once it is created. In that case you will need to recreate the whole database or table.

As you can see here, adding the default character encoding while creating your database or table is actually all you need to do to be able to store UTF-8 encoded data into a MySQL database. This is as simple as it gets about storing UTF-8 in a SQL database. In the third and last part of this blog post series we will see how to get UTF-8 correctly output to the web user.

UTF-8 is your friend – Part 1

Website monitoring full report

httpmon — Wed, 15 Sep 2010 16:42:07 +0000

As mentioned in a previous blog post, we were working recently on full reports for the monitored websites and are happy to announce that these are now available. The full reports enhances and complements the information already present in the various tabs (overview, last ten probes, server software, etc) of your monitored website and introduces dynamic charts. Thanks to the Google Visualization API (aka Chart Tools) we could make extensive use of its time series chart in order to graph your website’s responsiveness (or speed) as well as all the detailed connection timings to your monitored website.

To access the full report of your website, enter it as usual in the monitoring input field on the left side of HTTPMon, then in the first “Overview” tab you will find a new report icon under your website’s thumbshot, click that very same icon and you’re there! You can see an example with that icon in the image below.

The monitored website full report icon circled in red in the overview tab.

Note that your website will need to be at least monitored for 24 hours before you can access the full report. If your website has just been added to HTTPMon, the report icon will only appear after the first day of monitoring.

To summarise, the HTTPMon Full Reports provide you with the following monitoring information:

HTML meta description tag as well as title tag
Global overview
Web server software
IP address information
Last 24 hours probes
Theoretical speed chart
Name lookup/connection/transfer times chart
Overall uptime chart
Overall downtimes

With all the information in the full report, we hope that you will be able to get an even better insight into what is really happening on the hosting side of your website. We are also open to comments or suggestions if there are any particular monitoring data you would like to appear in this report.

UTF-8 is your friend – Part 1

httpmon — Fri, 10 Sep 2010 16:26:04 +0000

This is the first part of a series of three blog posts explaining how crucial it is to implement correctly UTF-8 character encoding in your web application, when dealing with content in various languages or encodings.

Having to deal with websites and their respective character encodings is not as easy a task as it may sound. This is even more true, when retrieving and storing content in various character encodings, and you might get caught in some pitfall if you don’t make sure you are handling the encoding correctly. A real case scenario is that of HTTPMon, which monitors websites in many different languages, such as english, german, greek and french, and so it must be able to parse HTML using any kind of character set.

The magic word to all your encoding problems is here UTF-8, and the key element is to handle your data all the way through, starting from the backend of your web application until the web browser, using this specific unicode character encoding. When I say backend, this means your SQL database or scripts that are running on the server side and processing some kind of data. Let me explain this on three possible levels (i.e. server side scripts, database, and HTML) of a web application and in three consecutive blog posts. This first part will focus on the server side scripting level.

Let’s assume your server side script is processing data, which could be in any kind of character encoding, and feeds these data to your database for storage. In this case you will have to make sure that the non-UTF-8 encoded data first get converted to UTF-8. The difficulty here all resides in the reliable detection of UTF-8 in the content, because in most of the cases there isn’t any ready-made function or method for this purpose.

For HTTPMon, we mainly use PERL as the server side scripting language for the backend, but since there are no ready-made functions to detect UTF-8, we had to come up with one. Luckily enough, we didn’t have to do much because Martin Dürst from the W3C already came up with a magical regular expression, in his Multilingual form encoding article, which does exactly that. All we did was to implement Martin’s regexp in an easy to call PERL subroutine named is_string_utf8 as you can see below.

sub is_string_utf8
{
    $metadesc =~
        m/\A(
           [\x09\x0A\x0D\x20-\x7E]            # ASCII
         | [\xC2-\xDF][\x80-\xBF]             # non-overlong 2-byte
         |  \xE0[\xA0-\xBF][\x80-\xBF]        # excluding overlongs
         | [\xE1-\xEC\xEE\xEF][\x80-\xBF]{2}  # straight 3-byte
         |  \xED[\x80-\x9F][\x80-\xBF]        # excluding surrogates
         |  \xF0[\x90-\xBF][\x80-\xBF]{2}     # planes 1-3
         | [\xF1-\xF3][\x80-\xBF]{3}          # planes 4-15
         |  \xF4[\x80-\x8F][\x80-\xBF]{2}     # plane 16
        )*\z/x ? return( 1 ) : return( 0 );
}

Thanks to PERL’s built-in UTF-8 support from the utf8 pragma, we could then simply encode the string with the utf8::encode() if it was not already in UTF-8 as detected by our is_string_utf8() subroutine. Basically, that’s all you need in your server side scripts, but make sure this is done correctly as you will have to store these into a UTF-8 formatted SQL database, which is the subject of part 2 of the “UTF-8 is your friend” blog post.

No related posts.

Alternative ways to access your monitored website

httpmon — Thu, 02 Sep 2010 18:01:11 +0000

Your life has just been made slightly easier: starting from today it is possible to access your monitored website without using its “www” prefix. This is of course only valid for websites using “www” in their fully qualified domain name (FQDN). For example instead of entering “www.gotrotting.com”, one could just enter “gotrotting.com”; both ways would now work.

In addition,it is also possible to access your monitored website without having to go through the home page of HTTPMon, but by simply using a specific URL. So to access directly your monitored website, use “http://www.httpmon.com/monitor/www.yourwebsite.com”. Using the same monitored website “www.gotrotting.com” as in the previous example, one would simply have to enter “http://www.httpmon.com/monitor/www.gotrotting.com” as URL directly on the web browser. Again, here you could omit the “www” prefix if your website uses one.

Validating HTML and CSS with W3C’s markup validation service

httpmon — Tue, 31 Aug 2010 17:58:53 +0000

As part of our milestone to reach to a rock-stable version of HTTPMon and get away from the current beta version, we have been recently cleaning and optimizing the HTML and CSS code. All our HTML code uses the HTML version 4.01 strict document type definition (DTD) and it is therefore possible to easily validate it using the markup validation service from the World Wide Web Consortium (W3C). The same applies for the CSS files as this validator can check many formats including XML. The outcome of this validation check was only two small warnings for the HTML code and none for the CSS. So we corrected the HTML code and revalidated it to be sure we had valid HTML without warnings or errors anymore, which is now the case. You can test our HTML and CSS code by yourself using the two appropriate buttons below.

Nowadays, it isn’t a must anymore to have 100% valid HTML code. In fact, most of the websites do not validate against their HTML DTD and in some scenarios it is not even possible to do what you want without generating some warnings or errors in your HTML code. An extreme example would be the home page of Amazon’s website: if you try to validate it you will find out that it has 492 errors and 96 warnings, still the website works fine… The take home message here is that if you are able to remove the HTML validation errors and warnings like us without harming the design and function of your website, why not give it a shot!

Large thumbshot of monitored websites

Uptime Graph section now includes monthly uptime/downtime graph

httpmon — Mon, 30 Aug 2010 18:30:36 +0000

After nearly four months since the first websites started to be monitored, we thought it would be interesting for their webmasters to graphically view the monthly uptime and downtime. So, to achieve this we added a new monthly graph under the Uptime Graph tab of a monitored website, as you can see in the example below.

Example of a monthly uptime/downtime graph for a website with increasing monthly downtime.

Previously, under the Uptime Graph tab you would only see the daily uptime/downtime graph for the last 30 days, which is informative but a bit limiting if you have already been monitoring your website for more than a month. Just like the daily graph, the monthly graph will show a percentage only if the website was down at any time during that month.

Stay tuned; more detailed graphs will soon come in a future full website monitoring report.

Monitoring probes visually enhanced with status icons

httpmon — Fri, 27 Aug 2010 19:14:41 +0000

All monitoring results originating from the HTTP probes of a website will now be presented with status icons to help the interpretation of these results. This currently applies to the Last Ten Probes as well as the Downtimes sections of a monitored website which contains the HTTP Code and Transfer Time columns. An example taken from the Last Ten Probes of a monitored website is shown in the image below.

Sample of last ten probes, hopefully not your website

The HTTP Code column can contain three different type of icons depending on the result of the web server: if the server responds with an OK (HTTP status code 200) then you will get a green OK icon, if the server responds with any other HTTP code there will be a yellow warning icon and if the web server is not responding you will see a red error icon with no HTTP code.

Likewise can the Transfer Time column contain three different types of icons, the same actually as the HTTP Code column but with a different meaning depending on the transfer time taken to access the main page of the website: if the transfer time is less than two seconds you will seen the green OK icon, if the transfer time is between two and five seconds there will be the yellow warning icon and if the transfer time is slower than five seconds there will be the red error icon.

With these simple but nice status icons you are now able to quickly get an overview if things are going wrong with your website without having to be an expert in order to interpret the associated results. A big thanks to Maria for the great design work she did with the icons.