Rapid increase in IPv6 penetration in non-US regions

Posted by Pieter Ennes on November 18th, 2009

Today, we received the final results from the second IPv6 questionnaire we sent out to our hosting partners.
While the results of the first poll, which was held back in 2007, revealed a total lack of interest in IPv6 at the time, the new results (two years later) show something different.

To summarize, almost 40% of our current partners offer native IPv6 already, or expect to be able to do so within the next six months. The table below shows a breakdown for different regions in the world:

Region# hostersIPv6 penetration
North-America714%
Europe1850%
Asia & Australia743%
Rest of world20%
Total3438%

Even though our statistics are coarse, the results seem to indicate a lack of interest in the North-American area. But they also show that in both Europe as well as in Asia and Australia, penetration is much better. Almost 50% of the parties in Europe can feed us with native IPv6 down the wire within six months. Go go go!

Are you a hosting provider offering quality IPv6-enabled hosting services in an area that is not currently covered by our network? Please do send us a note!

UPDATE: We’re very pleased to hear that WatchMouse were selected as one of the finalists for the Dutch IPv6 awards today!

Bookmark and Share

Extending MySQL 5 with IPv6 functions

Posted by Pieter Ennes on October 19th, 2009

Update: Post updated for v0.2…

Hi folks,

While our own migration to IPv6-enabled monitoring is progressing nicely, we are making available our version of some rudimentary MySQL User Defined Functions (UDF’s) that allow you to work with IPv6 addresses in pre-6.0 versions of MySQL.

The currently implemented functions are inet6_pton() and inet6_ntop() to convert IPv6 (and IPv4) addresses between presentation and numeric (or binary) form directly in SQL. The former function converts a readable IP address string to a binary representation of either 4 bytes (IPv4) or 16 bytes (IPv6) long. The latter function does exactly the opposite, as shown in:

mysql> select inet6_ntop(inet6_pton('2001:4860:a005::68'));
+----------------------------------------------+
| inet6_ntop(inet6_pton('2001:4860:a005::68')) |
+----------------------------------------------+
| 2001:4860:a005::68                           |
+----------------------------------------------+
1 row in set (0.00 sec)

mysql> select inet6_ntop(inet6_pton('1.2.3.4'));
+-----------------------------------+
| inet6_ntop(inet6_pton('1.2.3.4')) |
+-----------------------------------+
| 1.2.3.4                           |
+-----------------------------------+
1 row in set (0.00 sec)

mysql> select
length(inet6_pton('1.2.3.4')) as ipv4len,
length(inet6_pton('2001:4860:a005::68')) as ipv6len;
+---------+---------+
| ipv4len | ipv6len |
+---------+---------+
|       4 |      16 |
+---------+---------+
1 row in set (0.00 sec)

You can neatly store the output of inet6_pton() in a VARBINARY(16) column (using a VARCHAR(16) would not be wise because because of possible character set conversion problems).

Of course, there are many ways one could store IPv6 addresses in a database, but the method described here seems to work best for us and is unlikely to conflict with any native support for IPv6 that MySQL will implement in future versions. Furthermore, the binary notation seems to work nicely with ranges, so you can use WHERE ip BETWEEN a AND b to do something like:

mysql> select * from log
where ip between inet6_pton('2001:db8::') and inet6_pton('2001:db9::');

to get all IP addresses in the 2001:db8::/32 range.

We’ve also included two address lookup functions that are IPv6 compatible:

mysql> select inet6_lookup('www.watchmouse.com');
+------------------------------------+
| inet6_lookup('www.watchmouse.com') |
+------------------------------------+
| 2001:1938:80:73::2 |
+------------------------------------+
1 row in set (2.08 sec)

mysql> select inet6_rlookup('2a02:f8::ffff:ffe5'), inet6_rlookup('64.128.190.61');
+-------------------------------------+--------------------------------+
| inet6_rlookup('2a02:f8::ffff:ffe5') | inet6_rlookup('64.128.190.61') |
+-------------------------------------+--------------------------------+
| it.watchmouse.com | ny.watchmouse.com |
+-------------------------------------+--------------------------------+
1 row in set (0.07 sec)

As you can see, inet6_lookup takes a host name and converts it to a readable IP address. The reverse lookup function, inet6_rlookup, works on both binary as well as human readable IP addresses (since v0.2).

The inet6_pton() and inet6_ntop() functions should be quite fast, but as each call to inet6_lookup() involves a DNS lookup, the lookup functions can make your queries extremely slow. Make sure you LIMIT your result set to something sensible. In general, I would not recommend to use the lookup functions in production code as I don’t know the impact of these queries on MySQL’s performance.

Once the UDF’s are installed in the MySQL plugin directory, they can be loaded at runtime using:

mysql> CREATE FUNCTION inet6_ntop RETURNS STRING SONAME "mysql_udf_ipv6.so";
mysql> CREATE FUNCTION inet6_pton RETURNS STRING SONAME "mysql_udf_ipv6.so";
mysql> CREATE FUNCTION inet6_lookup RETURNS STRING SONAME "mysql_udf_ipv6.so";
mysql> CREATE FUNCTION inet6_rlookup RETURNS STRING SONAME "mysql_udf_ipv6.so";

These functions have helped us with our immediate needs for managing IPv6 addresses in our DBMS, and hopefully they can help you too. We have some more improvements in mind, so stay tuned if you’re interested!

The current version of the UDF’s can be downloaded from:

mysql-udf-ipv6-0.2.tar.gz

Note: Compilation was tested on Debian etch+lenny, Ubuntu jaunty+karmic, and on FreeBSD. If you get it running on other platforms, send us a patch!

Pieter Ennes
WatchMouse

Bookmark and Share

WatchMouse API 1.6 released. Now with IPv6 and UTF-8.

Posted by stan on October 13th, 2009

Earlier today we promoted the 1.6 version of the WatchMouse API to ’stable’. Next to some minor fixes, the new API now fully supports IPv6 and UTF-8.

The WatchMouse API enables developers to tie in features of the WatchMouse monitoring service in their own application or website. Using the API you can access the data and functions of the WatchMouse services.

Some example uses of the API:

  • Perform a check of your site from one of the worldwide monitoring stations of WatchMouse
  • Access your WatchMouse account and set up, activate, modify, or remove site monitoring rules
  • Schedule a Vulnerability Scan for your server
  • Send an SMS from one of the many SMS gateways of WatchMouse
  • Perform a traceroute from one of these stations to an IP address of your choice. The just-trace site was build using these services.
  • Find where a certain host is located (IP to location)
  • Compute how much 10 Euro is in US Dollars.

IPv6 support

IPv6 is an important feature in this release, and it actually took more than just the API to fully support it. We upgraded monitoring stations, the core monitoring software, our website, and the databases in our back-end systems to make it all work. And let’s not forget the hosting providers for our 40+ stations world wide. But that’s a story worthy of a full post coming soon :)

For most API calls you will need a WatchMouse account (or an account with one of our partners) and the number of calls is rate-limited on hourly or daily basis, but some calls can be used anonymously too. See the Access to the API chapter for details.

Your mashup here

Already a number of mash-ups were created, like the Just-ping site and the Down-or-Not site, and we will see a number of other applications down the road. If you use our API, just let us know in a comment below, and don’t forget to list it as a mash-up at Programmable web!

For documentation, please read on here:

Hope to see a thousand applications bloom!

Stan P. van de Burgt

Bookmark and Share

Filed under API, IPv6, Monitoring Tags: , , , , No Comments

Investigating IPv6 website monitoring

Posted by Pieter Ennes on September 8th, 2009

Which hosters are running IPv6, and what monitoring tools are needed? We are trying to find out!

With a bit of a delay we want to share the results of a poll we held among the hosting providers of our monitoring network in the beginning of 2008. We asked them whether they could offer native IPv6 to our stations, and if not, when they could.

The results were interesting, as none of them could at that time deliver native v6 traffic right down to our stations, and only two were up for a joint experiment with us. Very few responded that they had IPv6 on the roadmap at all; the rest indicating that they had not thought about it yet. Two responded that they were in fact waiting on some IPv6 address space from APNIC. Assuming that our selection of (at that time 25) providers is representative for hosters in general, pioneers are still in bad shape if they depend on upstream network infrastructure.

So what has changed since then? The AMS-IX IPv6 bandwidth charts for 2009 show that there has only been a minor increase in v6 traffic recently. You can also see that IPv6 traffic still accounts for only a tiny 0.2% of the total traffic volume. We will conduct a second poll now to see if matters have changed, although the above statistics are pessimistic. I will update this blog with a more detailed comparison study when we have new results.

With respect to WatchMouse, we think early adopters should have at least access to tools to monitor all the aspects of their systems during their transition to IPv6. We therefore are currently aggressively making our own infrastructure IPv6-ready to offer IPv6 monitoring as good as we can. For locations where we find no support from our upstream providers, we will be using v4 tunnels through HE, SixXS and other providers. Besides that, our developers are currently testing the necessary code changes for full IPv6 deployment.

Some of the things we are working on right now:

  • Finalizing deployment of a IPv6 capable monitoring network using tunnel brokers
  • Make our core systems, databases and checker software ready for IPv6
  • Conduct a second poll amongst our hosters to verify the current state of v6 penetration
  • Make the WatchMouse website itself available on v6

We will post any interesting information regarding our progress in this blog when we have it. In the meantime, we would like to ask what kind of tools you would need from us to be able to monitor your own transition; why not leave a comment on this site!

Bookmark and Share

Latest experiments

Categories