Fubra

First Mac Mini BGP routers on world's largest Internet exchange

This blog post what written on Thursday 26th April 2007 by Paul Maunders who has been working for Fubra as a Director and Co-Founder for 11 years. Our staff are encouraged to blog on this site, but the views expressed are individual and do not necessarily represent those of Fubra.

We think this could be a world’s first: Forget Cisco or Juniper, yesterday we hooked up two Mac Mini boxes running Quagga in to the London INternet eXchange (LINX) to act as BGP border routers for the Fubra Network.

LINX handles 95% of total UK Internet traffic, and as their newest members we had to come up with a clever solution to keep costs low and speeds high. Utilitising just 3U of rackspace, we were able to install 2 low latency HP gigabit switches and a pair of 1.83 GHz Mac Minis with 2 GB Ram, giving us fully redundant connections to the largest Internet Exchange Point in the world.

Fulfilling our environmental obligations, the total power draw of this setup is less than 2 ordinary household lightbulbs (< 120W).

Our engineers, Nigel Marett and Mark Sutton, who installed the kit said the contrast between our tiny Mac Minis and the existing telco router kit that other people were using was striking.

Nigel commented, “It is kinda a strange sight mate: you walk into that room, and there is a Juniper m120 and another even larger (half rack) router, one of them has an OC48 (STM-15) and three gbit fibres coming out of it, the other one a whole bunch of fibre, and then you get to our rack and there’s two mini’s!”

Background

Recently we have been re-designing our core hosting network and IP connectivity, and as part of this overhaul we decided to join LINX. For those of you who don’t know, an Internet Exchange is where a bunch of ISPs and content providers get together to swap traffic. This saves money and improves network performance as you don’t have to use a 3rd party transit provider to carry data on your behalf.

Since there is a shortage of power in most London data centres, and space is at a premium, our solution would have to work within tight constraints. After searching high and low for suitable dedicated hardware routers, it was clear that Cisco, Juniper and Extreme Networks’ offerings were all too big and power hungry for the job. Someone said “what about a Mac Mini running Quagga?” and the rest, as they say, is history.

Currently our total traffic is at 42Mbps, and we hope that Mac Minis will cope with anything up to several hundred megabits per second on their gigabit interfaces.

Setup

If you want to make your own iRouter setup, you will need the following:

• 2 x HP Procurve 1800 24G Switches
• 2 x 1.83GHz Intel Core Duo Mac Minis (with 2Gb Memory upgrade)
• Ubuntu 7.04 Feisty Fawn (Server Edition) + OpenSSH + Quagga
• 2 x 100Mbps connections to LINX (eXtreme LAN and Foundry LAN) or similar

Fixes

If you plan on using a Mac Mini as a server with Ubuntu 7.04, you need to add the following to the end of your rc.local

setpci -d 8086:27b9 0xa4.b=0

This will fix the power restore status, by telling the EFI not to reset the power flag on reboot. The machine will then auto-power on after a power cut.

Tags: Apple, BGP, LINX, Mac, Mac Mini

Network • Servers | RSS Feed | Trackback