This morning’s outage (Power, power, power)

Systems Administration
Maybe not this outlet

Dumb.

Over the last year we’ve significantly upgraded the power distribution within our mini-data center.  However, as often is the case when things are upgraded, we hadn’t yet worked out all of the kinks.

Yesterday, I upgraded the Brownrice phone systems by bringing a new Asterisk server online.  Apparently I plugged this machine into a circuit that was nearly overloaded.  All of our machines perform routine maintenance between 4am and 5am each morning, when most people are sleeping.  This causes the machine’s load and power consumption to increase.

This morning, because of the added Asterisk server load, one of the data center circuits blew.  Our batteries kept things running for a bit but eventually a few servers went down and, most importantly, one of our primary switches lost power.   (If the power was cut to the building our generators would have kicked in, but the power supply was not affected.)

As we are a small hosting provider we don’t have a technician in the building over night.  (If we did this person would have nothing to do 99.99% of the time.)  Dave came in, sorted it out, and stabilized us.

We’ll be spending the day analyzing our power distribution to prevent this from happen again (yes, its happened once before) and that if it does happen again we’ll make sure that the outage will be minimized (think: switches and routers on their own circuits.)

I’m very sorry for the outage and thank you for your business.

~ Oban

A little network maintenance

Systems Administration

We’re about to perform a little network maintenance (today, February 1st, 2009 at approximately 2:45 pm MT) which will provide us with an additional layer of redundancy and potentially greater speeds to both coasts.

We don’t expect any downtime, but, well, you never can be sure with tech upgrades!

And by the way, if we expected downtime we’d do this at night.  But we don’t.  When possible we like to do this type of thing during the day when we’ve got more eyes, and better rested eyes, watching the chicken coop.

How to (un)successfully advertise on the Internet, Part 1

Business Development, SEO, Web Development

This is Part 1 of  a series by Oban Lambie.  Part II is here.

I’ve done some hard time on the ‘Net.  For years, stretching into decades, I’ve coded sites, run servers, and held the hands of customers both big and small to get my little business to where it is now; a hosting and programming shop in Taos, New Mexico.  I love the work, my staff is great, our customers are loyal and happy, and though we’re not getting rich, it pays the bills.

Now I’m going to try to do three things, none of which I’ve successfully done before:

  1. Launch a new product on the ‘Net.
  2. Find some people that have never heard of me, or my business, and persuade them to buy the product online.
  3. Show you exactly how I do it.

The Promise

I’ll expose everything, pants on the ground style, while I do it  – from the under-the-hood technology, to the how-we-target-Keywords techniques, to the Search Engine Optimization tricks that we’ll employ and finally through a wee bit of email marketing.  I’ll also show you what tools I use to track my successes and failures and detail exactly how much money we make, or lose, in the process.

But first…
Continue reading How to (un)successfully advertise on the Internet, Part 1 →

Can’t login to Squirrelmail!

Systems Administration, Web Hosting, , ,

Squirrelmail, aka “The old webmail,” has gone wonkey on us as we migrate it to more robust servers and upgrade it.  i.e.  Its online but its broke.  So don’t use it.

For the four of you who actually use “The old webmail” to read your email, I apologize.  In the meantime you can use the real webmail, here.

We’ll update this blog, and our twitter account, when its fully functioning again.

Better uptime than Google, Rackspace, and Amazon?

Systems Administration, Web Hosting
Smooth Flying
Better than Air Kazakstan!

Yep.

In 2009 Brownrice provided our hosting, co-location, and virtual server customers with 100% network up-time for the entire year – soundly besting the volume-based hosting companies.

OK, full disclosure: Our network was down for a total of 3 minutes and 11 seconds for the calendar year.  I think that was when Dave rebooted one of our main switches.

How did we fair against the big boys?

Google had multiple network outages.  Amazon’s cloud went down.  Rackspace had multiple extended network outages which brought down many of the biggest sites on the internet.  Many hosting providers who co-locate in the Rackspace data centers suffered from their outages as well. And Dreamhost – who advertises a goofy 100% up-time guarantee (read the fine print, it’s funny) – seemed to have nearly daily network failures.

Why is our up-time so good?  Because our network is quadruple-redundant, impervious to fiber cuts, and because we are small.

Why is being a small provider good?  Because we run less than 100 servers – instead of hundres of thousands – most of which are in our own, on-site mini data center, where the size of our operation allows us isolate and fix problems extremely quickly.

I’ve always known that our staff is smarter, more experienced, and provide better – and friendlier – support than the volume-based hosting companies.  Now the raw data is proving that we are more reliable as well.