GuaranteedVPS - 1 (888) 466-2132

As of 2:22p PST Nov 30, all problems from the datacenter outage are resolved. Downtime varies depending on which server the customer was on; all servers except DoomBox experienced less than two hours’ outage, whereas DoomBox experienced an outage for some protocols for a further hour and one half. All VEs have been manually verified as running and routable. No data is lost. No hardware is damaged. All known results of downtime are resolved. Everything is once again hunky dory. If this outage has affected you as a customer, please let sales@ know. If you find that the outage has caused even minor issues and you would like assistance smoothing them out, please let us know.

We would like to thank our customers for their graciousness during this stress. Some hosts would brag about four hours of total downtime in eight months; we would not. Measures are being taken to harden the rack against such problems in the future.

Our lifetime uptime is now 99.9305%. Bye bye, 100%; we’ll miss you.

We are aware that there is still a problem on DoomBox which is affecting < 4% of our customers. We are making the necessary repairs immediately and will have the http standard port (80) up and running shortly (there currently is a route to host but a switching problem is preventing http requests from going through). We expect this problem will be resolved by close of business today.

Edit: this problem is resolved as of 2:14p PST same day (Nov 30 2007).

GuaranteedVPS was created because we were tired of the guesswork, vaguery, and misrepresentation that we constantly ran into with virtually every other hosting provider we encountered.  Our goal was, and continues to be, to create an environment where our customers get exactly what they expect, with nothing hidden, no overages, and no calculator required to figure out things that shouldn’t require such. 

With this said, there are certain times when Irony will reign supreme.  As you’ve probably read from our explanation post, we had quite the circus of events today.  I would like to reiterate that the incidents of today (which lasted 1 hour and 58 minutes) were the result of uncontrollable events which occurred when our upstream provider attempted to cooperate with us to ensure that the best possible setup was achieved and that a situation like today’s was virtually impossible.  Naturally, all the problems occurred when we downed our redundant systems to move them into their new homes. 

I would like to personally reassure our loyal GuaranteedVPS customers that we do have redundant systems in place.  We wholeheartedly believe, subscribe, and insist from our upstream providers that there be an excess of redundant systems, whether they be power supplies, routers/switches, or internet pipelines.  Please rest assured that under normal conditions very few things short of a veritable nuclear holocaust would disable our systems as they were so disabled today. 

I also understand that, while you all have explanations and comprehend that we did not create this problem, that it is still our problem to deal with.  Today, someone else’s problem became your problem, and our dedication to our customers makes this our problem.  As Director of Operations and Market Manager for GuaranteedVPS, I would like to encourage you to bring your problems that were caused by the incidents today, to me.  If there’s anything that I can do to fix, whether preventively, retroactively, or directly, please don’t hesitate to call me or write me an email (my extension is 801 and my email username is simply josh at the guaranteedvps.com domain). 

Please understand that we never expect to go through such an ordeal again, especially since we were attempting to have our setup such that it was even more bulletproof than before.  While I can’t predict the future, I can tell you that we are even more immune to such incidents than we were before.  You have my word on that, and our word is only one of the guarantees we give you at GuaranteedVPS.  As I said before, if there’s any result of this outage that I can help you deal with, my door is open to you. 

Have a safe and happy holiday season.

On Friday 30 November 2007, an unexpected carryover from yesterday’s rack transition incident caused a major outage for all of our servers. A neighboring rack blew a power coupling while two of the three redundant fiber switches were being relocated. This damage to the power coupling also damaged the remaining fiber switch. The switch was patched as quickly as possible. Ironically, it was the case that the relocation of the fiber switches was being done in an effort to put them on their own coupling to eliminate any chance of what actually happened.

While our servers actually remained on and up, there was no route to your hosts between 8:36AM and 10:34AM PST. The situation was resolved at 10:34AM PST today, and a more permanent solution will be in place by 12:01AM tomorrow, Saturday 01 December. Due to this permanent solution, there will be another downtime which is expected to last less than 5 minutes, and no more than 15 on the outside, at approximately just before midnight tonight.

You may have noticed that your servers have, in fact, reported a downtime. We used the debacles of late to go ahead and migrate the servers to their new and better homes since there was no route to your hosts from the public internet anyway. This allowed us to be more efficient within the parameters of the outage and prevented us from having to reschedule the downtime to relocate the machines later.

This issue is an isolated incident, and was the result of our demand of our upstream provider to make sure that an incident like this never actually happens. If you have any questions or wish to discuss further any of the issues, please do not hesitate to open a support ticket by either calling us directly or via email.

All four servers went down due to a rack transition from 4:57pm to 5:41..5:45p, Pacific time, varying per server. No server was down for in excess of 48 minutes. All servers are up and running. There was no loss of hardware, and all drives are intact. We apologize for the inconvenience. Please contact us if things are not as expected.