Server6 Issues

[Varun Shoor]

Hi,
Server6 has been down for almost a day now and this has affected about 60 hosted clients. We have been receiving confusing responses from our dedicated server provider (GNAX) on what the exact issue is.

It was claimed that the problem was due to a dying hard disk for which we got them to mount the drive as secondary and install another primary one so we can salvage the data out of it, However, We received an update from them on 8/22 12:33 PM IST saying that the problem might be due to some other hardware.

-- Quote --
It appears there is a hardware issue with the server. It was taken down for the OS reload but does not boot up with the cd rom plugged in. We are working on getting it fixed and will keep you updated.
-- Quote --

We are working to resolve this issue as soon as possible and I will be posting another update as soon as I receive one.

Thanks.

Regards,

Varun Shoor

[Ryan Lederman]

Hi,

I'm afraid I have some bad news. The data from server 6 could not be recovered from the hard disk. Right now we have two point of failures for all of our hosted platforms:

1) A Daily Backup onto the same HDD
2) Weekly Backup to an offsite location

The problem we now have is:

1) This data is gone with the HDD crash
2) Apparently due to a script error only the PHP files were being backed up to the offsite location. So what we now have is quite useless.

This has been a big blow for us and the overhaul of the entire system is now our top priority. We have already added an option in Members Area to allow clients to generate backup for hosted solutions without coming to us at any point of time and have placed an order for a 2.4 Terabyte server which will archive the data across our entire cluster on a daily basis. We are planning to send in the hard disk for recovery but from what I have heard the chances of recovery are quite slim.

We know that nothing can replace the invaluable data that has been lost, and as a client ordering a hosted solution you expect every aspect of data and application to be handled by us. As of now we have re-added all domains with new access information. As a gesture of good will, we have upgraded all clients from eSupport to SupportSuite free of charge. We are also prepared to offer free InstaAlert and SyncWorks add-on licenses to all affected clients. If you have been affected by the server 6 crash and would like to get your free add-on licenses, please e-mail support@kayako.com and include the following information:

- The e-mail address associted with your Kayako account
- Information about how many of each license you will require, and the Device IDs for each computer that you wish to install the add-ons on. The Device ID of each computer can be found by installing either InstaAlert or SyncWorks and choosing the Register option.

I will be posting an update again on this shortly to see if there is an update from the data center regarding the recovery of data from the HDD.

On behalf of everyone at Kayako, please accept my sincerest apologies for these awful circumstances.

Regards,