Administrator
Posts: 17,190
Threads: 2,153
Joined: 2008-09
Gender: Male
Sexual Orientation: Gay
IGN: Aesilyn
Server: Mardia
Level: 200
Job: I/L ArchMage
Guild: Animus
Host Wrote:Dear clients:
Please note that we experienced a network outage starting Monday, March 12, 2012 at 10:30 p.m. EST and lasting until Tuesday, March 13, 2012 at 3:00 a.m. EST. This outage affected all of our clients and services, including our own websites.
At this point in time, our Operations team is working with our Cisco and Dimension Data partners to find the root cause of this outage in order to avoid such instances in the future.
It's not having what you want - It's wanting what you've got.
Administrator
Posts: 17,190
Threads: 2,153
Joined: 2008-09
Gender: Male
Sexual Orientation: Gay
IGN: Aesilyn
Server: Mardia
Level: 200
Job: I/L ArchMage
Guild: Animus
Quote:Dear clients:
We would like to follow up with you regarding the recent network outage which we experienced across the entirety of our services and provide a post-mortem technical report for the causes of the outage.
Currently, we have two (2) physical servers that act as a firewall for the protection of our network. These servers act as a failover for each other. Should one of the units fail, the other will pick up the traffic to avoid any interruption to service.
Last night, one of the units did fail, and the other picked up the traffic as intended. The second unit, that had all of the traffic at this time, then also failed. There was no ability to pass the traffic back to the first unit in its failed state.
We worked to restore all traffic with in-house staff, having our Director of Operations present at our Toronto data center for troubleshooting. After initial tests could not determine the root cause of the server failures, a priority ticket was opened with Cisco, one of our hardware vendors.
After some trial and error, the cause was not found and an executive decision was made to force a restart of both physical units. This restart did restore network connectivity, but wiped out log files needed to determine the root cause of the failures. Cisco is attempting to recover any information to pinpoint the root cause, which we will be happy to update you with if it is located.
It's not having what you want - It's wanting what you've got.
Senior Member
Posts: 569
Threads: 15
Joined: 2008-07
Gender: Male
Sexual Orientation: Gay
Country Flag: canada
IGN: Scenarey
Server: Arcania
Level: 146
Job: Mercedes
Guild: Destiny
Guild Alliance: Providence
Now if only we had this kind of communication coming from NexAm...
