Weekend outage affects wide range of services
Monday, Nov. 9, 2009
On the morning of Sunday, Nov. 8, a controller on one of Duke's storage arrays failed. These controllers are redundant; if one goes off line, the array can continue to function without it. Because of that redundancy, the failure of the controller did not impact service immediately.
However, when our vendor, Hewlett-Packard (HP), began to replace the controller at about 4:15 p.m., the entire array went down for a brief time. This resulted in a complete loss of services hosted on that array, including many of Duke's web sites, personal web space, dCal, Lectopia lecture capture, the Duke Libraries AdViews services, iTunesU, Adobe Connect and DukeStream. The array was back online at 4:23 p.m., but affected services had to be brought back online individually. Service was restored in full by 9 p.m. Sunday.
At 1:15 a.m. Monday, Nov. 9, the array was taken down so that the failed controller could be replaced. Services were restored in full at 3:42 a.m.
We apologize for the inconvenience caused by this outage.