RESOLVED: Outage affecting MCS/CELS systems.
The aforementioned outage is resolved. A disk problem took out our virtual server master around 4 AM, which took out a handful of virtual servers. In trying to resolve that, at around 8 AM we ended up taking out the rest of the virtual machines hosted in that cluster due to dependencies. We’ve fixed the underlying problem and are going to implement some improvements in our monitoring to catch when this happens again. Sorry for the inconvenience. Please report any issues to [email protected].