Outage issues resolved
Quick summary: I just got back from the 221 data center (gee, it’s hot outside) having replaced what we suspect are bad power supplies in a Virtual Machine Host server. We isolated the issue to this specific server rebooting without offering any useful information as to why in its logs, coupled with a bad set of configs that prevented the virtual machines hosted on it from restarting without human intervention.
We’ve addressed the config issue, and replaced the power supplies as there were indications that one or possibly both were bad.
We’re ready to migrate these affected virtual machines to a new host if this last fix doesn’t stabilize things, but we’re feeling pretty good about this at the moment.
Thanks so much for your patience, and sorry for any troubles.