Just a heads up that over the next couple of weeks, you may get some mail from me regarding mailman list cleanup. The messages are legit, there should be no embedded links, and they will merely inform you that unless you take action, the list mentioned in the subject will be retired. As a first …

A number of people are getting mails from [email protected] telling them of weak passwords and sending them to a site to reset it. It looks very legit, but it is most definitely not. It’s a phishing attack. If you visited the site in question and entered any information, let us (and [email protected]) know immediately.

The cause of our issues was an external Denial Of Service attack based around NTP (Network Time Protocol). It was targeted at a server here that used to provide that service to the outside world. We haven’t run an externally-facing NTP server in years, certainly not since we moved from 221. Blocking all access to …

First of all, the actual cause of the problem was a bad network link.  It was difficult to find, and we were just about to begin migrating the services out of the 221 data center into 240 when it was finally discovered. Here’s the link to Networking’s summary of what happened: https://slack-files.com/T025KBW9U-F025WFY2W-0c97c9 We discovered things were …

More details will come later today once we’ve hammered out what happened, but everything is back (or should be back shortly). Thanks for your patience, sorry for the troubles, and stand by for details. Love, Your humble team of tech monkeys

Still struggling to get things back up. We’re moving the database services over to a different datacenter in the hopes it will be able to start providing service there. There’s a lot of data to move. I’ll provide updates direct to twitter from this point on, and reserve e-mail/blog posts for more detailed info. If …

We’re still battling an issue we have yet to diagnose. At this point, we’re not expecting things to come back tonight. We’re working to move the services off the affected infrastructure to bring the critical services back. We’re getting outage pages into place for affected websites. Once we’re through this, I’ll provide a detailed explanation …

Our primary mysql database server is not allowing connections (or only sporadically allowing them), which is causing a number of services to fail including many websites. We’re working feverishly on getting it working again, but we have no ETA at this point since we’re still not able to get any real diagnostic info after multiple …

If you’re running MacOS 10.9 (Mavericks), please run software update to get the latest version (10.9.2). Aside from some bugfixes (including improved Exchange interopability), this contains a fix for a serious security issue with SSL certificate validation. This update was just released, and should be available via the App Store/Software Update. It can also be …