outage

While we are using CloudFlare now, switching to them isn't what caused the outage.

If this is gloating about the EC2 outage, then ugh. I can't find out if it is because the whole of joyent.

A pay pal transaction not completing due to a network outage is an ops issue not a dev issue. Also, what's the pay?

Still, several hour long outage and lots of apologies to angry clients." There are two types of people in this world, those who have lost data, and those who are going to lose data"

There were a lot of things discussed here in the original post of this that were lost in the HN outage. Things that stood out to me from memory/ links and stuff/ some of my own thoughts: - OSM does not always show some data.

I can take them, I didn't ignore the last 8 problems GoDaddy has been responsible for lately and am not hurting from this outage.

Maybe it's just bad timing that I suggested we look at Amazon's new live streaming and a day later EC2 is suffering a half-day outage.

Some quotes regarding how Netflix handled this without interruptions: "Netflix showed some increased latency, internal alarms went off but hasn't had a service outage." [1] "Netflix is deployed in three zones, sized to lose one and keep going.

There's something beautiful about: * Tumbler reports its outage status using twitter. * Twitter reports its outage status using tumbler.

- A CEO who made the tradeoff of not having a dedicated 24/7 support team, and whines when the inevitable outage happens and there is no one to handle it. I believe the original author herself said it best: "So your best bet is to hire people who share your passion, willing to 'volunteer' on such occasions."

Or that there is absolutely no way to deep link to an ongoing outage, and users must reload, then expand the link every single time, or subscribe to an RSS feed. AWS needs to blantantly copy Heroku's status system, which is worlds better for people needing fast updates on their infrastructure.

What they need is a post-mortem and a transparent analysis of the situation posted on their website, similar to what Heroku did with their outage. If they can show that the reason why this occurred was because of a break in a process, and this process was fixed, then I wouldn't have any problems with Newegg.

Any serious outage or bug will be followed by a detailed post-mortem detailing a timeline of what went wrong and why. Sometimes this will go as far as establishing a "war room", possibly for weeks, dragging in people from a number of different areas to address a particular issue.

He does certain things for shock value, such as the super bowl commercials, which by the way caused outage with a completely different demographic of people. Slaughtering an elephant might offend some of your sensibilities, but hunting in Africa is wide spread and as long as the elephant is not endangered, I don't think your outrage has a leg to stand on.

We hope that makes it very robust against very "hard to fix" internet and power problems, and also API outages. So for an awkward "worst case" example; if there were a large networking outage in an intermediate transit provider your customers might not be able to reach your ELB, and likewise you may not be able to reach the API to make the changes necessary.

I'm more interested in seeing what the monitoring methodology is, what the response protocols are, and what the reported downtime has been over the last year has been, than I am about getting too concerned about the occasional outage.