top 9 Cloud Computing Failures

On August 3, 2009, PayPal‘s online payment service suffered a global outage for an hour, and after that, the service suffered partial outages for another three and a half hours. At the time, a company spokesperson said that the cloud-based service was processing an average of $2,000 in payments every second. If you do that math, that comes out to $7.2 million in payments per hour. Over four and a half hours, the total number of transactions affected could have been as high as $32 million.

Nirvanix was once one of the earliest pioneers of cloud storage. Originally called Streamload, the company was founded in 1998 as an Internet storage service. In 2007, the renamed company launched a product called the Storage Delivery Network, which included public, private and hybrid cloud storage capabilities. In 2011, it signed an important agreement with IBM, which saw IBM using Nirvanix technology for its own cloud storage service. Based on its promising technology, Nirvanix was able to raise $70 million in venture capital.

But in June 2013, IBM went another direction, announcing the purchase of SoftLayer and the formation of IBM Cloud Services Division. A few months later in September of that year, Nirvanix notified customers that they had just two weeks to retrieve their data before the Nirvanix cloud storage service would shut down permanently. The company filed for bankruptcy on October 1, 2013.

The Supplemental Nutrition Assistance Program (SNAP), which is known colloquially as "food stamps," uses Electronic Benefit Transfer (EBT) cards to allow recipients to purchase food using their government benefits. In 2013, Xerox was hosting the EBT systems for 17 states in its data centers.

On October 12, 2013, Xerox was conducting routine tests of its backup systems when a glitch caused the entire EBT system to go offline. For several hours on a busy Saturday, retailers had no way to determine the balances that shoppers had available on their EBT cards. As a result, some stores simply stopped accepting EBT cards, leading to some very unhappy customers. In other cases, the stores allowed benefits recipients to use their EBT cards anyway, even though the store had no way to know how much money the customers had left to spend. In some locations, shoppers took advantage of the situation, loading their carts with thousands of dollars‘ worth of food. Police were called in several states to quell "mini-riots," and the government later charged some of those shoppers with fraud.

Financial software vendor Intuit is known for popular cloud-based software products like Quicken, Quickbooks and TurboTax. But in early June 2010, the company‘s cloud computing services and its corporate website experienced an outage that lasted about 36 hours. During that time, small business owners and consumers were unable to access their financial records. The cause? A power outage knocked out both the company‘s primary cloud data center and its backup site. Less than a month later, Intuit experienced another outage, further shaking customer confidence in its cloud services.

Late in the evening on the West Coast on May 9, 2016, Salesforce.com‘s NA14 instance began to experience disruption due to a power outage at one of its data centers. The company quickly moved affected workloads to one of its other cloud data centers and restored service. But the next morning, NA14 went down again, and customers could not access their Salesforce accounts for nearly an entire day. Even worse, the company lost about five hours‘ worth of data for some customers. The company‘s official explanation said, "The service disruption was caused by a database failure on the NA14 instance, which introduced a file integrity issue in the NA14 database. The issue was resolved by restoring NA14 from a prior backup, which was not impacted by the file integrity issues. We have determined that data written to the NA14 instance between 9:53 UTC and 14:53 on May 10, 2016 could not be restored."

In late October 2012, the largest-ever Atlantic hurricane by diameter made landfall on the East Coast of the United States. Twenty-four states suffered damage, with New York and New Jersey getting the worst of it. Massive power outages ensued, impacting many cloud computing data centers. Although most data centers switched to generator power, some ran out of fuel, and others had to shut down due to flooding. Several popular websites, including Gawker, Huffington Post and BuzzFeed went offline. Total damage from the storm is estimated around $75 billion.

On August 1, 2012, Knight Capital, a financial services firm that was using cloud-based high-frequency trading software, deployed a new software algorithm. Unfortunately, the algorithm did not function as intended and began making money-losing trades up to 40 times per second. Those transactions caused the share price for 148 different stocks to drop dramatically, prompting the New York Stock Exchange to step in and temporarily halt trading. Before the algorithm was stopped, Knight Capital had lost $440 million, and within days, the company‘s stock had lost 75 percent of its value.

On October 1, 2013, the U.S. federal government rolled out HealthCare.gov, a new website intended to allow people to sign up to buy health insurance under the Patient Protection and Affordable Care Act, often called Obamacare. Almost immediately, users began experiencing difficulties, and some reports indicated that less than 1 percent of the people who wanted to sign up online were able to do so. The project also ran far over-budget, racking up more than $1.7 billion in costs, up from an original budget of just $93.7 million.

Many observers say that the government could have avoided these problems if it had used a well-known cloud computing vendor instead of trying to build its infrastructure on top of legacy equipment. They also faulted the developers for inadequate testing and a lack of oversight and accountability.