Carrie Fernandez, the social media company’s vice president of engineering, wrote in an internal message to Twitter engineers on Friday, “On September 5th, Twitter experienced the loss of its Sacramento (SMF) data center region due to extreme weather. The unprecedented event resulted in the total shutdown of physical equipment in SMF.”
Sacramento reached a dangerous level of heat at 113 degrees Fahrenheit on Monday, breaking the previous day’s heat record, 108 degrees. In addition, a leaked company email revealed another Twitter data center in California failed.
However, the server outage has not affected the users but made Twitter exposed as it took the backup data storage offline. In addition, this episode creates panic in tech companies due to climate change.
Servers must remain calm to function correctly, but temperature control is costly, mainly if they are located in places like California’s Central valley. Similarly, servers of both Oracle and Google went down in the UK during a heatwave earlier this year. In July, crypto mining operations had to be reduced due to stressed power grids and heatwaves.
The West faced extreme heat over the last week, with a triple-digit temperature all over California. The heatwave taxed the electric grid and escaped a narrow blackout by the texts sent to people on reducing energy usage.
According to CNN, Fernandez wrote in the staff memo, “But while the electricity stayed on, Twitter’s servers didn’t. The outage put Twitter in a non-redundant state.”
Twitter has centers in multiple locations, including Atlanta, Oregon, Portland, and Georgia, which store duplicate data to avoid data loss and site outages. However, Fernandez warned, “if we lose one of those remaining data centers, we may not be able to serve traffic to all of Twitter’s users.”
Peiter Mudge Zatko, a programmer, in his whistleblower complaint, referenced the vulnerability and “insufficient data center redundancy” of Twitter’s system.
CNN says, “Even a temporary but overlapping outage of a small number of data centers would likely result in the service [Twitter] going offline for weeks, months, or permanently,” according to Zatko’s whistleblower disclosure. (Twitter has criticized Zatko and broadly defended itself against the allegations, saying the revelation paints a “false narrative” of the company.)
Following the failure that occurred on Monday in Sacramento and Friday, the memo was sent to the staff, and five days of server disruptions occurred. It is unknown whether the data center is back online or the outage is still on.