Cloudbleed was a Cloudflare buffer overflow disclosed by Project Zero on February 17, 2017. Cloudflare's code disclosed the contents of memory that contained the private information of other customers, such as HTTP cookies, authentication tokens, HTTP POST bodies, and other sensitive data.[1] As a result, data from Cloudflare customers was leaked to all other Cloudflare customers that had access to server memory. This occurred, according to numbers provided by Cloudflare at the time, more than 18,000,000 times before the problem was corrected.[2][3] Some of the leaked data was cached by search engines.[3][4][5][6][7][8]

Discovery

edit

The discovery was reported by Google's Project Zero team.[1] Tavis Ormandy[9] posted the issue on his team's issue tracker and said that he informed Cloudflare of the problem on February 17. In his own proof-of-concept attack he got a Cloudflare server to return "private messages from major dating sites, full messages from a well-known chat service, online password manager data, frames from adult video sites, hotel bookings. We're talking full https requests, client IP addresses, full responses, cookies, passwords, keys, data, everything."[1]

Similarities to Heartbleed

edit

In its effects, Cloudbleed is comparable to the 2014 Heartbleed bug, in that it allowed unauthorized third parties to access data in the memory of programs running on web servers, including data which had been shielded while in transit by TLS.[10][11] Cloudbleed also likely impacted as many users as Heartbleed since it affected a content delivery network serving nearly two million websites.[4][11]

Tavis Ormandy, first to discover the vulnerability, immediately drew a comparison to Heartbleed, saying "it took every ounce of strength not to call this issue 'cloudbleed'" in his report.[1]

Reactions

edit

Cloudflare

edit

On Thursday, February 23, 2017, Cloudflare wrote a post noting that:[12]

The bug was serious because the leaked memory could contain private information and because it had been cached by search engines. We have also not discovered any evidence of malicious exploits of the bug or other reports of its existence.
The greatest period of impact was from February 13 and February 18 with around 1 in every 3,300,000 HTTP requests through Cloudflare potentially resulting in memory leakage (that’s about 0.00003% of requests).

Cloudflare acknowledged that the memory could have leaked as early as September 22, 2016. The company also stated that one of its own private keys, used for machine-to-machine encryption, has leaked.

It turned out that the underlying bug that caused the memory leak had been present in our Ragel-based parser for many years but no memory was leaked because of the way the internal NGINX buffers were used. Introducing cf-html subtly changed the buffering which enabled the leakage even though there were no problems in cf-html itself.[3]

John Graham-Cumming, Cloudflare CTO, noted that Cloudflare clients, such as Uber and OkCupid, weren't directly informed of the leaks due to the security risks involved in the situation. “There was no backdoor communication outside of Cloudflare — only with Google and other search engines,” he said.[6]

Graham-Cumming also said that "Unfortunately, it was the ancient piece of software that contained a latent security problem and that problem only showed up as we were in the process of migrating away from it." He added that his team has already begun testing their software for other possible issues.[7]

Google Project Zero team

edit

Tavis Ormandy initially stated that he was "really impressed with Cloudflare's quick response, and how dedicated they are to cleaning up from this unfortunate issue."[1] However, when Ormandy pressed Cloudflare for additional information, "They gave several excuses that didn't make sense,"[13] before sending a draft that "severely downplays the risk to customers."[14]

Uber

edit

Uber stated that the impact on its service was very limited.[10] An Uber spokesperson added "only a handful of session tokens were involved and have since been changed. Passwords were not exposed."[15]

OKCupid

edit

OKCupid CEO Elie Seidman said: "CloudFlare alerted us last night of their bug and we've been looking into its impact on OkCupid members. Our initial investigation has revealed minimal, if any, exposure. If we determine that any of our users has been impacted we will promptly notify them and take action to protect them."[10][15]

Fitbit

edit

Fitbit stated that they had investigated the incident and only found that a "handful of people were affected". They recommended that concerned customers should change their passwords and clear session tokens by revoking and re-adding the app to their account.[16]

1Password

edit

In a blog post, Jeffery Goldberg stated that no data from 1Password would be at risk due to Cloudbleed, citing the service's use of Secure Remote Password protocol (SRP), in which the client and server prove their identity without sharing any secrets over the network. 1Password data is additionally encrypted using keys derived from the user's master password and a secret account code, which Goldberg claims would protect the credentials even if 1Password's own servers were breached. 1Password did not suggest users change their master password in response to a potential breach involving the bug.[17]

Remediation

edit

Many major news outlets advised users of sites hosted by Cloudflare to change their passwords, as even accounts protected by multi-factor authentication could be at risk.[18][19][20][7][21] Passwords of mobile apps too could have been impacted.[22] Researchers at Arbor Networks, in an alert, suggested that "For most of us, the only truly safe response to this large-scale information leak is to update our passwords for the Web sites and app-related services we use every day...Pretty much all of them."[23]

Inc. Magazine cybersecurity columnist, Joseph Steinberg, however, advised people not to change their passwords, stating that "the current risk is much smaller than the price to be paid in increased 'cybersecurity fatigue' leading to much bigger problems in the future."[24]

References

edit
  1. ^ a b c d e "Issue 1139: cloudflare: Cloudflare Reverse Proxies are Dumping Uninitialized Memory". February 19, 2017. Retrieved February 24, 2017.
  2. ^ "About Cloudflare". Cloudflare. Archived from the original on March 4, 2017. Retrieved June 16, 2021. Every week, the average Internet user touches us more than 500 times.
  3. ^ a b c "Incident report on memory leak caused by Cloudflare parser bug". Cloudflare. February 23, 2017. Archived from the original on February 23, 2017. Retrieved February 24, 2017. 1 in every 3,300,000 HTTP requests through Cloudflare potentially resulted in memory leakage.
  4. ^ a b Thomson, Iain (February 24, 2017). "Cloudbleed: Big web brands leaked crypto keys, personal secrets thanks to Cloudflare bug". The Register. Retrieved February 24, 2017.
  5. ^ Burgess, Matt. "Cloudflare has been leaking private Uber, Fitbit and Ok Cupid details for months". WIRED UK. Retrieved February 24, 2017.
  6. ^ a b Conger, Kate (February 24, 2017). "Major Cloudflare bug leaked sensitive data from customers' websites". TechCrunch. Retrieved February 24, 2017.
  7. ^ a b c "CloudFlare Leaked Sensitive Data Across the Internet For Months". Fortune. Retrieved February 24, 2017.
  8. ^ Wagstaff, Jeremy (February 24, 2017). "Bug causes personal data leak, but no sign of hackers exploiting: Cloudflare". Reuters.
  9. ^ Marc Rogers interviewed on the TV show Triangulation on the TWiT.tv network
  10. ^ a b c Fox-Brewster, Thomas. "Google Just Discovered A Massive Web Leak... And You Might Want To Change All Your Passwords". Forbes. Retrieved February 24, 2017.
  11. ^ a b Estes, Adam Clark. "Everything You Need to Know About Cloudbleed, the Latest Internet Security Disaster". Gizmodo. Retrieved February 24, 2017.
  12. ^ "CloudBleed memory leak bug explained-why it all happened | PcSite". PcSite. February 25, 2017. Retrieved March 3, 2017.
  13. ^ "1139 - project-zero - Project Zero - Monorail".
  14. ^ "1139 - project-zero - Project Zero - Monorail".
  15. ^ a b Larson, Selena (February 24, 2017). "Why you shouldn't freak out (yet) about the 'Cloudbleed' security leak". CNNMoney. Retrieved February 24, 2017.
  16. ^ "Help article: How is Fitbit keeping my data secure in light of the Cloudflare security issue?". help.fitbit.com. Archived from the original on July 7, 2017. Retrieved July 13, 2020.
  17. ^ "Three layers of encryption keeps you safe when SSL/TLS fails | 1Password". 1Password Blog. February 23, 2017. Retrieved December 30, 2023.
  18. ^ "Cloudbleed: How to deal with it". Medium. February 24, 2017. Retrieved February 24, 2017.
  19. ^ "Cloudbleed Explained: Flaw Exposes Mountains of Private Data". Popular Mechanics. February 24, 2017. Retrieved February 24, 2017.
  20. ^ Constantin, Lucian. "Cloudflare bug exposed passwords, other sensitive data from websites". CIO. Archived from the original on February 25, 2017. Retrieved February 24, 2017.
  21. ^ Menegus, Bryan. "Change Your Passwords. Now". Gizmodo. Retrieved February 24, 2017.
  22. ^ Weinstein, David (February 24, 2017). "Cloudflare 'Cloudbleed' bug impact on mobile apps: Data sample of..." NowSecure. Retrieved February 24, 2017.
  23. ^ "Dark Reading - Cloudflare Leaked Web Customer Data For Months". www.darkreading.com. Retrieved February 25, 2017.
  24. ^ Joseph Steinberg (February 24, 2017). "Why You Can Ignore Calls To Change Your Passwords After Today's Massive Password Leak Announcement". Inc. Retrieved February 24, 2017.
edit