Jump to content

Facebook, Insta, Whatsapp down in major outage


Capricorn

Recommended Posts

What's our network gurus think happened here?what does it say about how their services are arranged?

Seems the outage cascaded globally starting around 5pm

Link to comment
Share on other sites

No idea, our peering is still up. So BGP sessions have not dropped.
Assuming it's some internal routing issue or maybe equipment failure.
Could be the servers hosting the platforms that fell over.

Or maybe someone messed coffee over their keyboard and when wiping it down hit the reboot button by accident????

Link to comment
Share on other sites

All of them are owned by Facebook so Im pretty sure they all use the same CDN.

I know Whatsapp is down because my wife asked a few minutes ago if the internet was down while trying to use it.

 

Seems its down to MTN and Afrihost fibre. The standard down detectors seem to think its down too so this is a pretty big outage.

Link to comment
Share on other sites

The comments on this tweet and the one from Instagram makes for some hilarious reading. All the "creators" loosing their **** about not being able to "influence" the masses 

Link to comment
Share on other sites

I no longer use facebook, but somewhat enjoy instagram.

Been pretty eyeopening now that its been down for approx 2 hrs how easily one becomes "reliant" on social media. And I don't even post anything, just enjoy looking at cool ****.

Link to comment
Share on other sites

For the ones interested.
It was a network config change that broke their DNS. And, they got locked out of their remote access.

So they ended up having to go to their DC to physically restart the servers and fix the config change.

Link to comment
Share on other sites

From last night obviously. still funny.

 

c437999c-5da9-4e79-977a-48097fd96ff7.jpg.3bb9b33679ab100dbf02916874ecfed0.jpg

Apparently Facebook is on the same BGP as the system that runs employee access cards for Facebook. lol. As a rough translation of the above, the security guard is locked inside the datacentre. The sysadmin is right outside but his keycard doesn't work, and the engineer lives in Mexico.. 

Edited by 100Tours
Link to comment
Share on other sites

22 hours ago, 'Dale said:

Not surprising and still shocking 

Interesting that she has the same lawyers as the famous Ukranian whistleblower that led to the Trump impeachment (same whistleblower who turned out to be VP Bidens top advisor on the Ukraine! Pure theatrics for "tighter tech laws".

 

 

Link to comment
Share on other sites

On 10/5/2021 at 6:23 PM, 100Tours said:

From last night obviously. still funny.

 

c437999c-5da9-4e79-977a-48097fd96ff7.jpg.3bb9b33679ab100dbf02916874ecfed0.jpg

Apparently Facebook is on the same BGP as the system that runs employee access cards for Facebook. lol. As a rough translation of the above, the security guard is locked inside the datacentre. The sysadmin is right outside but his keycard doesn't work, and the engineer lives in Mexico.. 

Not to nitpick, but BGP is a routing protocol, their issue had something to do with DNS going down due to a disconnect between their datacentres.
pretty decent explanation here - https://www.theguardian.com/technology/2021/oct/05/what-caused-facebook-whatsapp-instagram-outage

If you want a easy to understand analogy, imagine that DNS is a post office, it is able to identify a persons name with a street address. BGP would be the delivery driver getting your package to the destination.

In realworld, DNS resolves hostnames to IP addresses, so when you type in google.com into your browser, DNS resolves the hostname to an ip address. Your ISP will then use BGP routes to see where it is learning the route from and steer the packet accordingly.

Link to comment
Share on other sites

8 hours ago, Jurgens Smit said:

Not to nitpick, but BGP is a routing protocol, their issue had something to do with DNS going down due to a disconnect between their datacentres.
pretty decent explanation here - https://www.theguardian.com/technology/2021/oct/05/what-caused-facebook-whatsapp-instagram-outage

If you want a easy to understand analogy, imagine that DNS is a post office, it is able to identify a persons name with a street address. BGP would be the delivery driver getting your package to the destination.

In realworld, DNS resolves hostnames to IP addresses, so when you type in google.com into your browser, DNS resolves the hostname to an ip address. Your ISP will then use BGP routes to see where it is learning the route from and steer the packet accordingly.

Yah I know the difference - we just read different versions of the news.

What took Facebook down | ZDNet

Why Facebook went down, and what's BGP routing | CyberNews

More details about the October 4 outage - Facebook Engineering (fb.com)

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Settings My Forum Content My Followed Content Forum Settings Ad Messages My Ads My Favourites My Saved Alerts My Pay Deals Help Logout