GameTZ

Sign OnNon-Mobile

I'm not sure that I've seen like half the internet down before
21-Jun-22 3:17am
#1
metsfan718


At this very second like half of the internet is down. Sites like discord, Ring home security, door dash , MLB.com and much more.

I'm seeing that it's a cloud fare issue or something?

Kind of makes you think how fragile the internet is. Especially when the larger websites are hosted by so few companies.

But don't worry. The legendary GTZ will live forever!

Edit: starting to come back up. Took like 15-20 minutes. Went down a little past 3AM.
21-Jun-22 7:15am
#2
Boss
GameTZ Subscriber 700 Trade Quintuple Gold Good Trader Gold Global Trader (8)

I slept through the great internet crash of 2022.
21-Jun-22 8:53am
#3
MrBean
GameTZ Gold Subscriber 350 Trade Quintuple Gold Good Trader Gold Global Trader (7) Has Written 1 Review

Didn't know this happened until just now ... trying to find root cause, and cloudflare clearly isn't sharing what actually happened. Appears to have been related to some DNS issues.
21-Jun-22 9:14am
#4
longhornsk57
Quadruple Gold Good Trader

Probably someone pushed a bad BGP route into some core routers. Same thing that happened with Facebook a while back.
21-Jun-22 10:24am
#5
John
GameTZ Gold Subscriber GameTZ Full Moderator 400 Trade Quintuple Gold Good Trader Gold Global Trader (13)

Cloudflare really needs to change their error message. People not using NGINX are like "What the heck is NGINX??" and people that happen to also use NGINX themselves are like "Is my NGINX failing??" Just change the message to say Cloudflare. Grrr...
21-Jun-22 10:38am
#6
longhornsk57
Quadruple Gold Good Trader

It's an automatic message from their system. Changing an error message on the fly for a whole set of servers is probably not a good use of time.

Anyone who even knows what NGINX is and is getting that error knows it's not on their end.
21-Jun-22 11:30am
#7
Anxiouz
900 Trade Quintuple Gold Good Trader

I'm reading up on this, this morning. This sounds awfully similar to what happened a few months ago (during US business hours) which is concerning.
21-Jun-22 12:26pm
#8
sinnie
GameTZ Subscriber 600 Trade Quintuple Gold Good Trader Global Trader - willing to trade internationally

Yeah it's happened before but this one most of us didn't notice as it was in the middle of the night.

21-Jun-22 1:24pm
#9
longhornsk57
Quadruple Gold Good Trader

At the end of the day the entire internet is just a fiber optic cables, some core routers, and BGP routes (a protocol that tells the routers where to send packets). It really is pretty frail.
21-Jun-22 1:31pm
#10
Porksta
GameTZ Subscriber 400 Trade Quintuple Gold Good Trader Global Trader - willing to trade internationally

I thought it was a series of tubes?

21-Jun-22 1:56pm
#11
John
GameTZ Gold Subscriber GameTZ Full Moderator 400 Trade Quintuple Gold Good Trader Gold Global Trader (13)

longhornsk57 wrote:
It's an automatic message from their system. Changing an error message on the fly for a whole set of servers is probably not a good use of time.
Yeah, the hundreds of CloudFlare servers are not individually maintained, I'm sure. They would push it out as a new update to all of their servers at once. And we can agree to disagree on if it is a good use of time. (Note that, on the Reddit SysAdmin sub, it is full of other IT people like me wishing that they would change this stupid message.)

Anyone who even knows what NGINX is and is getting that error knows it's not on their end.
That's simply not true. It looks just like the standard NGINX message -- which makes it difficult to tell, at first look, if the problem is a local NGINX issue or CloudFlare. That's literally why it came up on Reddit recently -- because an admin thought it was his local proxy at first.
21-Jun-22 2:03pm
#12
longhornsk57
Quadruple Gold Good Trader

Ya I mean my thing is if there's still an issue with routing or something preventing connectivity they can't push a global config update to change an error message that by now any sysadmin probably knows isn't a local issue
21-Jun-22 10:47pm
#13
willyum
GameTZ Gold Subscriber 300 Trade Quintuple Gold Good Trader Gold Global Trader (17)

Didn't something like this happen before? I think it was an Amazon Web Sever at that time...

22-Jun-22 3:27am
#14
Reed
450 Trade Quintuple Gold Good Trader Global Trader - willing to trade internationally

This is why we need a decentralized internet built on the blockchain.
22-Jun-22 2:36pm
#15
John
GameTZ Gold Subscriber GameTZ Full Moderator 400 Trade Quintuple Gold Good Trader Gold Global Trader (13)

longhornsk57 wrote:
Ya I mean my thing is if there's still an issue with routing or something preventing connectivity they can't push a global config update to change an error message that by now any sysadmin probably knows isn't a local issue
I think the point is that -- when things are working -- they don't seem to update the "failure" message. It's not that people want them to change the message DURING the event. It is that they are tired of it coming up that way every time they have an event and think that they should change it when things are working -- like right now. So that next time it happens, SysAdmins don't have to go "Is this a local issue?"
22-Jun-22 2:37pm
#16
John
GameTZ Gold Subscriber GameTZ Full Moderator 400 Trade Quintuple Gold Good Trader Gold Global Trader (13)

Reed wrote:
This is why we need a decentralized internet built on the blockchain.
When you speak, I'm always like "Is Reed making a joke? Or does he believe that..."

laughing out loud
22-Jun-22 8:36pm
#17
longhornsk57
Quadruple Gold Good Trader

John wrote:
longhornsk57 wrote:> Ya I mean my thing is if there's still an issue with routing or something preventing> connectivity they can't push a global config update to change an error message that> by now any sysadmin probably knows isn't a local issue I think the point is that -- when things are working -- they don't seem to update the "failure" message. It's not that people want them to change the message DURING the event. It is that they are tired of it coming up that way every time they have an event and think that they should change it when things are working -- like right now. So that next time it happens, SysAdmins don't have to go "Is this a local issue?"
Ya I can agree with that point, they should just change the error message altogether when they can.
23-Jun-22 7:50pm
#18
metsfan718


Verizon FIOS had an East Coast internet outage during the overnight and into the morning. That happened last night. Not sure what was up with that. They're usually super reliable.

I'm not sure that I've seen like half the internet down before