For correction, I did not ask that question. I quoted David Sarokin’s email in my reply. Please do read thoroughly to avoid misquoting someone. To my knowledge, Microsoft Defender played no part in this outage so I'm not sure why it's being raised. Windows acted as it was supposed to, detected a security issue and crashed to prevent potential damage. The issue isn't with the OS here, it's with the third-party application. Crowdstrike also released a known working workaround to get systems back online so there is no need to restart "as much as 15 times" to restore systems. This workaround had been (to my knowledge) communicated to all of their clients. Regards, Christopher Hawker ________________________________ From: ADEGBOYEGA OYESANYA <aaoyesanya@live.com> Sent: Wednesday, July 24, 2024 7:56 AM To: philip virgo via InternetPolicy <internetpolicy@elists.isoc.org>; Christopher Hawker <chris@thesysadmin.au>; David Sarokin <sarokin@gmail.com>; philip virgo <virgo.philip@outlook.com> Cc: Michele Neylon - Blacknight <michele@blacknight.com>; Antony Van Couvering <avc@avc.vc>; At-Large Worldwide <at-large@atlarge-lists.icann.org> Subject: Re: [Internet Policy] [At-Large] Re: Fragility of the World's Internet Infrastructure? According to Chris Hawker, "How about designing Windows so that an inadvertent call to a null-value address leads to some result *other than* a BSOD. A simple error message would suffice. Is that doable?" Answer: Microsoft Operating Systems are equipped with Microsoft Defender built in Microsoft Security, once activated it acts as, antivirus, malware detector/notifier/blocker, spyware detector etc..etc it notifies you about malwares and malicious programs that can cause harm to you computer, it notifies and protects you and your computer from various suspicious attacks on the internet and on your intranet. System Administrators and individuals use this features and other firewalls to protect systems and deny unauthorized access to Microsoft Operating Systems. Other antivirus products and programs are also used. That is why I said initially and I repeat again, this global "BSOD" outage needs critical and strategic analysis. We really need to actually identify the cause, it will help us in future. THIS IS A VERY SERIOUS WORLDWIDE TECHNICAL ISSUE. Which error or bug in the Crowdstrike program caused it? On which system did it start? Which country/ state/city? What time? Our solution to this problem right now is restarting, (Microsoft Corporation recommends restarting as much as 15 times, rebooting systems and troubleshooting "This Works"). Luckily we have amazing Systems Administrators and Technology personnel worldwide doing a wonderful job restoring systems. Adegboyega Oyesanya President ComputerLab Incorporation ________________________________ From: InternetPolicy <internetpolicy-bounces@elists.isoc.org> on behalf of philip virgo via InternetPolicy <internetpolicy@elists.isoc.org> Sent: Tuesday, July 23, 2024 3:56 AM To: Christopher Hawker <chris@thesysadmin.au>; David Sarokin <sarokin@gmail.com> Cc: Michele Neylon - Blacknight <michele@blacknight.com>; Sivasubramanian M via InternetPolicy <internetpolicy@elists.isoc.org>; Antony Van Couvering <avc@avc.vc>; At-Large Worldwide <at-large@atlarge-lists.icann.org> Subject: Re: [Internet Policy] [At-Large] Re: Fragility of the World's Internet Infrastructure? Would it be acceptable to include this e-mail trail (edited to remove personal e-mail addresses and other identifying information) as an appendix to a blog on the theme “When a digital butterfly flaps it wings” for a political audience. I will also be seeking peer reviewers for a succinct set of draft high level, non-technical recommendations and to be able to refer credible professional/technical recommendations. P.S. I have been bemused since I joined ISOC in 1995 by the apparently widespread acceptance of coding practices which were a firing offence when I was trainee programmer/analyst in 1968/9 …. e.g. unclosed loops and decision points which do not lead to failsafe and relatively easy recovery … akin to those built in to cater to for restarts in the middle of a task because of hardware/comms failure. I naively expected the issues to be addressed during the widespread audits that took place during the run up to Y2K but …. I still like the definition of the IETF as “a relay team trying to keep one lap ahead of disaster”. From: InternetPolicy <internetpolicy-bounces@elists.isoc.org> On Behalf Of Christopher Hawker via InternetPolicy Sent: Monday, July 22, 2024 2:10 PM To: David Sarokin <sarokin@gmail.com> Cc: Michele Neylon - Blacknight <michele@blacknight.com>; Sivasubramanian M via InternetPolicy <internetpolicy@elists.isoc.org>; Antony Van Couvering <avc@avc.vc>; At-Large Worldwide <at-large@atlarge-lists.icann.org> Subject: Re: [Internet Policy] [At-Large] Re: Fragility of the World's Internet Infrastructure? Hi David, [Note: I got some information wrong. It wasn't trying to access 0x0, rather it was trying to access "NULL + 0x9c = 0x9c" which is an invalid region in memory. The end result is the same, in that the application was accessing an invalid memory region.] Your suggestion of a fix puts the responsibility on the upstream software providers. Yes, because IMO it's the responsibility of the software vendors (Crowdstrike in this case) to ensure their application has been tested for compatibility with Windows, or any other OS on which their applications run. I wonder if there's a downstream option as well. How about designing Windows so that an inadvertent call to a null-value address leads to some result *other than* a BSOD ... How about designing Windows so that an inadvertent call to a null-value address leads to some result *other than* a BSOD. A simple error message would suffice. Is that doable? Windows crashing and throwing a BSOD is by design - if an application attempts to access an invalid memory region, it treats this as a security threat and intentionally crashes to mitigate any potential security issue (because an application shouldn't be trying to access an invalid memory region in the first place). where/how did you learn of the technical details of the crash? There's plenty online. Hint: search for "Crowdstrike null pointer". Normally, any reasonable developer would implement checks to prevent this. Regards, Christopher Hawker ________________________________ From: David Sarokin <sarokin@gmail.com<mailto:sarokin@gmail.com>> Sent: Monday, July 22, 2024 10:18 PM To: Christopher Hawker <chris@thesysadmin.au<mailto:chris@thesysadmin.au>> Cc: ADEGBOYEGA OYESANYA <aaoyesanya@live.com<mailto:aaoyesanya@live.com>>; Sivasubramanian M <6.internet@gmail.com<mailto:6.internet@gmail.com>>; Sivasubramanian M via InternetPolicy <internetpolicy@elists.isoc.org<mailto:internetpolicy@elists.isoc.org>>; Antony Van Couvering <avc@avc.vc<mailto:avc@avc.vc>>; Michele Neylon - Blacknight <michele@blacknight.com<mailto:michele@blacknight.com>>; At-Large Worldwide <at-large@atlarge-lists.icann.org<mailto:at-large@atlarge-lists.icann.org>> Subject: Re: [Internet Policy] [At-Large] Re: Fragility of the World's Internet Infrastructure? Christopher Hawker wrote: A simple solution to this issue would have been to implement a check for null values. If it detects null then do this, otherwise do that. It's a very easy fix. Thanks for that explanation...I was wondering what the actual problem was. Your suggestion of a fix puts the responsibility on the upstream software providers. I wonder if there's a downstream option as well. How about designing Windows so that an inadvertent call to a null-value address leads to some result *other than* a BSOD. A simple error message would suffice. Is that doable? Also, where/how did you learn of the technical details of the crash? I had been looking for that sort of info, but didn't see any obvious source in the news articles I scanned. Thanks again... On Mon, Jul 22, 2024 at 1:07 AM Christopher Hawker via InternetPolicy <internetpolicy@elists.isoc.org<mailto:internetpolicy@elists.isoc.org>> wrote: So to me a global blue screen requires more critical research as to what caused it? and why it happened on multiple systems? at the same time? It was a null pointer in the C++ programming language that caused this crash. Address 0x0 represents no value (aka a null value). The Falcon application attempted to access this address for a value and because no value existed, it caused the crash. A simple solution to this issue would have been to implement a check for null values. If it detects null then do this, otherwise do that. It's a very easy fix. I find it hard to believe that 1 program can cause the blue screen to occur on multiple systems on the internet at the same time. It's actually very possible - all it takes is for one person to push an update with bad code and for companies to automatically accept these updates without testing. We are far too trusting of the applications that can access our devices and networks. Regards, Christopher Hawker ________________________________ From: InternetPolicy <internetpolicy-bounces@elists.isoc.org<mailto:internetpolicy-bounces@elists.isoc.org>> on behalf of Sivasubramanian M via InternetPolicy <internetpolicy@elists.isoc.org<mailto:internetpolicy@elists.isoc.org>> Sent: Monday, July 22, 2024 2:56 PM To: ADEGBOYEGA OYESANYA <aaoyesanya@live.com<mailto:aaoyesanya@live.com>> Cc: Sivasubramanian M via InternetPolicy <internetpolicy@elists.isoc.org<mailto:internetpolicy@elists.isoc.org>>; Antony Van Couvering <avc@avc.vc<mailto:avc@avc.vc>>; Michele Neylon - Blacknight <michele@blacknight.com<mailto:michele@blacknight.com>>; At-Large Worldwide <at-large@atlarge-lists.icann.org<mailto:at-large@atlarge-lists.icann.org>> Subject: Re: [Internet Policy] [At-Large] Re: Fragility of the World's Internet Infrastructure? On Sun, 21 Jul, 2024, 03:50 ADEGBOYEGA OYESANYA, <aaoyesanya@live.com<mailto:aaoyesanya@live.com>> wrote: Let us critically examine and explain the error message got on systems affected during the outage. We all know it is not an Internet Down situation. This is a System down situation affecting multiple systems at the same time. When we troubleshoot crashed systems, the reason for system crash defers, it varies. To me this is beyond Crowdstrike and we need to do more research into it. The blue screen of death means the system has crashed. The Operating System that controls all other applications and all other applications no longer works. In this case you have to manually reboot and troubleshoot, it could be caused by, power failure, power fluctuations, power surge, hardware failure or operating system failure, If 1 application "Crowstrike" has issues, the operating system will still run, but the crowstrike program will not launch and (if it needs an update you will get an alert/notification) I find it hard to believe that 1 program can cause the blue screen to occur on multiple systems on the internet at the same time, each system has its own level of protection, and if multiple corporations use Crowstrike an update/update will be done at different times. Having worked on Microsoft Operating Systems for a long time, I know that the systems are reliable, dependable, effective, efficient and rugged. So to me a global blue screen requires more critical research as to what caused it? and why it happened on multiple systems? at the same time? Yes, more critical research is needed on what caused the crowdstrike Microsoft issue; and definitely far more research needed on the machines and systems we built since James Watt invented the steam engine whether or not this falls within the mandate of the Internet Policy team and/or within ICANN's remit. Best regards, Adegboyega Oyesanya President ComputerLab Incorporation ________________________________ From: InternetPolicy <internetpolicy-bounces@elists.isoc.org<mailto:internetpolicy-bounces@elists.isoc.org>> on behalf of Sivasubramanian M via InternetPolicy <internetpolicy@elists.isoc.org<mailto:internetpolicy@elists.isoc.org>> Sent: Saturday, July 20, 2024 9:09 AM To: Antony Van Couvering <avc@avc.vc<mailto:avc@avc.vc>> Cc: internetpolicy@elists.isoc.org<mailto:internetpolicy@elists.isoc.org> <internetpolicy@elists.isoc.org<mailto:internetpolicy@elists.isoc.org>>; Michele Neylon - Blacknight <michele@blacknight.com<mailto:michele@blacknight.com>>; at-large@atlarge-lists.icann.org<mailto:at-large@atlarge-lists.icann.org> <at-large@atlarge-lists.icann.org<mailto:at-large@atlarge-lists.icann.org>> Subject: Re: [Internet Policy] [At-Large] Re: Fragility of the World's Internet Infrastructure? "This was not an Internet outage or shutdown. There was no loss of connectivity. The networks all continued to work. Data continued to flow" Correct. Due to this rationale, it is not an Internet problem? It is an Internet problem, and more than an Internet problem - a Technology problem. If this were the pre-Internet early 1970s with computers on the airplanes and airports running today's Microsoft O/S secured by Crowdstrike, not all computers would have been almost simultaneously been updated, and this wouldn't have been a problem of massive geographical scale. As Dan and Olaf acknowleges, "deploying software on Internet scale, in particular updates, can cause issues on a massive scale"; Also the "inter-reliance of systems and services" came to us bundled with the Internet. If our world is a world of "complex, interconnected systems", we have been careless about adopting and implementing these technologies, especially the Internet technologies. On the cloud, the crowdstike update could be done remotely, but the fix requires physical access - to one computer or another in a sequence ??? And, why does Security invariably brings about a Security issue of a totally different kind, especially on the Internet? Apart from diversity and redundancy (not merely five operating systems on a A 380, five different hardware architectures, alternating between analog and digital, five different communication systems from five different vendors across technogies, with a capacity to shut everything out and put the hands on the controls), we need something else - what I would call "backward functionality" as distinct from backward compatibility between versions of software or hardware. If the computer on the airplane wired to the "fly by wire" doesn't work, God forbid, mid-air, does the airplane's controls fall back to the foot pedal and mechanical levers and are there enough of these mechanical components left for the pilot to flip a switch here, flip a switch there, rotate a wheel with his foot doing something else and a copilot operating a few cables and a radio? Does the airline or the hotel have paper and ink on the table to write down the name of the passenger or guest and put them on the plane? Are there flip switches to the lights on the runway or on the traffic lights on the street? (This is very, very, very pertinent to the "Internet" issue raised here: "On July 14, 2013, India shut its state-run telegram service" Why? Because we have computers and the Internet? Sivasubramanian M [Sent by a Verified sender]<https://wallet.unumid.co/authenticate?referralCode=YQmHX1HBGdde> On Sat, Jul 20, 2024 at 5:16 PM Antony Van Couvering via At-Large <at-large@icann.org<mailto:at-large@icann.org>> wrote: Insofar as whereinso exists hereinbelow, whosoever as as may have spoken to whomsoever listens now explains that as wherasheretobefore Microsoft gained lots of market share, now asne’ertillnow we as noone everhereinbefore blame them throughoutaround and thoroughly forevermore so they should pay the damages notusnoteverus. On Jul 19, 2024, at 15:36, Michele Neylon - Blacknight via At-Large <at-large@icann.org<mailto:at-large@icann.org>> wrote: Which still doesn’t answer the question asked Mr Michele Neylon Blacknight Hosting & Domains https://www.blacknight.com<https://www.blacknight.com/> @mneylon Sent from mobile so typos and brevity are normal On 19 Jul 2024, at 22:51, Cody Tippie <cody.tippie@outlook.com<mailto:cody.tippie@outlook.com>> wrote: [EXTERNAL EMAIL] Please use caution when opening attachments from unrecognised sources. Michele Neylon, Incidence relational to Crowdstrike runs concurrent with examination as to that of Cyberespionage or simply to oligarchic competitioning. Crowdstrike was ranked first by that of the Gartner, seconded to Microsoft and to similar basis of relation to that of Forester Wave Cloud Workload Security Q1 2024. Wheresoin Microsoft has been steadily gaining with regard to that of Platform security technologies therein to the overtaking or ousting of capabilities typically offered by a diveristy of otherwise service providers and competitors; leveraging any such security therein directly integrated or built into such stated Technology providers platform. With any such outages occurring across that of the Security provider Crowdstrike itself; such forces firms to therein migrate to otherwise service providers; or to that of platform hosted services which can offer similar and integrated reliability and services that therein outcompete basis as to that of which smaller firms solutions and security providers provide. Per the WEF Global Risk Report 2024; consolidation of technological power was a merging basis of concern with relation to that of certain strata in or across society. https://www.weforum.org/publications/global-risks-report-2024/ [https://www.weforum.org/weforum.jpg]<https://www.weforum.org/publications/global-risks-report-2024/> Global Risks Report 2024 | World Economic Forum<https://www.weforum.org/publications/global-risks-report-2024/> The Global Risks Report 2024 from the World Economic Forum explores some of the most severe risks we may face over the next decade. Read the report to learn more. www.weforum.org<http://www.weforum.org/> Best Regards, Cody Ryan Tippie ________________________________ From: InternetPolicy <internetpolicy-bounces@elists.isoc.org<mailto:internetpolicy-bounces@elists.isoc.org>> on behalf of Michele Neylon - Blacknight via InternetPolicy <internetpolicy@elists.isoc.org<mailto:internetpolicy@elists.isoc.org>> Sent: Friday 19 July 2024 16:44 To: Roberto Gaetano <roberto_gaetano@hotmail.com<mailto:roberto_gaetano@hotmail.com>> Cc: Alejandro Pisanty <apisan@unam.mx<mailto:apisan@unam.mx>>; At Large <at-large@atlarge-lists.icann.org<mailto:at-large@atlarge-lists.icann.org>>; InternetPolicy@elists.isoc.org<mailto:InternetPolicy@elists.isoc.org> <InternetPolicy@elists.isoc.org<mailto:InternetPolicy@elists.isoc.org>> Subject: Re: [Internet Policy] [At-Large] Fragility of the World's Internet Infrastructure? Insurance maybe? I doubt that CrowdStrike or anyone else will accept liability Mr Michele Neylon Blacknight Hosting & Domains https://www.blacknight.com<https://www.blacknight.com/> @mneylon Sent from mobile so typos and brevity are normal On 19 Jul 2024, at 21:42, Roberto Gaetano <roberto_gaetano@hotmail.com<mailto:roberto_gaetano@hotmail.com>> wrote: [EXTERNAL EMAIL] Please use caution when opening attachments from unrecognised sources. <off topic> Any idea on who is going to pay for the damages? Cheers, R. On 19.07.2024, at 22:16, Michele Neylon - Blacknight via At-Large <at-large@icann.org<mailto:at-large@icann.org>> wrote: Yeah Software NOT internet related at all Those of us who don’t use that company’s products weren’t impacted at all Though I am glad I wasn’t flying with Ryanair today! Mr Michele Neylon Blacknight Hosting & Domains https://www.blacknight.com<https://www.blacknight.com/> @mneylon Sent from mobile so typos and brevity are normal On 19 Jul 2024, at 20:18, Dr. Alejandro Pisanty Baruch via At-Large <at-large@icann.org<mailto:at-large@icann.org>> wrote: [EXTERNAL EMAIL] Please use caution when opening attachments from unrecognised sources. Amazingly it still takes some effort to know what is and what is not an Internet issue. Better read this brief, clear text from ISOC, by Olaf Kolkmann and Dan York: https://www.internetsociety.org/blog/2024/07/global-tech-outage-demonstrates... Alejandro Pisanty [https://www.internetsociety.org/wp-content/uploads/2024/07/outage-ATM-screen-1.jpg]<https://www.internetsociety.org/blog/2024/07/global-tech-outage-demonstrates-need-for-resiliency-in-software-systems/> Global Tech Outage Demonstrates Need for Resiliency in Software Systems - Internet Society<https://www.internetsociety.org/blog/2024/07/global-tech-outage-demonstrates...> www.internetsociety.org<http://www.internetsociety.org/> Today's global outage of many systems was not an Internet outage, but it demonstrates the need for greater resilience in all our technical systems. ________________________________ De: Sivasubramanian M via At-Large <at-large@icann.org<mailto:at-large@icann.org>> Enviado: viernes, 19 de julio de 2024 05:59 a. m. Para: at-large@atlarge-lists.icann.org<mailto:at-large@atlarge-lists.icann.org>; internetpolicy@elists.isoc.org<mailto:internetpolicy@elists.isoc.org> Asunto: [At-Large] Fragility of the World's Internet Infrastructure? "it is a very, very uncomfortable illustration of the fragility of the world's core Internet infrastructure" said Ciaran Martin, Professor of Oxford's Blavatnik School of Government and former head of UK national cyber security center... _______________________________________________ At-Large mailing list -- at-large@icann.org<mailto:at-large@icann.org> To unsubscribe send an email to at-large-leave@icann.org<mailto:at-large-leave@icann.org> At-Large Official Site: http://atlarge.icann.org<http://atlarge.icann.org/> _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on. _______________________________________________ At-Large mailing list -- at-large@icann.org<mailto:at-large@icann.org> To unsubscribe send an email to at-large-leave@icann.org<mailto:at-large-leave@icann.org> At-Large Official Site: http://atlarge.icann.org<http://atlarge.icann.org/> _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on. _______________________________________________ At-Large mailing list -- at-large@icann.org<mailto:at-large@icann.org> To unsubscribe send an email to at-large-leave@icann.org<mailto:at-large-leave@icann.org> At-Large Official Site: http://atlarge.icann.org<http://atlarge.icann.org/> _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on. _______________________________________________ At-Large mailing list -- at-large@icann.org<mailto:at-large@icann.org> To unsubscribe send an email to at-large-leave@icann.org<mailto:at-large-leave@icann.org> At-Large Official Site: http://atlarge.icann.org<http://atlarge.icann.org/> _______________________________________________ By submitting your personal data, you consent to the processing of your personal data for purposes of subscribing to this mailing list accordance with the ICANN Privacy Policy (https://www.icann.org/privacy/policy) and the website Terms of Service (https://www.icann.org/privacy/tos). You can visit the Mailman link above to change your membership status or configuration, including unsubscribing, setting digest-style delivery or disabling delivery altogether (e.g., for a vacation), and so on. _______________________________________________ To view your Internet Society subscriptions or unsubscribe, log into your member profile at https://community.internetsociety.org/, select your profile picture and then My Groups. Select the Edit icon for Internet Policy and check the Leave Group box, then Save. Changes may take up to 3 hours to take effect. - View the Internet Society Code of Conduct: https://www.internetsociety.org/become-a-member/code-of-conduct/