The Internet has become a significant factor in geopolitical conflicts, such as the ongoing war in Ukraine. Tomorrow marks one year since the Russian invasion of that country. This post reports on Internet insights and discusses how Ukraine's Internet remained resilient in spite of dozens of disruptions in three different stages of the conflict.

Key takeaways:

• Internet traffic shifts in Ukraine are clearly visible from east to west as Ukrainians fled the war, with country-wide traffic dropping as much as 33% after February 24, 2022.

• Air strikes on energy infrastructure starting in October led to widespread Internet disruptions that continue in 2023.

• Application-layer cyber attacks in Ukraine rose 1,300% in early March 2022 compared to pre-war levels.

• Government administration, financial services, and the media saw the most attacks targeting Ukraine.

• Traffic from a number of networks in Kherson was re-routed through Russia between June and October, subjecting traffic to Russia’s restrictions and limitations, including content filtering. Even after traffic ceased to reroute through Russia, those Ukrainian networks saw major outages through at least the end of the year, while two networks remain offline.

• Through efforts on the ground to repair damaged fiber optics and restore electrical power, Ukraine’s networks have remained resilient from both an infrastructure and routing perspective. This is partly due to Ukraine’s widespread connectivity to networks outside the country and large number of IXPs.

• Starlink traffic in Ukraine grew over 500% between mid-March and mid-May, and continued to grow from mid-May through mid-November, increasing nearly 300% over that six-month period. For the full period from mid-March (two weeks after it was made available) to mid-December, it was over a 1,600% increase, dropping a bit after that.

In Ukraine, human Internet traffic dropped as much as 33% in the weeks following February 24. The following chart shows Cloudflare’s perspective on daily traffic (by number of requests).

Internet traffic levels recovered over the next few months, including strong growth seen in September and October, when many Ukrainian refugees returned to the country. That said, there were also country-wide outages, mostly after October, that are discussed below.

14% of total traffic from Ukraine (including traffic from Crimea and other occupied regions) was mitigated as potential attacks, while 10% of total traffic to Ukraine was mitigated as potential attacks in the last 12 months.

Before February 24, 2022, typical weekday Internet traffic in Ukraine initially peaked after lunch, around 15:00 local time, dropped between 17:00 and 18:00 (consistent with people leaving work), and reached the biggest peak of the day at around 21:00 (possibly after dinner for mobile and streaming use).

After the invasion started, we observed less variation during the day in a clear change in the usual pattern given the reported disruption and “exodus” from the country​. During the first few days after the invasion began, peak traffic occurred around 19:00, at a time when nights for many in cities such as Kyiv were spent in improvised underground bunkers. By late March, the 21:00 peak had returned, but the early evening drop in traffic did not return until May.

When looking at Ukraine Internet requests by type of traffic in the chart below (from February 10, 2022, through February 2023), we observe that while traffic from both mobile and desktop devices dropped after the invasion, request volume from mobile devices has remained higher over the past year. Pre-war, mobile devices accounted for around 53% of traffic, and grew to around 60% during the first weeks of the invasion. By late April, it had returned to typical pre-war levels, falling back to around 54% of traffic. There’s also a noticeable December drop/outage that we’ll go over below.

The invasion brought attacks and failing infrastructure across a number of cities, but the target in the early days wasn’t the country’s energy infrastructure, as it was in October 2022. In the first weeks of the war, Internet traffic changes were largely driven by people evacuating conflict zones with their families. Over eight million Ukrainians left the country in the first three months, and many more relocated internally to safer cities, although many returned during the summer of 2022. The Internet played a critical role during this refugee crisis, supporting communications and access to real-time information that could save lives, as well as apps providing services, among others.

There was also an increase in traffic in the western part of Ukraine, in areas such as Lviv (further away from the conflict areas), and a decrease in the east, in areas like Kharkiv, where the Russian military was arriving and attacks were a constant threat. The figure below provides a view of how Internet traffic across Ukraine changed in the week after the war began (a darker pink means a drop in traffic — as much as 60% — while a darker green indicates an increase in Internet traffic — as much as 50%).

Source: https://datawrapper.dwcdn.net/dsUSJ/2/

The biggest drops in Internet traffic observed in Ukraine in the first days of the war were in Kharkiv Oblast in the east, and Chernihiv in the north, both with a 60% decrease, followed by Kyiv Oblast, with traffic 40% lower on March 2, 2022, as compared with February 23.

In western Ukraine, traffic surged. The regions with the highest observed traffic growth included Rivne (50%), Volyn (30%), Lviv (28%), Chernivtsi (25%), and Zakarpattia (15%).

At the city level, analysis of Internet traffic in Ukraine gives us some insight into usage of the Internet and availability of Internet access in those first weeks, with noticeable outages in places where direct conflict was going on or that was already occupied by Russian soldiers.

North of Kyiv, the city of Chernihiv had a significant drop in traffic the first week of the war and residual traffic by mid-March, with traffic picking up only after the Russians retreated in early April.

In the capital city of Kyiv, there is a clear disruption in Internet traffic right after the war started, possibly caused by people leaving, attacks and use of underground shelters.

Near Kyiv, we observed a clear outage in early March in Bucha. After April 1, when the Russians withdrew, Internet traffic started to come back a few weeks later.

In Irpin, just outside Kyiv, close to the Hostomel airport and Bucha, a similar outage pattern to Bucha was observed. Traffic only began to come back more clearly in late May.

In the east, in the city of Kharkiv, traffic dropped 50% on March 3, with a similar scenario seen not far away in Sumy. The disruption was related to people leaving and also by power outages affecting some networks.

Other cities in the south of Ukraine, like Berdyansk, had outages. This graph shows Enerhodar, the small city where Europe’s largest nuclear plant, Zaporizhzhya NPP, is located, with residual traffic compared to before.

In the cities located in the south of Ukraine, there were clear Internet disruptions. The Russians laid siege to Mariupol on February 24. Energy infrastructure strikes and shutdowns had an impact on local networks and Internet traffic, which fell to minimal levels by March 1. Estimates indicate that 95% of the buildings in the city were destroyed, and by mid-May, the city was fully under Russian control. While there was some increase in traffic by the end of April, it reached only ~22% of what it was before the war’s start.

When looking at Ukrainian Internet Service Providers (ISPs) or the autonomous systems (ASNs) they use, we observed more localized disruptions in certain regions during the first months of the war, but recovery was almost always swift. AS6849 (Ukrtel) experienced problems with very short-term outages in mid-March. AS13188 (Triolan), which services Kyiv, Chernihiv, and Kharkiv, was another provider experiencing problems (they reported a cyberattack on March 9), as could be observed in the next chart:

We did not observe a clear national outage in Ukraine’s main ISP, AS15895 (Kyivstar) until the October-November attacks on energy infrastructure, which also shows some early resilience of Ukrainian networks.

As Russian troops retreated from the northern front in Ukraine, they shifted their efforts to gain ground in the east (Battle of Donbas) and south (occupation of the Kherson region) after late April. This resulted in Internet disruptions and traffic shifts, which are discussed in more detail in a section below. However, Internet traffic in the Kherson region was intermittent and included outages after May, given the battle for Internet control. News reports in June revealed that ISP workers damaged their own equipment to thwart Russia’s efforts to control the Ukrainian Internet.

Before the September Ukrainian counteroffensive, another example of the war’s impact on a city’s Internet traffic occurred during the summer, when Russian troops seized Lysychansk in eastern Ukraine in early July after what became known as the Battle of Lysychansk. Internet traffic in Lysychansk clearly decreased after the war started. That slide continues during the intense fighting that took place after April, which led to most of the city’s population leaving. By May, traffic was almost residual (with a mid-May few days short term increase).

In early September the Ukrainian counteroffensive took off in the east, although the media initially reported a south offensive in Kherson Oblast that was a “deception” move. The Kherson offensive only came to fruition in late October and early November. Ukraine was able to retake in September over 500 settlements and 12,000 square kilometers of territory in the Kharkiv region. At that time, there were Internet outages in several of those settlements.

In response to the successful Ukrainian counteroffensive, Russian airstrikes caused power outages and Internet disruptions in the region. That was the case in Kharkiv on September 11, 12, and 13. The figure below shows a 12-hour near-complete outage on September 11, followed by two other periods of drop in traffic.

In the Zaporizhzhia region, there were also outages. On September 1, 2022, the day the International Atomic Energy Agency (IAEA) inspectors arrived at the Russian-controlled Zaporizhzhia nuclear power plant in Enerhodar, there were Internet outages in two local ASNs that service the area: AS199560 (Engrup) and AS197002 (OOO Tenor**)**. Those outages lasted until September 10, as shown in the charts below.

More broadly, the city of Enerhodar, where the nuclear power plant is located, experienced a four-day outage after September 6.

In mid-September, following Ukraine’s counteroffensive, there were questions as to when Crimea might be targeted by Ukrainian forces, with news reports indicating that there was an evacuation of the Russian population from Crimea around September 13. We saw a clear drop in traffic on that Tuesday, compared with the previous day, as seen in the map of Crimea below (red is decrease in traffic, green is increase).

As we have seen, the Russian air strikes targeting critical energy infrastructure began in September as a retaliation to Ukraine's counteroffensive. The following month, the Crimean Bridge explosion on Saturday, October 8 (when a truck-borne bomb destroyed part of the bridge) led to more air strikes that affected networks and Internet traffic across Ukraine.

On Monday, October 10, Ukraine woke up to air strikes on energy infrastructure and experienced severe electricity and Internet outages. At 07:35 UTC, traffic in the country was 35% below its usual level compared with the previous week and only fully recovered more than 24 hours later. The impact was particularly significant in regions like Kharkiv, where traffic was down by around 80%, and Lviv, where it dropped by about 60%. The graph below shows how new air strikes in Lviv Oblast the following day affected Internet traffic.

There were clear disruptions in Internet connectivity in several regions on October 17, but also on October 20, when the destruction of several power stations in Kyiv resulted in a 25% drop in Internet traffic from Kyiv City as compared to the two previous weeks. It lasted 12 hours, and was followed the next day by a shorter partial outage as seen in the graph below.

In late October, according to Ukrainian officials, 30% of Ukraine’s power stations were destroyed. Self-imposed power limitations because of this destruction resulted in drops in Internet traffic observed in places like Kyiv and the surrounding region.

The start of a multi-week Internet disruption in Kherson Oblast can be seen in the graph below, showing ~70% lower traffic than in previous weeks. The disruption began on Saturday, October 22, when Ukrainians were gaining ground in the Kherson region.

Traffic began to return after Ukrainian forces took Kherson city on November 11, 2022. The graph below shows a week-over-week comparison for Kherson Oblast for the weeks of November 7, November 28, and December 19 for better visualization in the chart while showing the evolution through a seven-week period.

Throughout the rest of the year and into 2023, Ukraine has continued to face intermittent Internet disruptions. On November 23, 2022, the country experienced widespread power outages after Russian strikes, causing a nearly 50% decrease in Internet traffic in Ukraine. This disruption lasted for almost a day and a half, further emphasizing the ongoing impact of the conflict on Ukraine's infrastructure.

Although there was a recovery after that late November outage, only a few days later traffic seemed closer to normal levels. Below is a chart of the week-over-week evolution of Internet traffic in Ukraine at both a national and local level during that time:

In Kyiv Oblast:

In the Odessa region:

And Kharkiv (where a December 16 outage is also clear — in the green line):

On December 16, there was another country-level Internet disruption caused by air strikes targeting energy infrastructure. Traffic at a national level dropped as much as 13% compared with the previous week, but Ukrainian networks were even more affected. AS13188 (Triolan) had a 70% drop in traffic, and AS15895 (Kyivstar) a 40% drop, both shown in the figures below.

In January 2023, air strikes caused additional Internet disruptions. One such recent event was in Odessa, where traffic dropped as low as 54% compared with the previous week during an 18-hour disruption.

The US government and the FBI issued warnings in March to all citizens, businesses, and organizations in the country, as well as allies and partners, to be aware of the need to “enhance cybersecurity.” The US Cybersecurity and Infrastructure Security Agency (CISA) launched the Shields Up initiative, noting that “Russia’s invasion of Ukraine could impact organizations both within and beyond the region.” The UK and Japan, among others, also issued warnings.

Below, we discuss Web Application Firewall (WAF) mitigations and DDoS attacks. A WAF helps protect web applications by filtering and monitoring HTTP traffic between a web application and the Internet. A WAF is a protocol layer 7 defense (in the OSI model), and is not designed to defend against all types of attacks. Distributed Denial of Service (DDoS) attacks are cyber attacks that aim to take down Internet properties and make them unavailable for users.

The charts below are based on normalized data, and show threats mitigated by our WAF.

Mitigated application-layer threats blocked by our WAF skyrocketed after the war started on February 24. Mitigated requests were 105% higher on Monday, February 28 than in the previous (pre-war) Monday, and peaked on March 8, reaching 1,300% higher than pre-war levels.

Between February 2022 and February 2023, an average of 10% of all traffic to Ukraine was mitigations of potential attacks.

The graph below shows the daily percentage of application layer traffic to Ukraine that Cloudflare mitigated as potential attacks. In early March, 30% of all traffic was mitigated. This fell in April, and remained low for several months, but it picked up in early September around the time of the Ukrainian counteroffensive in east and south Ukraine. The peak was reached on October 29 when DDoS attack traffic constituted 39% of total traffic to Cloudflare’s Ukrainian customer websites.

This trend is more evident when looking at all traffic to sites on the “.ua” top-level domain (from Cloudflare’s perspective). The chart below shows that DDoS attack traffic accounted for over 80% of all traffic by early March 2022. The first clear spikes occurred on February 16 and 19, with around 25% of traffic mitigated. There was no moment of rest after the war started, except towards the end of November and December, but the attacks resumed just before Christmas. An average of 13% of all traffic to “.ua”, between February 2022 and February 2023 was mitigations of potential attacks. The following graph provides a comprehensive view of DDoS application layer attacks on “.ua” sites:

Moving on to types of mitigations of product groups that were used (related to “.ua” sites), as seen in the next chart, around 57% were done by the ruleset which automatically detects and mitigates HTTP DDoS attacks (DDoS Mitigation), 31% were being mitigated by firewall rules put in place (WAF), and 10% were blocking requests based on our IP threat reputation database (IP Reputation).

It’s important to note that WAF rules in the graph above are also associated with custom firewall rules created by customers to provide a more tailored protection. “DDoS Mitigation” (application layer DDoS protection) and “Access Rules” (rate limiting) are specifically used for DDoS protection.

In contrast to the first graph shown in this section, which looked at mitigated attack traffic targeting Ukraine, we can also look at mitigated attack traffic originating in Ukraine. The graph below also shows that the share of mitigated traffic from Ukraine also increased considerably after the invasion started.

The industries sectors that had a higher share of WAF mitigations were government administration, financial services, and the media, representing almost half of all WAF mitigations targeting Ukraine during 2022.

Looking at DDoS attacks, there was a surge in attacks on media and publishing companies during 2022 in Ukraine. Entities targeting Ukrainian companies appeared to be focused on information-related websites. The top five most attacked industries in the Ukraine in the first two quarters of 2022 were all in broadcasting, Internet, online media, and publishing, accounting for almost 80% of all DDoS attacks targeting Ukraine.

In a more focused look at the type of websites Cloudflare has protected throughout the war, the next two graphs provide a view of mitigated application layer attacks by the type of “.ua” sites we helped to protect. In the first days of the war, mitigation spikes were observed at a news service, a TV channel, a government website, and a bank.

In July, spikes in mitigations we observed across other types of “.ua” websites, including food delivery, e-commerce, auto parts, news, and government.

More recently, in February 2023, the spikes in mitigations were somewhat similar to what we saw one year ago, including electronics, e-commerce, IT, and education websites.

Network-layer (layer 3 and 4) traffic is harder to attribute to a specific domain or target because IP addresses are shared across different customers. Looking at network-level DDoS traffic hitting our Kyiv data center, we saw peaks of DDoS traffic higher than before the war in early March, but they were much higher in June and August.

In our Q1 2022 DDoS report, we also noted that 12.6% of Ukraine’s traffic was DDoS activity, compared with 1% in the previous quarter, a 1,160% quarter-over-quarter increase.

Several of our quarterly DDoS reports from 2022 include attack trends related to the war in Ukraine, with quarter over quarter interactive comparisons.

On February 24, 2022, Russian forces invaded Ukraine's Kherson Oblast region. The city of Kherson was captured on March 2, as the first major city and only regional capital to be captured by Russian forces during the initial invasion. The Russian occupation of Kherson Oblast continued until Ukrainian forces resumed control on November 11, after launching a counteroffensive at the end of August.

On May 4, 2022, we published Tracking shifts in Internet connectivity in Kherson, Ukraine, a blog post that explored a re-routing event that impacted AS47598 (Khersontelecom), a telecommunications provider in Kherson Oblast. Below, we summarize this event, and explore similar activity across other providers in Kherson that has taken place since then.

On May 1, 2022, we observed a shift in routing for the IPv4 prefix announced by Ukrainian network AS47598 (Khersontelecom). During April, it reached the Internet through several other Ukrainian network providers, including AS12883 (Vega Telecom) and AS3326 (Datagroup). However, after the shift, its routing path now showed a Russian network, AS201776 (Miranda-Media), as the sole upstream provider. With traffic from KhersonTelecom passing through a Russian network, it was subject to the restrictions and limitations imposed on any traffic transiting Russian networks, including content filtering.

The flow of traffic from Khersontelecom before and after May 1, with rerouting through Russian network provider Miranda-Media, is illustrated in the chart below. This particular re-routing event was short-lived, as a routing update for AS47598 on May 4 saw it return to reaching the Internet through other Ukrainian providers.

As a basis for our analysis, we started with a list of 15 Autonomous System Numbers (ASNs) belonging to networks in Kherson Oblast. Using that list, we analyzed routing information collected by route-views2 over the past year, from February 1, 2022, to February 15, 2023. route-views2 is a BGP route collector run by the University of Oregon Route Views Project. Note that with respect to the discussions of ASNs in this and the following section, we are treating them equally, and have not specifically factored estimated user population into these analyses.

The figure below illustrates the result of this analysis, showing that re-routing of Kherson network providers (listed along the y-axis) through Russian upstream networks was fairly widespread, and for some networks, has continued into 2023. During the analysis time frame, there were three primary Russian networks that appeared as upstream providers: AS201776 (Miranda-Media), AS52091 (Level-MSK Ltd.), and AS8492 (OBIT Ltd.).

Within the graph, black bars indicate periods when the ASN effectively disappeared from the Internet; white segments indicate the ASN was dependent on other Ukraine networks as immediate upstreams; and red indicates the presence of Russian networks in the set of upstream providers. The intensity of the red shading corresponds to the percentage of announced prefixes for which a Russian network provider is present in the routing path as observed from networks outside Ukraine. Bright red shading, equivalent to “1” in the legend, indicates the presence of a Russian provider in all routing paths for announced prefixes.

In the blog post linked above, we referenced an outage that began on April 30. This is clearly visible in the figure as a black bar that runs for several days across all the listed ASNs. In this instance, AS47598 (KhersonTelecom) recovered a day later, but was sending traffic through AS201776 (Miranda-Media), a Russian provider, as discussed above.

Another Ukrainian network, AS49168 (Brok-X), recovered from the outage on May 2, and was also sending traffic through Miranda-Media. By May 4, most of the other Kherson networks recovered from the outage, and both AS47598 and AS49168 returned to using Ukrainian networks as immediate upstream providers. Routing remained “normal” until May 30. Then, a more widespread shift to routing traffic through Russian providers began, although it appears that this shift was preceded by a brief outage for a few networks. For the most part, this re-routing lasted through the summer and into October. Some networks saw a brief outage on October 17, but most stopped routing directly through Russia by October 22.

However, this shift away from Russia was followed by periods of extended outages. KhersonTelecom suffered such an outage, and has remained offline since October, except for the first week of November when all of its traffic routed through Russia. Many other networks rejoined the Internet in early December, relying mostly on other Ukrainian providers for Internet connectivity. However, since early December, AS204485 (PE Berislav Cable Television), AS56359 (CHP Melnikov Roman Sergeevich), and AS49465 (Teleradiocompany RubinTelecom Ltd.) have continued to use Miranda-Media as an upstream provider, in addition to experiencing several brief outages. In addition, over the last several months, AS25082 (Viner Telecom) has used both a Ukrainian network and Miranda-Media as upstream providers.

In the context of the Internet, “resilience” refers to the ability of a network to operate continuously in a manner that is highly resistant to disruption. This includes the ability of a network to: (1) operate in a degraded mode if damaged, (2) rapidly recover if failure does occur, and (3) scale to meet rapid or unpredictable demands. Throughout the Russia-Ukraine conflict, media coverage (VICE, Bloomberg, Washington Post) has highlighted the work done in Ukraine to repair damaged fiber-optic cables and mobile network infrastructure to keep the country online. This work has been critically important to maintaining the resilience of Ukrainian Internet infrastructure.

According to PeeringDB, as of February 2023, there are 25 Internet Exchange Points (IXPs) in Ukraine and 50 interconnection facilities. (An IXP may span multiple physical facilities.) Within this set of IXPs, Autonomous Systems (ASes) belonging to international providers are currently present in over half of them. The number of facilities, IXPs, and international ASes present in Ukraine points to a resilient interconnection fabric, with multiple locations for both domestic and international providers to exchange traffic.

To better understand these international interconnections, we first analyze the connectivity of ASes in Ukraine, and we classify the links to domestic networks (links where both ASes are registered in Ukraine) and international networks (links between ASes in Ukraine and ASes outside Ukraine). To determine which ASes are domestic in Ukraine, we can use information from the extended delegation reports from the Réseaux IP Européens Network Coordination Centre (RIPE NCC), the Regional Internet Registry that covers Ukraine. We also parsed collected BGP data to extract the AS-level links between Ukrainian ASes and ASes registered in a different country, and we consider these the international connectivity of the domestic ASes.

A March 2022 article in The Economist noted that “For one thing, Ukraine boasts an unusually large number of internet-service providers—by one reckoning the country has the world’s fourth-least-concentrated Internet market. This means the network has few choke points, so is hard to disable.” As of the writing of this blog post, there are 2,190 ASes registered in Ukraine (UA ASes), and 1,574 of those ASes appear in the BGP routing table as active. These counts support the article’s characterization, and below we discuss several additional observations that reinforce Ukraine’s Internet resilience.

The figure above is a cumulative distribution function showing the fraction of domestic Ukrainian ASes that have direct connections to international networks. In February 2023, approximately 50% had more than one (100) international link, while approximately 10% had more than 10, and approximately 2% had 100 or more. Although these numbers have dropped slightly over the last year, they underscore the lack of centralized choke points in the Ukrainian Internet.

For the networks with international connectivity, we can also look at the distribution of “next-hop” countries – countries with which those international networks are associated. (Note that some networks may have a global footprint, and for these, the associated country is the one recorded in their autonomous system registration.) Comparing the choropleth maps below illustrates how this set of countries, and their fraction of international paths, have changed between February 2022 and February 2023. The data underlying these maps shows that international connectivity from Ukraine is distributed across 18 countries — unsurprisingly, mostly in Europe.

In February 2022, these countries/locations accounted for 77% of Ukraine’s next-hop international paths. The top four all had 7.8% each. However, in February 2023, the top 10 next-hop countries/locations dropped slightly to 76% of international paths. While just a slight change from the previous year, the set of countries/locations and many of their respective fractions saw considerable change.

Russia’s share grew by 50% year to 11.6%, giving it the biggest share of next-hop ASes. Germany also grew to account for more than 11% of paths.

Cloudflare observed a rapid growth in Starlink’s ASN (AS14593) traffic to Ukraine during 2022 and into 2023. Between mid-March and mid-May, Starlink’s traffic in the country grew over 530%, and continued to grow from mid-May up until mid-November, increasing nearly 300% over that six-month period — from mid-March to mid-December the growth percentage was over 1600%. After that, traffic stabilized and even dropped a bit during January 2023.

Our data shows that between November and December 2022, Starlink represented between 0.22% and 0.3% of traffic from Ukraine, but that number is now lower than 0.2%.

One year in, the war in Ukraine has taken an unimaginable humanitarian toll. The Internet in Ukraine has also become a battleground, suffering attacks, re-routing, and disruptions. But it has proven to be exceptionally resilient, recovering time and time again from each setback.

We know that the need for a secure and reliable Internet there is more critical than ever. At Cloudflare, we’re committed to continue providing tools that protect Internet services from cyber attack, improve security for those operating in the region, and share information about Internet connectivity and routing inside Ukraine.

As part of the recently passed Infrastructure Investment and Jobs Act (Infrastructure Act) in the United States, Congress asked the Federal Communications Commission (FCC) to finalize rules that would require broadband Internet access service providers (ISPs) display a “label” that provides consumers with a simple layout that discloses prices, introductory rates, data allowances, broadband performance, management practices, and more.

A sample Broadband Nutrition Facts from the original 2016 FCC proposal.

While the idea of a label is not new (the original design dates from 2016), its inclusion in the Infrastructure Act has reinvigorated the effort to provide consumers with information sufficient to enable them to make informed choices when purchasing broadband service. The FCC invited the public to submit comments on the existing label, and explain how the Internet has changed since 2016. We’re sharing Cloudflare’s comments here as well to call attention to this opportunity to make essential information accessible, accurate, and transparent to the consumer. We encourage you to read our full comments. (All comments, from Cloudflare and others, are available for public consumption on the FCC website.)

Six years can change a lot of things, and the Internet is no exception. For example, Tiktok barely existed as a company at the start of 2016; now it is the most popular site in the world. The global population that uses the Internet increased from 3.4 billion people in 2016 to 5.2 billion in 2021, which represents a growth of 52%. According to Statista, users in 2015 spent around 5.5 hours with digital media; now users spend almost 8 hours with digital media. The amount of data consumed on the Internet in 2021 was 79 zettabytes, which is a number that is expected to more than double in only two years. Users are more dependent on the Internet now than ever before.

Users being more dependent on the Internet has been amplified during the pandemic. According to Pew Research, 90% of American adults say the Internet has been essential or important for them personally during the coronavirus outbreak. Forty percent of American adults say they used digital technology or the Internet in new or different ways compared with before the beginning of the outbreak. A home broadband connection is no longer primarily for recreation, but a necessity for equitable access to education, healthcare, and as of 2020, it’s now even essential for many employment opportunities.

With that dependency, though, comes a higher expectation of quality. In 2016, users were more tolerant of poor performance: they were just happy if their Internet worked. Furthermore, applications were typically less latency sensitive: things like VoIP and video chats were less prevalent than they are today. Nowadays, however, video chats are almost ubiquitous: we use them at work and at home with increasing frequency. If these applications are slow or perform poorly, it’s hugely impactful to the user experience. We think of it as “our Internet cutting out,” and we lose the engagement with whomever we’re talking to.

Our increased dependence on the Internet has in turn increased our expectations for good Internet performance.

Because the Internet has become more focused on performance in 2022, we believe that your Internet providers should disclose to you how good they are at providing a good experience for these applications that are now mission critical.

Previously, performance was measured by bandwidth, or the size of the pipe between you and what you want to access. However, bandwidth is much more widely available today than it was six years ago. Median download throughput increased from 39 Mbps in 2016 to 194 Mbps in 2021. This increase in throughput has opened up new uses of home Internet connections, and new opportunities to look holistically at the Quality of Experience (QoE) of home broadband. We believe that metrics beyond bandwidth such as latency and jitter (the variance in latency) have grown appreciably in importance and that should be reflected in policy going forward.

Transparency into broadband Internet performance isn’t just important to consumers, though. With more and more enterprises relying on the Internet to reach both customers and also employees, it has become a foundational part of the American economy. So many businesses rely on Cloudflare because they want their digital assets delivered to customers, partners, and employees quickly. Enterprises want to secure their network with our cloud because our edge services are physically close to users and can be reached with low latency. Performance is no longer a luxury — it is increasingly a necessity.

The FCC defined latency in 2016 as “the time it takes for a data packet to travel from one point to another in a network.” While technically true, the vagueness of this definition presents certain issues. The latency between two points could be arbitrary, or as is the case with current speed tests, measuring a path that is never traversed by consumers in daily Internet usage. To put it succinctly: we don’t know what is being measured or whether what’s measured reflects reality.

While there is ambiguity about what latency ISPs would show on their broadband label, Cloudflare, and other content providers, can see latency from the other side – from our edge servers that are serving websites to consumers. What we see is that rural states have higher latency than more dense states.

Figure 1: 50th percentile TCP Connect Time (ms) to Major Content Delivery Networks

*Alaska and Hawaii have TCP Connect times of 263ms and 160ms respectively. Data compiled by Cloudflare from the HTTP Resource Timings API‌‌

As an example, Cloudflare offers a browser isolation product that runs a web browser in our cloud, an application that is extremely sensitive to latency. To achieve these latencies, we’ve connected directly with 10,000 distinct networks across more than 270 global data center locations. We estimate that 95% of Internet users globally can reach Cloudflare-protected websites and services in under 50 milliseconds.

So while Cloudflare supports the FCC’s effort to increase understanding of cost and privacy of Internet Service Provider offerings and wants the labels to be expedited to provide real consumer value, we have suggestions to significantly augment the labels to provide a better view of how your Internet does at providing services to you. Standardizing technical measurements across the Internet is a big topic, and in some cases we suggest the FCC build stakeholder consensus on additional future changes to the label.

For the broadband performance section of the label, we recommended:

• Renaming “download speed” (and “upload”) to “throughput,” “bandwidth”, or “capacity.” We can’t deny “speed” has become conversationally interchangeable with throughput, but they aren’t the same. As the Internet continues to grow, “speed” will mean how fast the Internet is, which will be measured in latency and overall quality of service, not just throughput. The latter is simply the amount of bits a connection can handle in the downstream direction at any given time.

• Adding “jitter” to the label. With the pandemic-driven rise of video conferencing, jitter —the variation, or stability, of latency in an Internet connection—has become a common cause of issues. Found yourself saying “my Internet is cutting out” or “am I frozen? Oh, I’m back”? That’s likely jitter.

• Add methodological transparency and work towards standards for how latency, jitter, and packet loss are measured. Consumers should be able to make apples-to-apples comparisons between ISP offerings, but to do that,  a standard in how ISPs measure these numbers is needed. Rather than a hasty mandate from the FCC, our suggestion is to take the time to engage stakeholders on the best approaches.

The end goal of these recommendations is to make sure that standards on performance match the experiences users have on the Internet. Today, speed tests and other forms of Internet measurement often query endpoints that are embedded into ISP networks that don’t see any traffic beyond measurements, and this can produce misleading results that may lead users to think that their Internet experience is better than it actually is. If your measurements don’t follow the same paths and are treated the same as normal Internet traffic, your measurements will look better. We believe that performance measurements should closely approximate the user experience, so that you have the complete picture of how your Internet is performing.

However, network performance isn’t only about how well your provider takes bits from your device to where they need to go. Sometimes network performance can be impacted by network management techniques. Providers may institute techniques like traffic shaping, which will slow down traffic to and from specific high-bandwidth sites to ensure that other sites don’t see congestion and degraded performance. Other providers may implement bandwidth caps, where specific users who consume lots of data may be slowed down if they exceed a threshold, a technique commonly used for mobile networks.

To help address these issues, we recommended including policy level line-items in the network management section instead of merely a yes-or-no answer. For example, if an ISP slows traffic after a certain amount of data has been consumed in a month, that information should be accessible on the label itself.

For the privacy section of the label, our recommendation is that a link to a dense and rarely-read ISP privacy policy is not sufficient transparency into how an ISP will use subscribers’ data. We recommended a privacy section that gives consumers insight into:

• Collection and retention of information: The label should indicate whether the ISP collects and retains any information beyond what is strictly necessary to provide services to the subscriber, including web browsing history and location data, as well as how long that information is retained.

• Use of information: The label should indicate whether data collected by the ISP is used for purposes other than what is strictly necessary to provide the broadband service to the consumer, such as for advertising.

• Sharing of information: The label should indicate whether the ISP shares or sells the data collected, including location or browsing information data, with third parties.

• Opt out: The label should indicate whether the ISP provides options to opt-out of data use and sharing (whether the ISP receives consideration for such sharing).

• Security of information: The label should indicate whether the ISP provider has technical mechanisms in place to secure data from unauthorized access, including whether it encrypts metadata about a consumer’s browsing habits, and mechanisms in place to report breaches.

We also suggested that the FCC make the data presented in the label accessible in a machine-readable format for researchers and consumers.

We commend Congress for including broadband nutrition labels in the Infrastructure Investment and Jobs Act, and the FCC for moving quickly to implement the labels. The current broadband label, the product of years of work, will be a significant improvement over what we have now – nothing.

However, we don’t believe that the labels should stop there. While the labels from 2016 go a long way towards providing clarity into how much money users pay for their Internet and create a good standard for pricing, the Internet and the way people interact with it is so different now than it was six years ago. We need to ensure that we are representing the user experience to its fullest, as this will ensure that our Internet experience can continue to improve over the next six years and beyond.