Отключение интернета в Мары, Туркменистан, апрель-май 2020

Internet censorship all around the world Turkmenistan
2

@ValdikSS’s post was too long for the forum’s auto-translation. Here is the text translated using DeepL Translate: The world's most accurate translator.

Internet blackout in Mary, Turkmenistan, April-May 2020

In the evening of April 27, 2020, a strong hurricane with heavy downpours took place in Lebap and Mary regions of Turkmenistan, which fell trees, demolished roofs of houses and flooded various buildings. Over the past week, the authorities of Turkmenistan kept silent about the incident without reporting it to the official media.
Residents of Mary, Chardzhou, and Daşoguz Oblasts were disconnected from the Internet on April 28, 2020, the day after the storm, after photos and video footage from the scene appeared.

The website hronikatm.com reports the following:

The organization reminds that there is no freedom of media in the country - state media are dominant, while the authorities block most independent and Western media.

The authorities have tried to prevent residents from publishing visual documents documenting the destruction of the hurricane. Prague-based human rights group “Rights and Freedoms of Turkmen Citizens” managed to talk to a woman who was detained by MNB for two days along with 29 others, accusing them of sending the video “abroad”. For the same reason, DHS detained 19 other women in Turkmenabat, releasing them May 3. "The Turkmen Human Rights Initiative (TIHR) said police are watching people in cars taking pictures of injuries on their mobile phones.

Censorship and efforts by the Turkmen authorities to prevent the damage from being made public make it difficult to accurately assess the damage. Radio Liberty reported 30 deaths. Turkmen news talked to a medical official who reported 300 deaths in Turkmenabat. Most buildings in the city have been damaged. Small towns are also heavily damaged by winds and floods. A video of houses without roofs and severely damaged inside is being distributed.

I had the opportunity to get an Internet channel with access from outside Turkmenistan, and to test availability in the conditions of an Internet shutdown in Mary. The provider is TMTelecom, a home wired connection via ADSL.
The Turkmen, who provided the Internet channel, reports that the inaccessibility of sites is observed in 3 regions of the country, on wired and mobile Internet, from which I conclude that the government deliberately shut down the Internet. I have not been able to find any public notes about it anywhere on the Internet, including sites specializing in news from Turkmenistan.

Briefly: technical IP connection with other countries is preserved, but access to virtually all Internet sites and services is blocked using DPI system. Sites are not opened, applications do not work.

DNS

The operation of DNS, compared to the usual situation in Turkmenistan, has not changed: popular third-party public DNS resellers are partially unavailable due to blocking the ranges of large hosting providers to Internet companies over IP ranges (a typical situation for Turkmenistan, not a feature of this week).

Not working: 1.1.1.1, 8.8.8.8, 9.9.9.
Working: 77.88.8.8, 193.17.47.1, 185.43.135.1

Requests for IP addresses for some domains are subject to DNS spoofing, at any IP address and UDP port.
For example, the rutracker.org request returns 127.0.0.1 after 77.88.8.8:

# host rutracker.org 77.88.8.8
Using domain server:
Name: 77.88.8.8
Address: 77.88.8.8#53
Aliases: 

rutracker.org has address 127.0.0.1
rutracker.org has address 127.0.0.1
rutracker.org has address 127.0.0.1

A similar situation occurs when attempting to query this domain on port 1253, a little-known undocumented port on which Yandex.DNS 77.88.8.8 is running.

HTTP and HTTPS

In Internet Shutdown conditions the site can be more or less accessible via HTTP, but not via HTTPS, and vice versa (!).
At attempt to open a site, connection either is not accepted (as if there is no network connection, the answer to TCP SYN does not come), or is accepted, but breaks off or “freezes” (stops coming reply TCP ACK and any other packets).
There can be a delay of 10-12 seconds between receiving a connection and starting data transfer. This is probably due to the DPI connection checks.
Ports 55, 88, 89, 1080, 3128 are blocked (this is a typical situation) and the connection to them is always reset (TCP RST), regardless of the destination IP address. Some (high) ports stop working at times, the pattern is not detected.

These do not work (they may occasionally respond in some way, particularly with unencrypted HTTP, but they never open until the end):

These work:

  • google.com (actually the only domain more or less normally operating over HTTPS that has been detected, with subdomains and other services like gmail not working)
  • online.tm and telecom.tm (provider sites)

An example of a gismeteo site not working. An HTTP request is accepted, but the connection is broken:

* Connected to www.gismeteo.ru (185.134.203.108) port 80 (#0)
> GET / HTTP/1.1
> Host: www.gismeteo.ru
> User-Agent: curl/7.64.0
> Accept: */*
>
* Empty reply from server
* Connection #0 to host www.gismeteo.ru left intact
curl: (52) Empty reply from server

Over HTTPS:

*   Trying 185.134.202.21...
* TCP_NODELAY set
* Connected to www.gismeteo.ru (185.134.202.21) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):

And so to infinity, the packets stop moving.

Example of a yandex connection. An HTTP connection is accepted, but no response is received, and the connection “hangs”. I forged the headers on the browser to exclude curl filtering:

* Connected to yandex.ru (77.88.55.55) port 80 (#0)
> GET / HTTP/1.1
> Host: yandex.ru
> Accept-Encoding: deflate, gzip
> User-Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0
> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
> Accept-Language: ru,en;q=0.7,en-US;q=0.3
> DNT: 1
> Connection: keep-alive
> Upgrade-Insecure-Requests: 1
> Pragma: no-cache
> Cache-Control: no-cache
>

Attempt to open mail.yandex.ru via HTTPS:

*   Trying 77.88.21.37...
* TCP_NODELAY set
* Connected to mail.yandex.ru (77.88.21.37) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):

Similarly, no further responses are received from the server.

Other protocols

The situation with other protocols is generally similar to HTTP/HTTPS. The host can somehow be accessed via HTTP (TCP port 80), but it will not be possible to connect to it via SSH: either the connection will be broken or “hanging”.

root@debian-tm:~# ssh example.com -vvv
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1d  10 Sep 2019
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolve_canonicalize: hostname example.com is address
debug2: ssh_connect_direct
debug1: Connecting to example.com [1.2.3.4] port 22.
debug1: connect to address 1.2.3.4 port 22: Connection refused
ssh: connect to host 1.2.3.4 port 22: Connection refused

UDP allows traffic from 7 to 10 seconds, after which the packets are no longer sent both ways on this bundle of client and server UDP ports. About 200 KB of data can be transmitted in 10 seconds. Presumably, the blocking is designed to leave DNS, NTP and other important UDP traffic running.

ICMP packets are generally blocked for hosts outside the country, only the (conditionally) first request and response (of any ICMP type) is delivered, probably to avoid ICMP tunnels. Because of this, ICMP Echo Request/Echo Reply (better known as ping) manages to get a maximum of one in a few tens of seconds (when the system “forgets” the first packet sent). The provider’s site is normally “pinged” in this case.

On TCP in general most often connection is established normally, but only the first kilobyte of the traffic after connection establishment is transferred or accepted, then packets stop to go in both parties (that causes “hanging” of connection on different protocols). The problem is not caused by MTU: I reduced TCP MSS to low values and made sure that even small packets (<400 bytes) stop transmitting.
I assume that this is how DPI works: up to 1 KB of response traffic from the server is analyzed and the system decides whether to allow or block the request.
To confirm this assumption, I redirected traffic from my server on the Internet to www.google.com on port 443, and connected to the IP server as if it were Google, over HTTPS - it worked, the traffic went both ways normally, the connection did not stop working after 1 KB of transmitted or received data. Apparently, the DPI in this kilobyte expects an X.509 (HTTPS) certificate from a trusted site (www.google.com in this case).
I tried to tamper with the Google certificate by making an exact copy of it (with the same fields), but it looks like the entire body of the certificate is being checked, not just the headers.

How to get the Internet when it’s turned off

Despite the government’s disconnection of the Internet and the inoperability of sites, the region still technically maintains IP connectivity to the external Internet.

There are at least two working ways to get the Internet on the channel provided to me:

  • Since at least one site (www.google.com) is still accessible and the site is checked by domain and certificate, but not IP address, you can use Cloak from @antikythera to create a tunnel disguised as a Google site.
  • Due to the stability of DNS, you can, as a last resort, organize a DNS-tunnel program iodine: it is slow and unstable due to its specifics, but enough to connect to the server via SSH.
  • TCP packets SYN and SYN-ACK (start of connection and start response) continue to be delivered generally stable, you can organize a tunnel based on them. I don’t know the ready-made programs for this, but it is probably possible to adapt udp2raw.

Conclusions

In conditions of the Internet shutdown caused by actions inside the country, instead of malfunctions of transit channels, it is still possible to find ways of access to the Internet, but for this purpose it is necessary to possess profound knowledge in network technologies, a set of programs and understanding of principles of their work, and also in advance prepared server out of the country.

The most reliable backup method of communication with a foreign server is, in my opinion, the DNS-tunnel - the performance of DNS, with a high probability, will be supported by providers for the longest time, and therefore, you can use a pre-configured DNS-tunnel for further configuration of the foreign server in the absence of full network connectivity.