Blocking of archive.org in India, circa 2017-08-05

Analysis of the blocking of archive.org in India between 2017-07-20 and 2017-08-30, using data from OONI, the Open Observatory of Network Interference.

Related links:

David Fifield <david@bamsoftware.com>


Updated
Updated
Updated

Analysis

Gray dots are normal (unblocked) replies. Colored dots indicate some kind of error.

Consistent with reports in the blog post, only blocking of HTTP was detected, not HTTPS. Excluding one ambiguous measurement on 2017-08-03, the first measurement with evident blocking is at 2017-08-05 09:35:47.

These autonomous systems exhibited blocking (and also had an unblocked measurement prior to 2017-08-05):

AS24560 AIRTELBROADBAND-AS-AP Bharti Airtel Ltd., Telemedia Services (Netsweeper block page)
AS45609 BHARTI-MOBILITY-AS-AP Bharti Airtel Ltd. AS for GPRS Service (connection reset)
AS55836 RELIANCEJIO-IN Reliance Jio Infocomm Limited (Netsweeper block page)

These autonomous systems showed no HTTP blocking after 2017-08-05:

AS17488 HATHWAY-NET-AP Hathway IP Over Cable Internet
AS17803 BSES-AS-AP BSES TeleCom Limited
AS133982 EXCITEL-AS-IN Excitel Broadband Private Limited

These autonomous systems had no HTTP measurements after 2017-08-05, so their blocking status is unknown. I am not sure why AS4134 appears once in the data (that AS should be in China, not India).

AS4134 CHINANET-BACKBONE No.31,Jin-rong Street
AS9829 BSNL-NIB National Internet Backbone
AS45820 TTSL-MEISISP Tata Teleservices ISP AS
AS55410 VODAFONE-NET-AS-AP C48 Okhla Industrial Estate, New Delhi-110020
AS55824 NKN-CORE-NW NKN Core Network

AS24560: Bharti Airtel Ltd., Telemedia Services

In AS24560, requests for http://archive.org received an explicit block page:

Screenshot of http://www.airtel.in/dot/ with an iPhone 6 SE User-Agent
<meta name="viewport" content="width=device-width,initial-scale=1.0,maximum-scale=1.0"/><style>body{margin:0px;padding:0px;}iframe{width:100%;height:100%}</style><iframe src="http://www.airtel.in/dot/?dpid=1&dpruleid=3&cat=107&dplanguage=-&url=http%3a%2f%2f207%2e241%2e224%2e2%2f" width="100%" height="100%" frameborder=0></iframe>

The page at the embedded URL, http://www.airtel.in/dot/?dpid=1&dpruleid=3&cat=107&dplanguage=-&url=http%3a%2f%2f207%2e241%2e224%2e2%2f [archive], says "Your requested URL has been blocked as per the directions received from Department of Telecommunications, Government of India. Please contact administrator for more information."

The format of the URL shows that the censorship device is made by Netsweeper, a company that is notorious for aiding censorship. Incidentally, essentially the same blocking [archive] has been documented on Airtel going back at least to July 2016.

The block page includes advertisements for ISP-affiliated movies, music, and games. The ads only appear if you are using a mobile browser (specifically [archive], if the browser User-Agent matches Android|webOS|iPhone|iPad|iPod|BlackBerry|IEMobile|Opera Mini).

The page also includes some kind of Akamai-hosted performance-measuring script [archive] based on episodes.

Raw OONI measurements:

AS45609: Bharti Airtel Ltd. AS for GPRS Service

In AS45609, a request for http://archive.org was terminated with a TCP connection reset.

Raw OONI measurements:

AS55836: Reliance Jio Infocomm Limited

In AS55836, requests for http://archive.org received a Netsweeper block page, similar to the one in AS24560:

<meta name="viewport" content="width=device-width,initial-scale=1.0,maximum-scale=1.0"/><style>body{margin:0px;padding:0px;}iframe{width:100%;height:100%}</style><iframe src="http://49.44.18.34:8080/webadmin/deny/index.php?dpid=1&dpruleid=3&cat=107&dplanguage=-&url=http%3a%2f%2farchive%2eorg%2f" width="100%" height="100%" frameborder=0></iframe>

The URL it embeds, http://49.44.18.34:8080/webadmin/deny/index.php?dpid=1&dpruleid=3&cat=107&dplanguage=-&url=http%3a%2f%2farchive%2eorg%2f, timed out when I tried to access it.

A earlier measurement on 2017-08-03 is also marked "http-failure" because of a timeout. But the simultaneous control measurement also had an error (HTTP status code 503), so it could be a temporary server failure, not blocking.

Raw OONI measurements:

Update 2017-08-14

Another two weeks later, we find that blocking Three days later, new measurements show that blocking has ceased in AS24560 and AS55836. In AS45609, there have been no new measurements since the blocked one at 2017-08-08 15:07:14, so it is unknown whether blocking continues.

Raw OONI measurements:

Update 2017-08-30

Three days later, new measurements show that blocking has ceased in AS45609, but after a general break in blocking, there is new blocking AS17488 and AS17813, which were previously unblocked.

AS17488 HATHWAY-NET-AP Hathway IP Over Cable Internet (block page)
AS17813 MTNL-AP Mahanagar Telephone Nigam Limited (block page)

AS17488: Hathway IP Over Cable Internet

In AS17488, a request for http://archive.org got a block page (the same as in AS17813):

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
  <head
  <title>
  </title>
</head>
<body>
  <h0>
    <font color="black">
      <font size="4.5">
        “This website/URL has been blocked as per instructions from Department of Telecommunications of India.”
      </font>
    </h0>
  </body>
</html>

Raw OONI measurements:

AS17813: Mahanagar Telephone Nigam Limited

In AS17813, a request for http://archive.org got a block page (the same as in AS17488):

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
  <head
  <title>
  </title>
</head>
<body>
  <h0>
    <font color="black">
      <font size="4.5">
        “This website/URL has been blocked as per instructions from Department of Telecommunications of India.”
      </font>
    </h0>
  </body>
</html>

Raw OONI measurements:

How to reproduce

Prerequisites:

archive.org-in-20170830.zip
Source code and data
ooni-sync
OONI report downloader
jq
JSON processor
R, ggplot2, data.table
For graphing

Download recent OONI reports from India:

ooni-sync -directory reports.in -xz since=2017-07-20 until=2017-08-30 test_name=web_connectivity probe_cc=IN

Filter reports into a CSV file:

(echo date,report_id,country,asn,url,blocked,blocking_type; xz -dc reports.in/*.json.xz | jq -j -f makecsv.jq --unbuffered) | tee in.csv

Generate a graph (blocking-in.png):

Rscript blocking-in.R