Python script for finding blocked sites on Indian ISPs

Messages
1,940
Location
Delhi
ISP
TriplePlay/Excitel
Now I'm running a the search on 260 million domains. The csv file was ~4GB so I had to split it to ~130 smaller files with 2million domains each.

After ~14hrs of running it has already found ~620 domains which was kinda surprising considering the large amount to be checked. I'm running the same on another Oracle VPS to make sure no domain is missed.
 
Messages
1,940
Location
Delhi
ISP
TriplePlay/Excitel
260 million search is only partly done but i've combined all the blocked domains from both vps+all the blocked domains from earlier search (1mill, 10mill, adult gambling sites etc) and removed subdomains and duplicates. Right now its at 12896 blocked domains.

 
Messages
1,940
Location
Delhi
ISP
TriplePlay/Excitel
This is not completely related to the original topic but I thought it was worth mentioning.

Apparently Mikrotik RouterOS has a feature to match packets by TLS Host. This can be used to easily policy route blocked sites over a VPN without relying on IP Based policy routing (by resolving blocked sites using DNS) which can have issues because of subdomains, ip changes etc. Its a shame that this feature isn't available on PFSense.

 
Messages
1,940
Location
Delhi
ISP
TriplePlay/Excitel
Ok so one of the oracle server is done.


Its 20350 domains. That said I think it did miss a few domains that should be picked up by the other VPS.

I'm trying to run the same on my homeserver on TriplePlay. My scripts tries connecting to 1.1.1.1 with various domains from the sites list.

And on TriplePlay, 1.1.1.1 is routed through Jio. This means I'm hitting Jio's middleboxes whenever a site is blocked. I've been finding some odd blocked sites. For example "fantasyfacesbybelle.com.au" is blocked on Jio for some reason. It makes zero sense for the site to be blocked, its not an Indian site, isn't pornographic or promote anything illegal, doesn't violate trademarks. Anything. But when I open it on TriplePlay it gives me connection reset (on Excitel it opens fine).

So it only makes sense that Jio is blocking sites that government doesn't even want blocked!

Edit: It appears that fantasyfacesbybelle.com.au was in the past a torrent site, here is a archived version of page from 2018 Torrent Downloads - download free torrents! which explains why it was blocked lol (strange name for a torrent proxy site, might have been trying to evade blocking). But still, its only blocked on Jio transit and not blocked on Excitel or on Namecheap VPN (TATA Transit).
 


Last edited:
Messages
1,940
Location
Delhi
ISP
TriplePlay/Excitel
Can anyone on Jio access tataskysales.com ? This site I think is blocked on Jio but not on Airtel.


I also found certain sites on Airtel transit which were blocked on http but not on https.
 

Similar threads