-
Amazon Deals - ToS - Warp
Reliance Jio to construct 2 other submarine cable system after BBG and AAE-1
- Thread starter swatkats
- Start date
- Replies 34
- Views 4,428
Let me try to answer that while re-sharing only thing which is already in public domain.
The answer to "Total Indian bandwidth" greatly depends on what you are counting. One may be tempted to just look at submarine cable circuits and find total bandwidth there. But that will far from actual country level bandwidth usage number because so much of caching happens locally. There are thousands and thousands of caching server of Google GGC, Facebook FNA, Netflix OCA, Akamai, etc besides caching PoPs of larger content holding networks like Cloudflare, Fastly, Limelight, Telegram etc. So most of cached content never hits the submarine cable network (it just hits for cache fill). To get a sense of how deep is the caching, you can look at my post on Facebook FNA map, list across India here. This is for Facebook alone but I can imagine Google won't be that different.
Total bandwidth including what is flowing on submarine + locally cached + locally peered etc should be well beyond 100Tbps. For TRAI or anyone it's a hard number to get because one can easily have multiple counts as for one provider x 10G flows are on IP transit, for other they might be on sold wave length and hence the counting challenge.
Some hints here, look for talk by my good friend and VP at Tata Comm - Mr Shailesh Gupta at INNOG 2 in 2019. His talk was on their journey from 64Kbps to 2Tbps and we invited him as a keynote speaker (slides here and video here). Slide 7 gives a hint on submarine capacity and in the video check from here where he mentions "It's around 1Tbps for India, 20Tbps globally for them. Large operators together have 3.5 - 4Tbps".
- Messages
- 391
- Location
- Bhubaneswar
- ISP
- AS140171, AS141804, AS23860, AS24186 (Former), Jio 5G (Mobile), Airtel 5G (Mobile)
But I thought TATA was the best in case of International routing? I'm pretty sure Airtel is the best Nationally but I hoped TATA was the better Tier 1 ISP here.
- Messages
- 391
- Location
- Bhubaneswar
- ISP
- AS140171, AS141804, AS23860, AS24186 (Former), Jio 5G (Mobile), Airtel 5G (Mobile)
Lolita_Magnum
ᓚᘏᗢ
I think latencies will improve since, right now for all overseas Jio Connection the data is being routed through Jio Singapore Exchange.
Hence, overall throughput and communications with international servers should improve too, since hops will decrease that will be more direct connectivity.
Hence, overall throughput and communications with international servers should improve too, since hops will decrease that will be more direct connectivity.
It depends on destination but typically for networks with large traffic volumes you will find Airtel, Jio etc (in Indian context) doing better than Tata Comm specifically in terms of latency and here's why:
Tata Comm AS6453 is a transit free (sometimes also called as tier1 network). They are one of 15-16 networks (list here on Wikipedia) in the world which can reach all networks directly (as they are it's customer) or indirectly as they are customer of it's remaining 15 peers. So typically a transit free network peers only with another large transit free network and would typically have a closed peering policy. So one side here is their (downstream) customer in India like say Siti, ACT, Excitel, etc. Other side is either their peerings with other large transit free networks or simply their customer in US/Europe. This is quite different from a network like Jio, Airtel etc which would have more relaxed peering policy and infact would actively reach out to large networks outside of India to peer (for free). They would pick traffic in US/Europe/Singapore via direct peering (PNI) over internet exchanges.
So if I am a content player and have to send traffic to a customer of Tata Comm (and other similar large transit free networks with closed peering policy) I would have to pay Tata Comm directly or have to pay any other large network which peers with them. This can make routing quite bad in certain areas. Transit free networks typically do not peer at any public exchange (they stay there to sell IP transit over the IX but do not peer openly) and that further degrades the quality. This reduces the "entry points" in a given backbone. Plus people typically have more capacity towards their peers compared their transit.
So here's a real world comparison:
innog.net (164.52.202.232) that's routed behind Tata Comm AS4755 behind AS6453 while IP of my provider in Haryana (IAXN) - 202.9.120.253 is routed behind Airtel. Let's check latency from all RIPE Atlas probes in Singapore towards each of them. Expected latency should be less than 90ms as both end points are within Delhi NCR region.
RIPE Atlas Singapore -> innog.net via Tata Comm (results here)
16 out of 50 probes had a latency of over 100ms. 5 out of 50 probes have over 150ms latency.
RIPE Atlas Singapore -> 202.9.120.253 via Airtel (results here)
2 out of 50 probes had over 200ms latency. Rest all with less than 82ms latency.
Why this difference - because aggregate of IAXN's IP (202.9.120.0/24) is visible at Equinix Singapore to all these networks while for innog.net covering aggregate (164.52.202.0/24) entry has to be a customer of Tata Comm or customer of a peer of Tata Comm within Singapore or a customer of customer of Tata Comm etc. Another factor is that within India both Airtel & Jio are large eyeball networks. They hold end customers and hence many content networks optimise their routing with them & focus on them instead of a backbone transit/enterprise network like Tata. Take case of Cloudflare - they have nodes inside Airtel, Jio etc across India and typically latency is way lower on Airtel/Jio.
Here's trace to Cloudflare 1.1.1.1 from innog.net server from Noida:
anurag@host01:~$ traceroute 1.1.1.1
It goes to Mumbai even when via Airtel it is much lower. Here's what I get from their looking glass DEL1 PoP:
With that being said please remember that peering, interconnection etc is one element of the whole connectivity picture. There are many other things including capacity on transport, submarine cable agreements, overall finances, middle mile network, last mile delivery network etc. All that impacts numbers, uptime etc. My experience has been that Tata Comm is much better engineered backbone. Typical latency between point A to B within their network - Tata Comm is slightly lower than Airtel. Plus also the fact that a large part of traffic these days concentrates on a few ASNs and hence latency optimisation has diminishing returns once you have good routing to say top 20-30 ASNs in terms of traffic. Airtel/Jio etc would typically have better routing within Asia but amount of traffic flow is actually lower. (As a network engineer I would still love to see it being optimised though for peer to peer VoIP traffic etc) 🙂
I my city in Haryana I have 14-15 options for FTTH and actually I feel worried by fact that 10 out of them run over Airtel circuits and it's becoming a massive single point failure. It's not a Airtel issue but the fact that they are more aggressive in wholesale market & in many cases are the only option. The ones not on Airtel are: Jio fibre, BSNL, Railwire LCO and Excitel (strong guess as they would typically prefer Tata Comm). Rest everyone is taking Airtel circuits for transit as well as peering.
Disclaimer: Posting here in my personal capacity and I do work for a global IP backbone which does follows a open peering policy.
opomiyulen
🇮🇩🇬🇧🇹🇷🇩🇪
Even i think so, they are using STT GDC India for NOC/datacenters need, which is a Tata company. Here is Excitel CEOs message on STT Website. Seems like they have outsourced most of things in their business or may be other smaller isps follows the same practice.
Edit: STT GDC is not a Tata company. They sold their data centers business in 2016 to STT GDC, although Tata still have 26 percent stake in STT GDC data centers business in India.
Last edited:
Just important to mention here - STT is not a Tata Comm's company. Basically Tata Comm / VSNL sold it's datacenters to STT around India, Singapore etc - ST Telemedia Global Data Centres Acquires Tata Communications’ Remaining 26% Stake in Its Data Centre Business in Singapore
Many of these business whether it's wholesale bandwidth, datacenter capacity, storage, compute is becoming business of economies of scale.
Many of these business whether it's wholesale bandwidth, datacenter capacity, storage, compute is becoming business of economies of scale.
Last edited:
Thanks a lot for the Inputs @Anurag Bhatia 🙂
Do you have some idea as to why the Bandwidth charges are higher for the Indian servers. Scene doesn't seem to Improve from the last 5 years, Even the Internet prices have dropped low?
I can get 100mbps unshared bandwidth for a server in the US and for that price in India it is just 10mbps bandwidth, It is 10x high still.
I think Jio needs to setup DC or start providing these existing DC's with ultra cheap bandwidth.
Do you have some idea as to why the Bandwidth charges are higher for the Indian servers. Scene doesn't seem to Improve from the last 5 years, Even the Internet prices have dropped low?
I can get 100mbps unshared bandwidth for a server in the US and for that price in India it is just 10mbps bandwidth, It is 10x high still.
I think Jio needs to setup DC or start providing these existing DC's with ultra cheap bandwidth.
I can check different ISP and DC's bandwidth upstream providers. Most of them prefer Airtel.
Railwire also uses Airtel Backbone.......!!!!
You are welcome.
Regarding expensive bandwidth at Indian datacenter - I think primarily it has less to do with ISPs and more to do with overall lack of competition in datacenter as well as lack of ecosystem of smaller providers who are competitive for smaller scale capacity (few U's of colocation, dedicated servers and low end VMs).
What we have got are the hyperscalers like Google Cloud, AWS, Azure etc. They offer nice uptime, very high resiliency but aren't really into plain vanila compute that much. Infect on the question of bandwidth they are most expensive ones. As more & more datacenters are built, eventually they would have a fair amount of business proposition for the other tier of hosting players which would have US/Europe like bandwidth pricing. If you see the pricing DigitalOcean Bangalore, Linode in Mumbai has comparable pricing to their offerings outside of India.
Also for cheap 100Mbps bandwidth in US you are certainly looking at competitive providers as there's a fair amount of business for the competitive providers in the US. Better regulation, fact that there was lot of dark fibre being put before the .com bubble burst in 2000s. So you see there are few dozen options from world's largest telcos to relatively small competitive providers and people pick subject to their requirement. For large telcos (India, US and everywhere else) a large focus remains on mobile spectrum, last mile access etc and small datacenter customers is extremely niche market for their focus (so far). In other words they focus very much on businesses with very high user base (mobile, retail fixed line etc) and business with with less users but very high ARPU's like enterprise. And hence then pricing reflects it.
Another factor here is that most of small city/state level ISPs are typically are not allowed to pull their own dark fibre to these datacenters (with some handful exceptions here and there). Look for datacenter & small ISPs in any Indian city - Mumbai, Noida or Chennai on PeeringDB. Call them & check how many of those hundreds of small ISPs have their own dark fibre in the datacenter. Large number of these providers are end user serving and thus are inbound heavy. Their outbound side on circuits are barely used (typically 1:5 ratio). Lack of those providers again circles back to same issue - lack of competition, investment & options in the datacenter industry. On good part the investment in datacenters in India has suddenly spiked up and as those projects are completed, there would be a massive datacenter capacity hoping leading to more competition.
Full disclosure: I am on board of an Indian Cloud hosting company as independent member
Let me clarify a bit: Railtel uses Airtel as it's transit provider and that's a important but small part of overall network.
Railwire/Railtel would typically take aggregated capacity in a 2-3 cities and from there it just carries it all on it's fibre which is laid across the tracks and hands it off to local LCO who delivers it upto the home.
Thus:
1. Traffic flows for a large part on Railtel's own fiber (52000km+) across India
2. They do peer at various exchanges and that would be covering a very large of their traffic. Probably somewhere in 60-70% range.
3. They do have caching node of Facebook (my blog post with data here) and I can imagine they would have same for Google, Netflix, Akamai etc. Thus another 10-15% traffic atleast is on-net and cached for them and would typically not leave their network.
4. Cost of wholesale bandwidth is much lower at aggregate scale in multiple of 10G's and they can always switch their provider to anyone else without much of an effort. <- That matters as while their customers in many place would have limited options, they would have a fair amount of options for wholesale bandwidth purchases.
Additional they do sell capacity to many other Indian networks and thus you may not realise when a part of your call or traffic flow might be flowing over their fibre. 37% of their total revenue of 1100+ crore came from NLD services (which is double of their ISP service revenue which was at 15%). Checkout their 2019 annual report here. ISP revenue % must have changed during 2020 due but this does gives an idea of how large scale leasing of capacity to other players.
Note: This page gives a brief idea of their network. Poor quality of images but still quite a detailed one.
Edit: Added note about their network page.
Last edited:
