BYOD for 30,000 people creates extraordinary network demands
Ottawa, Canada — Avaya engineers are putting the final touches on a network capable of handling up to 54Tbps of traffic when the Winter Olympics opens on Feb. 7 in the Russian city of Sochi.
wifi network
Sochi itself is a sprawling city of 350,000 people located on the Black Sea. Archeologists have found human remains in the area that date back tens of thousands of years. Today, Sochi’s subtropical climate makes it a popular Russian tourist spot.
The two locations where the Olympics will take place — the Olympic village in Sochi and a tight cluster of Alpine venues in the nearby Krasnaya Polyana Mountains — are completely new construction, so this project represents a greenfield environment for Avaya.
In addition to investing in a telecom infrastructure, Russia is spending billions of dollars to upgrade Sochi’s electric power grid, its transportation system and even its sewage treatment facilities. (Watch a slideshow version of the story.)
“The whole town is nothing but a constantly changing, $50 billion construction site; for instance we’ve seen the road outside our hotel be torn up at least four times. As for modern IT infrastructure? There was none to speak of. We have really had to start from scratch, right down to the laying of conduit before we could even begin installing fiber and cabling,’’ says Dean Frohwerk, Avaya’s chief network architect.
+ ALSO ON NETWORK WORLD Avaya’s Wi-Fi set up in the last winter Olympics +
That’s quite a contrast from 2010, when Frohwerk and his team provided telecom and networking services for the games in Vancouver.
And this time around, the demands for bandwidth and connectivity dwarf anything that Avaya delivered in Vancouver, where the network was capable of handling only 4Tbps.
The Sochi network will serve 30,000 athletes, administrators and staff, media, IOC officials, and volunteers with data, voice, video, and full Internet access through the Games sites.
Adding to the challenge, “We expect these people to be carrying and using multiple wireless devices,” says Frohwerk. “In Vancouver, we only had to provision one device per user. This means that we really have to have the capability to support up to 120,000 users on the Sochi Wi-Fi network, without issues or interruptions.”
Plus, Avaya has to deliver 30 IPTV dedicated HD Olympic channels via its telecom backbone, and has to make these channels available to Olympic family users over the converged network. (IPTV support is an Olympic first on the network, eliminating the need for a separate CATV HFC network.)
Network upgrade
In Vancouver, Avaya installed the first all-IP converged voice, data and video network at Layer 2. “That network was laid out like a single mammoth installation, which worked well given that wired traffic outnumbered wireless four to one,” says Frohwerk.
“But we expect this equation to turn on its head at Sochi, with wireless being the four and wired traffic being the one. That’s why we have had to change our approach.”
Another lesson from Vancouver, he says, is that “requirements evolve and change during the Games, and that you have to be able to adapt the network configuration to accommodate these changes. We have also seen that ease of use is paramount: With so much going on, network operators must find it simple to make changes on the fly.”
In Sochi, Avaya’s Wi-Fi network will be split into five virtual SSID-based networks. There will be one network for the athletes, two for media (one free, one paid), one for Olympics staff, and one for dignitaries.
Each group will have its own access password, and extra layers of password protection will be added where needed. The Wi-Fi traffic will be distributed using about 2,000 802.11n access points across the Olympics Game sites; including inside the stands for the first time.
The network will be headquartered in a primary Technical Operations Center (TOC) in the coastal city of Adler, alongside the Primary Data Center. The secondary TOC and Data Center will be at the Sochi Olympic Park, located 10 miles northwest at the Games site.
Each TOC will be in a 50-foot by 70-foot control room. While one TOC is in use, the other will be kept in standby mode by a skeleton crew. Each TOC will be connected to the outside world by 10GB pipelines provided by Rostelecom, Russia’s national telecom operator.
“We have built the TOCs in separate locations to ensure redundancy in the case of a natural disaster or man-made incident,” says Frohwerk. “Should the Adler TOC go down, we would simply send the next shift to the Sochi TOC and carry on.”
The data and voice backbone is built on Avaya’s Fabric Connect, an open virtualization platform based on IEEE 802.1aq Shortest Path Bridging that enables a network fabric within/between data centers and the sites they serve.
At the core of the network are four Virtual Enterprise Network Architecture (VENA)-enabled Virtual Service Platform (VSP) 9000 switches, one in each TOC and one more in each of the mountain cluster points of presences.
Using Avaya ERS 8800 switches located at the network’s edge, the whole Sochi network will be virtualized at Layer 3 instead of Layer 2.
“Using a Layer 3 virtual software layer means that our switches can act intelligently locally, and do a better job of routing traffic,” Frohwerk says. “This reduces traffic jams, which means more uptime and better network speeds. It’s a step up from what we did in Vancouver, because the demands we’re facing are so much bigger here.”
Using Layer 3 will let Avaya’s network operators serve many more endpoints than they could in Vancouver. Each device logging in will get its own media access control address: Avaya will use pre-installed 802.1X certificates, the MAC address, or a captive portal to authenticate the device, while controlling the access level and bandwidth with the company’s Identity Engines software.
Avaya is also providing voice services and 6,500 voicemail boxes at Sochi using Avaya Aura Communication Manager (CM), Session Manager (SM), System Manager (SYMGR) and CM Messaging.
Challenges
Moving goods into Russia can be time-consuming in the best of situations. But moving massive amounts of equipment in time for the Games has been a real challenge for Avaya.
“This is why we have had people in Sochi for the past 18 months, to keep things coordinated and to make sure supplies get where they need to be,” says Frohwerk. “You can’t leave things to chance.”
Even with this level of supervision, it’s been a nerve-wracking experience for Avaya, like the time one of its equipment trucks lost radio contact for days while travelling through rural Kazakhstan.
Another truck arrived in Sochi with unprotected/uncushioned computer hardware after driving over hundreds of miles of bumpy, rough roads. “In both cases, the equipment finally arrived in usable shape,” he says. “But we had a few tense moments there for sure.”
Another challenge is training. In line with their agreement with Avaya’s Russian partners, the company is training 170 Russian technicians to provide Tier 1 and Tier 2 network support during the Games. A 30-person team from Avaya Global Support Services will provide Tier 3/4 support from Sochi’s TOC, supported by Avaya R&D staff around the world.
The training of these Russian technicians is under way and Avaya staff has rotated on site to “train” for the Olympics.
“We are doing our best to be well-prepared for whatever the Games throw at us,” Frohwerk says. Avaya’s outdoor systems are designed to handle extreme weather: “We’re not worried if it snows,” he says. “In fact, we hope it does, because these are the Winter Games, after all.”
At press time, Avaya had completed installing all of its equipment in Sochi. The company is now moving into test mode, pushing the network’s limits by putting it through multiple types of failure scenarios.
Apres ski
After the games end on Feb 23, much of Avaya’s infrastructure will be removed. But the telecom facilities it has built for the Games – including the telephone and IP networking for the Olympics skiing venue in the Caucasus, where a new resort town is being erected, will remain.
The company will also be helping to develop telecom facilities for the Grand Prix auto races that will held in Sochi later in 2014, and soccer matches there that will be part of the 2018 World Cup.
“We will be leaving behind quite a legacy telecom system when we leave Sochi,” says Frohwerk.
Best Microsoft MCTS Certification, Microsoft MCITP Training at certkingdom.com
