While the rest of the world was celebrating the beginning of 2009, one France Telecom-Orange team was busy mending damaged cables, some few thousand metres under the surface of the Mediterranean Sea. Sathya Mithra Ashok gets a blow-by-blow account of how it was done.
If the average IT manager in an enterprise thinks he is having trouble with his cables, it is time for him to think again. After all, he does not have to constantly monitor and rectify faults that occur thousands of metres below water, right at the ocean floor.
“We have between 20 to 30 repairs in the Mediterranean Sea alone every year. And at least some across other parts of the world, maybe three to four repairs per year on cables that we operate. And if we see just some of the cables for which we have stations in France and, or French-overseas territories, there are times when Orange teams have conducted one repair every month,” says Alain Polloni, head of submarine operations and maintenance at the France Telecom-Orange group.
Immediately after the disruptions, alarms went off at the Orange Network Operating Centre located in Bagnolet, France. The NOC then sent out SMSes to the alert distribution list.
If cables on the ocean floor are liable to be damaged easily, there is a necessity to expedite the repairs as well. To do so, France Telecom-Orange, along with many other telecom operators and service providers operate ships across the world, in order to be able to rush technicians and experts to the sites of damage as soon as any network traffic is disrupted.
“When we have disruptions, cable faults are localised by the transmission and measurement of signals between station A and station B. This initial signal is electric and this is reassessed with the transmission tests that are conducted from the stations. With these we are able to exactly pinpoint the location of the failure,” says Polloni.
Due to this constant vigilance brought about by the necessity to maintain customers, these companies have built up expertise in the area of repairing cables mid-sea in adverse conditions. But Polloni points out that each case can prove a challenge, regardless of the expertise at hand. And what happened on 19th of December last year, put the France Telecom-Orange group to the test.
Cable cut
On that particular day, six cables in the Mediterranean Sea off the coast of Sicily, Italy were cut. Cuts of these cables – SeaMeWe3 (SMW3), SeaMeWe4 (SMW4) and FLAG – had a direct impact on Orange’s customer networks, cutting off traffic flow between Europe, the Middle East and Asia.
“Immediately after the disruptions, alarms went off at the Orange Network Operating Centre located in Bagnolet, France. The NOC then sent out SMSes to the alert distribution list. Our customers were informed real-time by our customer service centres worldwide,” says Polloni.
He adds: “Based on our experience and because of the different localisations on the terminal stations of each cable, the almost certain preliminary diagnosis was a fault at sea probably in the same area and affecting several cables.
“This diagnosis was confirmed later in the same morning by the different measurements performed by terminal stations.”
The job of traffic re-routing began as soon as there was information of cable damage. As only a branch of the SMW3 cable had been cut, traffic that could be restored off FLAG and SMW4 was routed through the main trunk of the SMW3 cable – the only immediately available route. In addition, Orange also called in its terrestrial network to restore some traffic to Asia and the Middle East.
Most network traffic heading ‘west’ from the Middle East was diverted eastbound before heading back to Europe. In spite of these efforts, the need for rapid repairs loomed large, since the re-routing was causing longer round-trip delays and there remained some traffic that could not be re-routed at all.
To sea, ahoy!
Since these cuts affected most of the major carriers in the area, Orange conducted audio conferences and worked in close co-ordination with other members of the consortium to ascertain the priority of repairs on the cables. This done, it dispatched its cable repair ship Raymond Croze with 20km of spare cable from the French Riviera, to repair the SMW3 and SMW4 faults. Cable ship Teliri, which belongs to Elettra (an affiliate of Telecom Italia) was assigned to the FLAG repair.
“The network that we maintain makes us well prepared for such expeditions. We maintain ships in different parts of the Atlantic, Mediterranean Sea, Indian Ocean and North Sea. Additionally, we have all the data on these cable routes under the sea as well as the main operations of the various cables. We maintain this as sensitive information, and when we do need to conduct a repair we have all the information available,” says Christian Brault, head of the network and traffic management centre at the France Telecom-Orange group.
Polloni adds:”When we have to conduct repairs on cable faults, when the ship is mobilised, we need to secure different things. We need to get enough spares aboard the ship to be assured the repair is performed in good time. We also need working permits if the cables are in territorial waters. We need to secure the terminal stations and make sure there is necessary staff to conduct tests during the repairs. These are the basic things to secure before the ship sets out.”
Polloni adds that even when starting out, the team expected certain challenges in these repairs.
“Well, the major challenge that we face in undersea cable repairs is the condition of the sea itself. If the ocean is very rough and conditions are over operational limits it can lead to suspended repairs due to security reasons on board the cable ship. Though our ships are huge, and have capabilities to station and remain stable in rough conditions, there are always limitations.
In this particular project, the other major challenge was the fact there were multiple faults to handle. We had only two ships and six cables to repair. The order of repairs had to be agreed based on the volume of impact, and this had to be properly conveyed to the crew. If we have one cable to repair and one ship, things go smoothly. When we have multiple faults, things get complicated,” explains Polloni. Suture time
Knowing these challenges well, the almost 60-member crew of Raymond Croze remained confident of success as they reached the repair zone for the SMW4 cable on 22nd December. The ship deployed a remote operated vehicle (ROV) undersea to inspect the cable and find the exact location of the fault.
“ROVs are part of the equipment used on cable ships. These are submarines linked to the ship by an umbilical cord and they go down to around 1500m . They can localise precisely the fault, visualise it and provide video on board. They are also used after repairs for inspection and cable burial. At depths below 1500m, ships have to use grapnels to hook cables. Ships also have some equipment to preserve the continuity of transmission between stations while repairs are conducted on the damaged cable,” says Polloni.
The normal repair process involves hooking the damaged cable to the ship. Following this, the strength of the cable is tested with station A, and then with station B. Using the equipment on board, continuity between these stations are ensured while the actual fixing of the cable takes place. When the cable is repaired, the signal is again tested between the two stations and when it is the right strength, the cable is released onto the sea bed.
Following this standard procedure, the cable ship Raymond Croze was able to repair the slice on the SMW4 cable on December 24th and traffic activation was scheduled to be completed the next day. However, complications arose when a new fault was located on the SMW4 cable late on 25th, around 400km off Egypt and some 1400km from the original repair. After rapid co-ordination with the consortium members, Raymond Croze headed off to the second site of SMW4 repairs, instead of to repair the SMW3 fault as originally planned.
Meanwhile, the Italian ship Teliri repaired the FLAG cable on December 29th. Orange and all operators using FLAG started re-establishing capacity on the fixed cable. As of 31st December, all capacity built on the FLAG cable was stable, and traffic had resumed on this link. Following this, Teliri sailed to repair the SMW3 fault.
On 4th January, the second SMW4 fault was successfully repaired, and the next day the SMW3 cut was rectified.
Adventure on the high seas
“The only challenge we encountered apart from the ones that we had expected previously was the experience of getting people in during the holiday season. Many people had to come in to the office and give up their vacations. That was a major challenge. Fortunately though, Orange personnel were willingly mobilised and worked hard until the environment was back in shape,” says Polloni.
Brault adds, “Communication was also a huge challenge. Throughout the crisis, the teams had to communicate between each other and precisely. Moreover, customers had to be constantly informed on the progress. This factual communication is key to maintaining confidence.”
Despite the many things that could potentially go wrong with such mid-sea repairs, or maybe because of that, Polloni continues to find the work fascinating.
“It is fascinating to work on such repairs because it is quite challenging and because of the tempestuous nature of the work environment. However, I have to mention, that there needs to be a method of repair in place. For every repair process the teams should be prepared in advance so that even the worst conditions can be managed,” he says.
Polloni urges cable repair teams to be on their toes, pointing out that the likelihood of damage to submarine cables remain high across the world due to the amount of cables laid out as well as the nature of their location.
“There are more than a million kilometres of cable undersea and there are lots of ways they could be damaged. It can be via ship’s anchors, or fishing trawlers close to the shore or they can be crushed by geological phenomenon like quakes or landslides. We try to prevent these by avoiding laying cables over routes that are known for unstable geological factors. We also explain our cable routes to shipping and fishing companies and allow them to download these routes from our website to avoid damaging them,” says Polloni.
In spite of these preventive measures, submarine cables continue to be damaged, but as long as we have teams like Orange’s around – with their interest and dedication in the work, as well as their taste for adventure on the high seas – we can rest assured that these faults will be rectified in no time.
Follow us on
