For Telcos, deploying IPTV in remote areas is a market sweet spot – it offers customers the perfect blend of choice and services and it’s often an area unsupported by cable providers. The challenge for many Telcos is to provide customers with the same or better quality of video that they expect from their broadcast or satellite providers at a cost that makes delivering IPTV a successful business.

The Rural Telephone Service Company, Inc.’s (Rural Telephone) IPTV service area covers 5,100 square miles in Western Kansas. Their network has one headend and 35 central offices with 4 x OC-48 rings. This network architecture allows Rural Telephone to access IPTV content from any node in the network and to easily add content such as local (community-based) channels and advertising as required.

Rural Telephone realized early that its IPTV network had to be built, tested and then continuously monitored to ensure quality of experience for customers. Unlike data and voice traffic, live video over IP creates its own special traffic flow and behaviors. A key requirement in testing and monitoring IPTV/IP networks is to ensure the transport of live packetized video without loss. Unlike voice and data, video on IP is highly time sensitive; with loss (dropped packets) clearly and quickly impacting a viewer’s experience.

For instance, in an IPTV network hundreds/thousands of queues of video data packets ebb and flow through routers, switches and edge devices. Any time any of these queues fills, a loss event can occur and result in degraded video quality. Since the routers, switches and edge devices are distributed throughout an IPTV network, finding out where the problem is occurring can be an expensive proposition. In addition, with IPTV, waiting for a customer call to report poor quality is too late. Customers will quickly switch back to broadcast and assume the new technology is not ready for ‘prime time.’

Rural Telephone chose a solution designed to pre-empt these problems; namely, proactive monitoring and remote troubleshooting. Since Rural Telephone covered a large area, they needed tools that would allow remote monitoring. They used the IneoQuest Singulus™ G1-T Video over IP Monitoring and Analysis tool with Media Delivery Index (RFC 5444) measurement at remote locations to monitor the transport of video streams coming into several of their most critical routers.

The IneoQuest tools were deployed throughout the network and Rural Telephone began to continuously monitor all video streams simultaneously and in real time for any anomalies. When the monitoring began, they found a 10 percent media loss on one of the IPTV streams.

The measurement was taken using the Media Delivery Index (MDI) (RFC 4445) MLR – Media Loss Rate monitor. This measurement identifies media loss rate within the MPEG Transport Stream. In its current configuration, Rural Telephone has a few hundred variable bit rate IPTV streams that are MPEG2 Transport Stream encoded. It is essential to monitor all Video over IP streams simultaneously to be able to identify which stream is impaired due to packet loss. If an IP packet is dropped, the media loss counter will increment by a count of 0-to-7 as there are 7 TPS packets in a typical IP frame.

When Rural Telephone detected this media loss they hadn’t received any customer complaints – but as stated previously, that can be too late. In fact, the media loss score of 10 percent was the result of several hundred media loss events within a few hours. Given that the business of a service provider can thrive or die depending on service quality, this information put Rural Telephone in a better position to service its customers.

First, Rural Telephone had to determine if the IP packet was dropped or if the video transport stream payload was incorrect before its encapsulation on the IP network. Monitoring the network at critical nodes was essential to isolate the problem. Rural Telephone placed one IneoQuest Singulus G1-T at an encoder and the other at an aggregation switch. This allowed Rural Telephone to monitor several hundred IPTV channels and to probe several different points in the network.

It was a challenge to find out not only where the loss was occurring, but also its cause and how it could be fixed. Was the loss on the IP or the Video network? Each component in the network was suspect -- from the encoder to the cabling to the routers. Then there was (and is) the issue of interoperability standards. At that time as today, there are few if any interoperability and test and measurement standards.

From a remote lab location an IneoQuest engineering team was able to analyze the data from the Singulus G1-T monitoring tool. The data included analysis of counters, log events, video and protocols. The test results indicated that extra information was being inserted into the video payload and showing up every 15 to 30 minutes on average per affected stream. This showed up on only one encoder and affected 10 to 15 IPTV streams and affected the customer as an audio glitch only.

In addition to capturing the network protocol, the IneoQuest Singulus G1-T recorded the MPEG2 TS video and audio to replay the identified fault for debug purposes and further analysis by equipment manufacturers.

Video over IP monitoring and analysis tools enabled Rural Telephone to quickly solve a network interoperability and configuration issue that would not have been detected using traditional layer 2 and 3 test equipment. The nature of streaming video over IP required the monitoring of the transport stream loss, PCR bit rate and cumulative jitter on a per stream basis to isolate impairments caused by the encoder. The ability to simultaneously monitor and analyze all streams allowed for quick fault identification, which saved time and reduced travel costs by service personnel.

MDI – the measurement delivery index, helped track, identify, monitor and debug the problem. Without MDI the issue would have been undetected for a long time causing customer frustration and a possibly a reduction in take rates.

As more and more Telcos look to provide IPTV to their customer offerings, ensuring quality of service at a reasonable cost (limited truck rolls, etc.) is essential for turning this technology into a successful business. Troubleshooting and remote monitoring of these ‘new video networks’ puts the Telco in charge of delivering quality service and provides a predictable cost per service for an IPTV offering.