The Metro Ethernet Forum is trying to unravel the toughest questions surrounding inter-carrier Ethernet interfaces — and they're just getting started.

In January 2006, a carrier called Broadwing touted a trans-Pacific native Ethernet service it delivered in tandem with another carrier, Hutchison Global Communications: a 10 Mb/s Layer 2 virtual local access network (VLAN) stretching from California to Hong Kong. Broadwing used a “q-in-q” method for the handoff, a standards-based technology common in corporate LANs. A new standard was needed, Broadwing said then, to enable more streamlined, scalable inter-carrier Ethernet services. The Metro Ethernet Forum took on that task, setting to work on an external network-to-network interface (E-NNI). Today, thanks to an acquisition, Broadwing is known as Level 3 Communications. And that E-NNI? It's in the works. Honest.

A year ago, sources close to the process predicted commercial gear based on the MEF's E-NNI standard would hit the market in late 2006, enabling live services in 2007. But today the E-NNI isn't expected to be complete till sometime later this year, according to Nan Chen, MEF president. Approved implementations would follow in early 2008, he said, but early-adopter gear could emerge later this year after the standard ossifies.

Chen attributed the slow pace of the standard's progress to the large number of cooks in the kitchen — a function of the broad popularity of the group's cause. “New members have come in, raising issues that were already resolved or closed,” Chen said. “That's always adding more time to get things ironed out.”

The MEF will be satisfied if it earns acceptance from 75% of participants, but even in that endeavor it has its work cut out for it. The group of vendors and carriers weighing in on the process have a host of issues on which to try to find consensus, including whether interfaces should be set at 1 Gb/s or include a variety of bandwidth speeds; how to maintain service quality levels from one carrier to the next; how to administer operations and maintenance to those services and so on.

One challenge facing the group is trying to determine how much visibility one carrier should have into the network of another carrier, which might also be a competitor.

“That's a sticky wicket,” said John Hawkins, Nortel Networks' senior marketing manager. “It comes down to the control plane — how these things are set up and taken down. In the T-1 world, very little control needs to be handed off because T-1s are so bloody simple; everyone knows what they're supposed to do. But Ethernet has more knobs to turn. The question is: Who gets to turn the knobs?”

In general, it's hard to understand how one carrier might, for example, turn up the bandwidth allotted for a particular service without visibility into the other network carrying that service and how much bandwidth it has to spare. At the same time, carriers probably won't want to let other carriers — especially competitors — know how much bandwidth is available in various parts of their network and where their network is stressed.

One possible method for dealing with the issue might be to segregate all inter-carrier traffic into predictable tunnels. Another would be to include information in the interface tags that specifies each service's requirements in terms of bandwidth, service quality and so forth so that carriers get all the information they need at the interface.

“The way we'd like to see it is we pass address and tagging information but nothing beyond that — no customer information,” said Patrick Sullivan, Verizon Business's Ethernet marketing product manager. “To our E-NNI connector, the customer would be Verizon.”

The MEF standard is likely to allow as many as 4000 services per inter-carrier interface. It will use as its underlying encapsulation scheme the “provider bridges” specification of the Institute of Electrical and Electronics Engineers' 802.1ad standard. That standard entails adding a second tag on top of the end customer's VLAN tag. Ethernet frames will pass from one carrier's network to the next using two tags: an S-TAGs and a C-TAGs. “Think of them as a provider tag and a customer tag,” Chen said.

“We want to keep knowledge local within carrier networks instead of in-between, so lateral agreements can be formulated very quickly,” Chen said. “You want to deal with interfaces on a need-to-know basis.”

Another challenge might be maintaining service quality among carriers with different classes of service with different definitions. What happens when a carrier with five classes of service hands off to one with only three classes, or vice versa? One way to resolve the issue might be to require service providers to map their own service classes into those defined by the MEF. It's more likely that carriers with more classes of service would be forced to narrow their classifications, rather than force those with fewer classes to create new ones. Carriers with more advanced service tiers than their partners might simply need to tell their customers that different service level agreements (SLAs) apply for services that make use of a third-party network.

The MEF is “shooting for” eight classes of service to be defined in the E-NNI, Chen said, arranged in a three-tier hierarchy that dictates which traffic to protect when the network gets congested.

Perhaps one of the most daunting challenges is how much operations, administration and management (OA&M) functionality to include in the interface. OA&M is a particularly complex part of the carrier Ethernet business — one of the last pieces of the puzzle to be tackled by the MEF. Separately from its E-NNI efforts, the MEF is developing an implementation agreement (IA) for carrier Ethernet OA&M. Like the E-NNI, the MEF expects the OA&M IA to be complete later this year. So it's unclear whether the group will wait until that effort bears fruit before including it in an E-NNI. “We really tried to limit the scope to expedite development,” Chen said. “The fewer the parameters, the better it is.”

The MEF is already planning a second phase of its E-NNI work that would theoretically allow meshed networks spanning multiple service provider networks, not just pairs of carriers. It might also up the interface speed from 1 Gb/s to 10 Gb/s. And whereas the first E-NNI will use provider bridges for encapsulation, the second phase might include a variety of protocols, such as multiprotocol label switching (MPLS), Chen said. That brings the MEF into new territory: Layer 3.

“It gets the MEF into questions it has typically been agnostic about, like how the Ethernet services are carried through the metro Ethernet network,” Nortel's Hawkins said. “Whether it's over [resilient packet ring] or MPLS, [provider bridge transport] — the MEF has typically not cared. But when you're talking about multicast behaviors, it may lead to complications unless you consider the underlying transport.”

Most of the demand driving the process is from carriers hoping to terminate circuits for other carriers, Chen said. International carriers are eager to use North American networks to extend their reach. But not every Ethernet service provider is anxiously awaiting the E-NNI. Executives at Yipes, a prolific provider of carrier Ethernet services, already have custom NNIs with a number of service providers in Europe and Asia. Although an E-NNI standard might speed up the interconnection process, Yipes executives said, the company doesn't spend much time thinking about it.

“It's not like we're waiting for the E-NNI to come out,” said Kamran Sistanizadeh, Yipes' chief technology officer. “This is not just about a box. There's lots of operational complexity. In about a year, a year-and-a-half, there might be some indications of progress. But we're proceeding on our own.”

Although carrier Ethernet interconnectivity is in some ways modeled after preceding technologies, there are important differences, Yipes said. Peering points on the public Internet grew out of a demand for best-effort services. In stark contrast, carrier Ethernet services — replacing private WAN services — entail much higher expectations for reliability, availability, latency and overall performance, said Keao Caindec, Yipes' vice president of marketing. “Simply putting together a technical NNI won't allow carriers to gain the service reliability to exchange traffic. That will only be done based on the economic and business relationships put in place. Some people are ignoring that.”

Masergy, which offers metro Ethernet services over Sonet and TDM connections — often from third-party network providers — doesn't expect to make use of an E-NNI standard, insisting that only its current provisioning methods afford sufficiently high levels of service quality.

Still, other providers aren't content to have to buy a Sonet pipe for every Ethernet service they want to provide. And they say it's too time consuming for two carriers to negotiate individual services at the user-to-network interface (UNI) level every time one wants to provision an Ethernet service. When Time Warner Telecom interfaces with another carrier today, the two parties have to start from scratch, defining physical interfaces before they can move on to discuss SLAs and other issues.

“With a standard, you could eliminate that first layer of discussions,” said Mike Rouleau, TWT's senior vice president of business development and strategy. “Having that first piece in place is helpful.”

“We have a lot of financial customers in New York City,” Verizon's Sullivan said. “Every one also has a location in Chicago. We're going to [serve both] by hook or crook, even if we have to extend TDM links in there. We're doing it now, and competitors are doing the same thing in our territory. But there are better ways to do it.

“Instead of giving me an OC-12 interface, give me an Ethernet interface,” Sullivan continued. “I want to be able to connect [UNIs] on my network, across the NNI, to UNIs on your network. To the customer, it looks like one seamless network.”