Younger vendors of networking equipment and software often have used benchmarking results as tools to put themselves (in their minds) on par with more established competitors. Not to diminish the accomplishments of the engineers and developers who deserve credit for continuously raising the bar on performance and moving the industry forward, but sometimes these benchmarks have to be taken with a grain of salt.

As rigorous and independent as some of the benchmark procedures may be, their shelf-life in a world of hyper-adherence to Moore's Law can be as little as weeks and sometimes days, since they are timed to beat the results of their competitors' own benchmarking results, which soon follow.

Service providers, on the other hand, have subconsciously always followed the Max Ehrmann rule when it comes to benchmarking. In his 1927 "Desiderata" he wrote: "If you compare yourself with others, you may become vain or bitter; for always there will be greater and lesser persons than yourself."

Consciously, they have gone by another rule, one that is broadly applicable to the military, sunbathers, telecom and elsewhere: CYA. Service providers have been reluctant to compare themselves to others because the middle managers who would pay the price for lousy results are reluctant to get fired. Besides, there was no real need to do so when they were isolated in their respective regions.

But I'll be darned if the TeleManagement Forum hasn't convinced service providers to give benchmarking a whirl. Yes, there have been other benchmarking initiatives. They have been conducted primarily by and for individual telcos. But as TMF President Jim Warner said, "They have been extremely expensive one-off projects where carriers hire a consulting company that costs maybe $150 grand, but all those do is tell carriers how they stack up against other carriers at the time. That's fine, but if all carriers suck [in a certain department], the only thing that tells them is that they may suck 10% less."

Nobody wants to suck, not even 10% less. You can't use that in an ad campaign, unless you're advertising the new Orick vacuum.

The forum has proposed being the neutral third party in an industrywide benchmarking service that will help service providers measure themselves--in terms of their processes, profitability and customer experience--with other service providers on an ongoing basis, discreetly and securely. In fact, they have already started the process of putting together the IT infrastructure and the participants to begin building benchmarks and defining metrics (see story below).

Long term, the TMF hopes to extend the benchmarking process so that these telecom companies can measure themselves against corporations in other industries. They can compare, for example, how the experience of ordering and receiving DSL service compares to the purchase and delivery of some obscure, out-of-print book from Amazon.com.

The service will look at the fulfillment process end-to-end and determine the quality of the customer experience.

Well, anyone who has been a phone company customer can already tell service providers one process they should fix before they begin benchmarking--and for a lot less than $150 grand: Answer the phone.

With cheap, quality customer care available around the clock and around the globe, how hard can it be? And how expensive, compared to unhappy customers who in the best-case scenario buy less products and in the worst, depart? It has always been a mystery why companies pour gazillions of dollars into R&D, network quality and advertising, then let their reputation be sullied by making customers with legitimate service concerns wait on the phone for more than 30 seconds.

Service providers (mostly landline) have never been either able or willing to get this right. So the Warner rule probably applies here. Some may just suck a little less than others. However, at least the data won't be skewed if service providers are all working off similar criteria.

But when telcos take their performance out to the wider world and compare it to, say, L.L. Bean, they may wish they had listened to Max Ehrmann. Hopefully, this intra-industry process will help service providers begin comparing apples to apples and identify areas of improvement before they start comparing themselves to real service companies.

E-mail me at tmcelligott@primediabusiness.com.