To test the signalling protocol of a 3G device can require, depending on the capability of the device, over 500 test cases to be executed. These tests cover RF performance, signalling protocol, SIM/USIM and audio aspects. This requirement has proven to be highly effective in improving the quality of service offered to users. Opening a recent GCF meeting, a senior manager for one the European network operators stated that GCF-certified terminals suffered half the rate of dropped calls on their network compared to non-certified ones.The conformance testing must be carried out in an approved lab. There are a number of specialist labs around the world that offer certification-test services, using test equipment approved by GCF and PTCRB. There has been an increase in the number of authorised test labs over the past 18 months, resulting in more competition and hence lower hourly rates. However, device certification is still an expensive process, requiring hundreds of hours of test time.

Managing the costs

Larger manufacturers, with sufficient numbers of new models being introduced each year, can justify the expense of establishing their own labs, which often have their own personnel, separate from the development teams. Smaller manufacturers make use of the commercial test labs, often calling upon more than one in order to achieve the required test coverage and meet the required timescales.Either way, there is a significant management overhead involved in the certification process, which lies on the direct time-to-market path of a new device. It is essential that the certification process happen smoothly and predictably in order to allow all aspects of the new product launch to be planned accurately. A stalled product launch due to test failures may not just shorten a product's life resulting in loss of income but can easily result in a new design being scrapped entirely, never to be seen by the public. To help drive down the overhead caused by conformance-test failures - an issue faced by both large and small cellular device manufacturers - Aeroflex introduced the WCDMA ACE, a low-cost, compact, single-box instrument (Figure 1) suitable for use on each protocoldevelopment-engineer's desk. It supports 3GPP R99, Rel-5 (HSDPA) and Rel-6 (HSUPA) protocols. The instrument makes it possible to implement a cost-effective pre-certification test phase that will filter out failures earlier in the development cycle. Regression-test campaigns can be established for new software releases due to full automation capabilities. Certification test failures can be analysed off-line, offloading the main conformance test system.

Pre-certification testing

The instrument allows handset developers to run the majority (more than 75%) of the 3G protocol conformance tests before the device ever leaves the lab but without the large investment that a full conformance system requires. Although it runs exactly the same test cases that run on full conformance-test platforms such as Aeroflex's 6401 AIME/CT, it is not a formally validated test platform, which helps reduce its cost (see Figure 2). As such, there is a good chance that a device that successfully passes the tests on an engineer's bench with the WCDMA ACE will also pass the tests during the certification process.Test failures at the certification stage are among the most expensive failures to rectify, and they cause bottlenecks in the test lab due to the time it can take to diagnose the cause of the fault. Many test labs tend to have a single system to cover each type of testing (protocol, RF, etc.) due to the high cost of the test systems. Full conformance-test systems are expensive due to the specialised nature of the equipment and the need for them to be continuously re-validated in line with detailed changes to the standards - this includes adding new functionalities to the conformance-test regime as required by the certification bodies.In any case, diagnosing failures using a conformance test system, possibly in an overseas test lab, may not be the most effective use of a development team's time. For a smaller manufacturer, the typical design cycle consists of buying a chipset, licensing the protocol stack and developing a customisable user-interface, possibly based on a standard operating system. The main development activity is to successfully integrate the various sub-systems and then test that the design meets the specifications. This is greatly helped by ensuring that the protocol stack and chipset have already been tested by the supplier, but this procedure increases the cost and is not always an option. In this scenario, the development and integrgration teams are unlikely to have access to an in-house conformance-test system and will rely on standard instrumentation to do as much testing as possible before sending the device to an external test lab for certification. This increases the risk that the device will not pass first time.In this scenario, the WCDMA ACE can be used for conformancetest-failure diagnosis. The log files captured on the Aeroflex 6401 AIME/CT during conformance-test execution can be analysed back in the lab by an engineer equipped with the WCDMA ACE. After the necessary re-work has been carried out, the engineer can re-test the device himself prior to re-submitting the device to the certification lab. If necessary, the standard test cases can be extended using the optional TTCN-2 development environment available for the instrument.

Not just for small operations

Larger manufacturers tend to use multiple chip-set sources, including both in-house developments and bought-in ones, usually with an in-house protocol stack and user interface. There will typically be multiple design teams working in parallel on different device types covering a model range, usually based at different locations around the world. The company's in-house test-lab will provide the equipment and expertise needed to achieve the appropriate certification (GCF/PTCRB or network-operator-specified, depending on the market to be addressed). However, even the largest test labs would usually expect to sub-contract some testing to other labs. This can be in order to handle temporary overflows due to non-aligned delivery from multiple design centres, or due to small gaps in test coverage. 100% test coverage would require purchase of equipment from nearly all approved test equipment vendors, since they nearly all have some unique test cases (although this a situation the GCF is working to eliminate). This centrally managed certification-test lab (often a multi-site operation itself) represents a significant investment for the large manufacturers. The efficiency of such a large operation relies on the quality of the designs being received for certification. Poor, non-conformant designs will result in unpredictable test-cycle times, making it difficult for the test-lab manager to schedule resources accurately. The WCDMA ACE is priced to allow multiple development teams to be equipped with the same tools to ensure correct and consistent behaviour before the device is sent to the central test lab, allowing the latter to work more efficiently.