New TPoX Benchmark Results Available
March 4, 2011
TPoX stands for “Transaction Processing over XML” and is an XML database benchmark that Intel and IBM have developed several years ago and then released as open source. TPoX is an application-level benchmark, which means that it simulates an actual application that performs queries, inserts, updates and deletes in a concurrent multi-user workload.
Of course, no single benchmark can represent the characteristics of all types of database applications. In fact, a benchmark should not even try to do that because it would end up being an unrealistic workload. Take the traditional relational database benchmarks as an example: the TPC-C benchmark is an OLTP workload while the TPC-H benchmark mimicks a decision support scenario, two very different usage patterns for relational databases, hence two different benchmarks.
TPoX is designed to be an XML transaction processing (OLTP) benchmark with data-oriented XML structures, very large numbers of relatively small XML documents (1kb to 20kb), short read/write transactions, and a high degree of concurrency.
Many characteristics of real-world financial XML applications have gone into the design of TPoX. For example, the “order” messages are defined by the FIXML standard that is used in the financial industry. The actual FIXML schema is used for document validation in TPoX. To give you an idea of its complexity: FIXML Version 4.4 consists of 41 schema documents (XSD files) and defines more than 3600 elements and attributes that can appear in FIXML messages.
Obviously, TPoX does not intend to be a content-oriented XML benchmark where full-text search and other content-centric operations would be exercised. Another XML benchmark is needed to cover that space.
Since 2006 both Intel and IBM have published TPoX results for various different scale factors and hardware platforms. In 2009 and 2010 Intel has continued to run the TPoX benchmark internally to evaluate their ever evolving fleet of Xeon processors. Four benchmark results from that period have now been published on the TPoX web site, showcasing the performance of the Intel Xeon processors X7560, X5680, X5570, and X7460.
These benchmark results from Intel are vary valuable in several ways:
- The results prove XML processing performance on different hardware choices, ranging from 2 x 4 cores all the way to 4 x 8 cores in a single server.
- They quantify the performance gain that each new generation of Xeon processors provides over the previous.
- These TPoX benchmarks continuously push hardware, operating system, and database server to higher levels of performance. This helps to identify and exploit opportunities for performance enhancements which ultimately drives innovation, advances technology, and improves products quality.
- The range of TPoX results serves as yard stick for companies that need to choose hardware and software for their XML database needs.
Other software vendors (including Oracle) as well as Universities have also been using TPoX for a variety of purposes. Although some database vendors other than IBM have been using TPoX, to the best of my knowledge they haven’t disclosed the details of their testing, such as the exact results together with detailed information about how the hardware and database were configured.