DSBENCH Article Released

It’s done! After nearly 2 years we finished compiling the results collected with DSBENCH-marking several servers and desktops.

Abstract

To find means to qualify arbitrary computers regarding their capability to act as a database server we perform a large number of tests to understand which characteristics of a computer system influence the performance most. We qualify performance by measuring storage system throughput and by extensively testing the system with a self–developed TPC-B like performance benchmark program called DSBENCH. Benchmarking produces a number of characteristic diagrams for a system as well as a single qualifying performance indicator (DSI) which we demonstrate to be useful to compare system performance easily.

Some conclusions regarding DSBENCH

• DSBENCH is able to test PostgreSQL and Firebird database servers with both SELECT and standard TPC-B strategies.
• Transactions are implemented as PSQL database functions. This turned out to be a performance gain compared to pgbench which realizes transactions as a set of primitive SQL commands.
• A single DSBENCH run makes a comprehensive test by varying the database size, by acquiring all transaction residence times as well as CPU and IO load data. It writes result tables and diagrams that also contain a detailed residence time frequency distribution analysis. Load data and statistics help to identify the influence of the system stack layers to the overall performance. For quick comparison of plenty systems a single digit (logarithmically scaled) performance index DSI is calculated from the transaction rate function.
• DSBENCH can also disclose network related bottlenecks by simultaneously stress testing a database server from many remote clients.

Download-Links:

• The full paper including appendix, ca. 50 MB.
• The paper without appendix, ca. 1.4 MB.