The Computer Journal Advance Access originally published online on July 19, 2006
The Computer Journal 2006 49(5):509-526; doi:10.1093/comjnl/bxl037
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
An Empirical Performance Study on PSIM

Department of Computer Science, Iowa State University USA
*Corresponding author: jxia{at}iastate.edu
It is well known that errors caught in the early stages of system development life cycle greatly helped reduce the expense for later error fixing and mitigate risks and complications. Performance analysis is one of the methods to ensure satisfactory system performance that can avoid redesign and patch in the later stage of system development when performance problems emerge. Much work has been done to address such concerns collectively known as the performance engineering problem. However, a comprehensive method is still needed to provide an end-to-end support for performance evaluation on software architecture design. Automated tools to support the method must be developed, and its validation must be carefully planned and conducted. In response to this need, we propose our approach, PSIM (performance simulation and modeling). PSIM is a performance simulation and modeling tool that integrated performance properties into software architecture specifications expressed in several major UML diagrams. PSIM models can be transformed into Colored GSPN (colored generalized stochastic Petri nets) via an automated tool. As a result, the Colored GSPN models can be simulated to perform model-based performance evaluation. In this paper PSIM is first briefly reviewed and illustrated through modeling a web-based electronic conferencing system, called M-Net, to derive performance metrics. We then conduct runtime performance testing to the implementation of M-Net and compare the simulation results against the runtime testing data. PSIM is shown to be effective in predicting system performance and in identifying system performance bottlenecks through this experimentation.
Key Words: empirical study performance metrics performance analysis performance testing software architecture