Performance Management and Testing

Performance management is the art and science of monitoring, predicting, and controlling the performance of your computer systems. The aim of Flexion's performance management practice is to give you the tools to avoid these kinds of surprises:

• server crashes when you can least afford them
• batch processes that do not terminate in time, causing a breakdown in reporting and auditing
• unplanned labor expense due to poor computer system performance
• unnecessary expenditure on excess or unsuitable systems
• customer dissatisfaction

Today's business environment includes wide-spanning integrations between legacy mainframe systems, third-party software, in-house systems built with last year's technology, and new frontier systems that span all these. Managing system performance in such an environment is not for the faint of heart; it requires deep and wide-ranging expertise combined with an agile and active intelligence.

The purpose of performance management is to give the responsible executives the knowledge and power to make choices, and to give the organization the ability to put those choices into effect.

The discipline of performance management can be divided into three parts:

Performance monitoring

    Observing, recording, and analyzing crucial performance metrics in your computer systems, in order to prevent problems, react to them promptly, and diagnose them efficiently.

Performance prediction

    Modeling, simulation, and testing to predict the performance of your system under a variety of projected loads and proposed changes.

Performance tuning

    Finding and eliminating bottlenecks and inefficiencies, in order to deliver better and more predictable performance while minimizing expenditure.

The Discipline as a Whole

The three parts of performance management do not stand independently. Without the ability to monitor and measure performance, a test has little value; without the ability to predict, we can only manage in retrospect; without realistic load characterization, tuning and prediction lack a sound foundation. By combining all three aspects, the Flexion performance practice can ensure the best business value for your computer system expenditure.

Our Practice

The Team

Our performance management practice was founded by Roger Hayes, Ph.D (Comp. Sci.). Dr. Hayes has worked in testing, performance measurement, systems architecture, and software tools at Sun Microsystems Laboratories, Keynote Systems, and Orbitz. He has presented at conferences in Germany, Japan, and Silicon Valley. His interest is in applying science to solve business problems.

The performance management practice draws on Flexion's wide expertise in mainframes, monitoring tools, network administration, systems architecture and programming, and modern application servers and SOA. We pool our expertise, both in-house and with our partners, in order to be able to draw on top-quality talent to deal with any problem and explain any mystery.

Our recommendation is to use open-source tools when feasible, because they are cost-effective and easy to support; commercial tools when they are the best value or are mandated by other concerns.

Best Practices

Our recommendation for the most effective engagement follows these steps:

1. Assess your performance monitoring environment. If you are just getting started, we can recommend procedures and tools, and help with configuration and training. If you have well-established monitoring procedures, we can help relate that data to performance predictions, to coding practices, and to business needs, and can use that data as the basis for performance testing and tuning.
2. In collaboration with your QA and development groups, establish a solid performance tuning cycle. The fundamental rule of computer system optimization is that you must always measure first; “premature optimization is the root of all evil” (Donald Knuth). But omitted optimization is a waste and a danger; the ideal is to use appropriately-scaled performance tests as a integral part of your development cycle. Your developers and systems administrators will gain a depth of understanding about performance consequences that will provide a continual improvement in efficiency and predictability.
3. Applying the lessons learned in performance testing, about crucial performance indicators and monitoring, closes the loop to add to the business expertise of the performance monitoring group. Thus the three aspects of performance management are joined into a positive spiral that delivers higher employee and customer value in an efficient manner.

While this is our recommendation for the best practice, we do not expect any real engagement to adhere to this outline. Central to Flexion's value proposition is that we meet the actual needs of your business, rather than following a set script. This is a harder way to earn a living, but we enjoy the challenge.

An initial engagement typically lasts six to nine weeks, depending on customer need and budget. In that time, we can help improve operational monitoring, establish a performance test and tuning cycle, and deliver an initial performance assessment and tuning recommendations. After the initial engagement, it is often cost-effective to engage Flexion for periodic intensive performance workshops, perhaps in preparation for a major software or architecture change, a migration, or in preparation for the busy season. We can help you plan, set up for, and get the most value out of a large-scale performance test, and can help make sure that performance management is delivering business value rather than pretty reports.

Our goal is to make you successful.