Developing business rules can be pretty complex sometimes and bugs could easily slip through if you do not test business rules the right way. In the previous episode, I discussed the pitfalls in testing business rules for planning. In this episode, I will outline the test process that will help you to avoid these pitfalls and deliver bug free business rules.
how to rightly test business rules?
Too often, I see clients struggle with the delivery of correct business rules. Bugs are detected not earlier than the acceptance environment and sometimes only in production! Only after a fair amount of analysis by the support team, debugging, bug fixing and release build, the bug is fixed. Nice? Not really: you have wasted a lot of time and money, not to mention, end-user goodwill. Completely unnecessary, if you would have applied the correct testing method.
Want to know how to test business rules the right way? I will tell you my secret. It is the overlooked good old process called “peer review testing”.
What peer review testing is?
In software development, peer review is a type of software review in which a work product (document, code, or other) is examined by its author and one or more colleagues, in order to evaluate its technical content and quality. Wikipedia
A bit generic description, but when applied to business rule development, it provides a very powerful safety net that prevents bugs from slipping through.
Peer review test is a disciplined approach and therefore a structured process that should be part of your development practice. With regard to business rule development it involves:
- code inspection;
- white box testing;
- performance assessment.
In the next paragraph, I will detail how this works in practice.
A very cost-effective method to visually check on inconsistencies and use of standards. Although this part of the peer review test does not necessarily guarantee bug free code, it helps in providing maintenance friendly and consistent code.
Code inspection is nothing more than a co-developer letting have a look at the rule:
- is the code easily accessible for inspection (code vs. graphical interface)?
- is the code properly documented (can someone else maintain it if required)?
- is the code properly formatted (preferably use a template)?
- is the code maintenance friendly (use of Essbase structures like outline, UDA, shared members vs. hard coded references)?
- proper use of scripts and macro’s vs. repetitive code?
- are the right variables used (run time variables and/or substitution variables)?
- are the team’s naming conventions applied?
Processing a code review can be as simple as sitting down with the developer and going through the code together. In about 10 – 15 minutes you will have a good assessment of the rule’s quality with regard to this point.
If your project requires a more formal approach, you can easily put these questions into a Word template and have the peer fill in the answers to these questions.
white box testing
The white box test focuses on the functionality of the business rule: does it really do what it should do? The white box test should focus on the correct working of the business rule with regard to any possible data.
A white box test contains an Excel sheet with the test cases, including the expected results (also known as unit tests) and the description of the steps to execute these. With these two documents, the team member (the peer) should be able to verify the completeness of the test, the quality of the test and the correct working of the business rule. As the developer is the most knowledgeable about the rule’s functionality it is best to have him/her create the white box test as well.
The Excel sheet with the test cases should – in its simplest form – contain the following four tabs:
- data entry tab, a SmartView tab page with the data entry to be submitted on a clean year – scenario – version – entity combination
- IST tab, a SmartView tab page ready to retrieve the actual results of the business rule.
- SOLL tab, the same IST tab page, but with the expected results calculated with Excel formulas
- check tab, the same IST tab but with formulas producing a “1” if there is a difference between the SOLL and IST amount and a total summarizing the errors
The test cases provided in the white box test should focus on the exception cases and provide the correct expected results for each of them.
The document with the steps for testing should describe in detail the functionality that is tested (e.g. reference to business documents), the focus of the test (what is tested and what is not tested) and the test steps for the peer to execute. One of the first steps in this document should always be to clear the data slice with a “CLEARBLOCK ALL” statement as to test the rule on block creation. Also, part of the execution should be a short peer assessment on the quality of the white box test:
- can you easily add new test cases in the white box?
- is the expected result calculated by formulas (or hardcoded)?
- are the test cases documented?
Big words for something pretty straightforward. In this part of the peer review test, you check the Essbase log for information about the process. Warnings like switching to “cell mode” and/or “sequential processing” should raise red flags. Also, multiple passes through the database indicate that the performance could be sub-optimal.
If the rule is part of a frequent used (user initiated) process, a more thorough assessment of the performance should be done using a standard data set to measure the duration. Preferably, this data set represents data from the real-life situation. If the test set is set-up carefully, the results can be extrapolated to the production environment. If the results hint at a high likelihood on bad performance, you can spent time right away in finding a more optimal alternative.
benefits of peer review testing
Using the peer review test on your business rule development gives the following benefits:
- Assurance of a high quality of a release (saving time and money for rework);
- Additional confirmation by the team that the rule is working as expected;
- Assurance of an acceptable performance when used on production;
- Rules that are maintenance friendly;
- Re-usable test sheets for future maintenance;
- Buildup of testing evidence to comply to SOX (if required).
objections to peer-review test
objection 1 the developer is responsible for unit testing, there is really no need for another test
Really? Don’t you want the assurance by a fellow team member that the developer did a good job or not? Don’t you want other team members to be able to also verify the correct working of a rule and to become knowledgeable of all the logic in the planning application?
Giving the developer the sole responsibility for rule development will create a high dependency on this one person that might end up in an unhealthy business relation, like being hooked to an external consultant for years. I have seen that happen.
objection 2 – we already have an integration test to ensure the application works as expected
Integration tests have some use for testing planning applications, but the focus is not on finding errors in business rules. They focus on the correct workings of all parts together and they focus on the regular results the solution must provide (not the exceptions). Integration tests in the EPM world, almost never have documented test cases. And when they have, the focus is not on the exploration on what could go wrong in a business rule but rather if the business rules are producing the right results on regular data.
objection 3 – we already have an acceptance test; the business will test the rules
I do not think that your business will like this statement. Have you ever done a test drive in a car that had not been properly tested by the manufacturer? How would you react to find the engine does not start, or your wheel blocks when turning left?
Do not deliver poorly tested rules to the business. It will give them lots of frustration and your team lots of trouble in solving all the issues, not to mention a bad working atmosphere.
Not an option if you want to become the trusted EPM advisor for your business.
option 4 – there is no time for this kind of testing
Oh, but do you have time for a lot of rework then? And do not forget to include extra administrative work with regard to release build, issue management, installation and “frustration” management.
Implementing the peer-review testing on those part that need this additional focus – business rules – will help you speed up the delivery process of new functionality because of the quality of the work delivered; there are little to no bugs to find. Ergo, no rework, no administrative work and a smooth acceptance and go-live process.
Example 1 – Building a peer test process in an upgrade to include forecast functionality in a Planning application
During one of my projects, in the role as core Essbase architect, all existing business rules needed to be updated to enable forecast functionality. An ambitious endeavor, but we introduced the peer review process. We decomposed the complete change into small functional parts that could be developed in parallel. Most parts consisted of a single business rule supporting a specific process step.
As core architect, I almost did most of the work on the business rules, including the creation of white box tests. And although it took quite an effort to create these, the results were amazing. Before this process was introduced, the client reported around 400 bugs on the acceptance environment (we were not involved at that time). After introduction of this process none were reported that were related to the business rule development. And as a happy side-effect, my team members started to understand the logic of the business processes, making them less dependent on my expertise.
Example 2 – amending HR logic in an existing FTE model for a governmental organization
As one-person implementer of an FTE planning model upgrade at another client, I introduced the concept of the white box testing, using the four Excel tabs. By using this method, I could deliver bug free business rules with the new HR logic. The sheets were not only beneficial to me, now that these white box sheets exist, the planning administrators themselves use them as well. They now feel comfortable enough to apply small changes themselves to the business rules, knowing that there is a working safety-net that will prevent them from introducing bugs in the production environment.
Developing business rules is difficult and mistakes are easily made. By using peer review testing you can substantially increase the quality of your team’s work, resulting in smooth acceptance tests by the business and less bugs in production. As a side effect, you’ll also educate your team into the business logic, reduce dependency on a single developer and grow to become a trusted partner for your business.
Give peer review testing a try and let me know how it turns out for you and your team.-----
share this post