test business rules - beware of the pitfalls

pitfalls in testing business rules

In “testing” by Arthur van den Berg

Developing business rules is a skill. It is not as easy as it looks and you easily make mistakes that could become critical when your rule is promoted to production. Testing business rules the right way is the answer to provide bug free business rules. But just applying standard test methods to business rule development could not be the answer you are looking for… Beware of the following pitfalls in testing business rules!

pitfalls in testing?

“Come on, tell me something new. I already test my business rule during development.”, would you argue. And my response is: “Does it give your business a bug free experience?” The testing you have now is most likely not good enough: How many of you have not applied a last-minute change and you forgot to check the unchanged part. Or, how many of you did not check that it was your business that created the Essbase blocks (and not you yourself by testing it)? Or how many of deemed testing not needed as the change was “so easy”. All ending with a disaster on production? But no worry, there is a simple process that can help you and your team in producing bug free business rules. It is the extra safety line that will prevent you from falling. And no matter if you are new to business rule development or a seasoned veteran, you are all entitled to have one.

Last summer, I went on a multi-day hike on a via ferrata in Italy. A via ferrata is a mountain hike with a couple of dangerous and difficult passages which are secured with cables. You “walk” a via ferrata using a climbing harness that you secure to the cables. With the assurance of the “safety net”, it is real fun to climb and walk the steep passages, but without it your brain will shut you down with the shivers. And although me and my family have never slipped, I can assure you, that without a harness, we would not even talk about a via ferrata, let alone walk one. And of course you should not! (More on the via ferrata we hiked here.)

Developing business rules is like hiking a via ferrata; lots of challenges, and obstacles to climb, but also pitfalls to avoid. Very doable and even fun with the right safety “harness”, but without it, there is only the threat of misery ahead.

In this episode, I will take you along the three pitfalls in testing business rules and also give you some hints in how to avoid them:

pitfall 1 – reliance on integration and acceptance tests only

The classical approach with regard to testing is using integration tests and acceptance tests. These tests do have a specific place and function, but they are not adequate for testing the correct functioning of a business rule: integration tests only focus on the correct working of the combined application parts and acceptance tests discover issues too late in the process (and only if you have knowledgeable key-users available, which is not always the case).

The major issue with this approach is twofold:

  1. you depend 100% on the business to detect errors in your rules and
  2. it takes a lot of additional administrative work to fix bugs found on the acceptance environment.

The symptoms of this way of working are: lots of bugs reported during acceptance tests (and lots of administrative work to fix them) and several critical incidents on production shortly after a new release. Resulting in temporary outage of the application to the planners and considerable amount of rework for the developers, the testers and the support department.

You can avoid this pitfall. All you need is structural upfront tests of your business rule on the development environment. If you find bugs, you can easily correct them without much elaborate administrative work, as you are still on the development environment.

pitfall 2 – each developer using its own test sheets

The developer uses test sheets, great! But, every developer uses his/her own methodology. Most of the time, the sheets are only usable by the developer itself. And as only the developer knows the details of the processing logic, only he/she can check if the rule works as expected.

Very inconvenient when issues with the business rule arise; only the developer knows how the test sheet works, provided that it was not too long ago (do you still remember the exact details of your business rules a month later?).

The introduction of white box testing will change this completely! In a white box test, the developer not only applies the logic in the business rule, he/she also rebuilds the same logic into the expected result in Excel. When you test the rule you can easily compare the results against it. Very insightful and convenient. White box testing gives you also the following advantages: The logic in Excel can be verified by the end-user and, in addition, provides the documentation to the peer developers. You can easily add new test cases, if needed. The Excel formulas will automatically generate the new expected result. And lastly, with the white box test at hand, any team member can verify that a rule is working as expected, without the need to precisely know what the rule should do.

pitfall 3 – no test on block creation

This is the number one Essbase pitfall! Not testing on the creation of blocks. This does not happen on purpose, of course. Developers know of the block creation issue. It is just, that during the development process, the developer creates – unintentionally – the blocks with his/her test sheets. And when the rule gives the desired result, the developer omits to explicitly test on block creation and assumes everything is ok.

This is how it goes: the rule is promoted to acceptance and the end users start testing. But after running the rule, the results do not appear (no blocks are created), leaving the testers in frustration as they cannot even begin their tests. And only after a fair amount of analysis by the support team, debugging, bug fixing and release build, a correct rule (one that create blocks) can be delivered. And you have wasted a lot of valuable testing time. Not to mention, end-user goodwill. Completely unnecessary if the developer did fall into this pitfall.

A simple test on block creation would have avoided this waste of energy.

pitfall 4 – no clue with regard to the effect on the performance

Although the last pitfall mentioned here, it is certainly not the least dangerous. When developing and delivering rules, it is key that you check the effects on the performance in detail during development. There is nothing more frustrating than having passed the acceptance tests to only discover that the rule does not perform when applied to real life data.

is it really that bad with rule testing?

I guess you should tell me, but I have seen some really nasty business rule deliveries in the past.

One implementation had over hundreds of bugs in the acceptance environment, blocking even the acceptance process. Lots of frustration among the key users that wanted to complete their tests and high stress levels and lots of overtime in the development team. Not a good team vibe and not effectively building trust.

The team in this example stepped in all three pitfalls in testing business rules as mentioned in this article. Completely unnecessary. These can be easily avoided with the right testing method.

I will tell you more about that in the next episode: testing business rules the right way.

about the author

Arthur van den Berg


I love the calculation power of Essbase and the financial problems you can solve with it. I have developed, implemented and improved dozens of EPM solutions with Essbase at its core and I love sharing what I learned with professionals that need to build and/or support these solutions.

share this post