SAP Business Objects report regression testing and more precisely, automated comparisons of BO report datasets is a tricky but very important task. Maintaining accuracy and consistency of SAP BusinessObjects reports is critical. Automated SAP BusinessObjects testing allows to mitigate risks due to manual testing. The current state of the art is manual testing. But how is it done manually? There are Alt+tab users displaying report.rpt in two browsers, for instance in XI3 and BI4 and then compare outputs (going down to pixel level can be tricky for human eyes). Then, there are dualscreen users; but I am pretty sure this solution is less accurate compared to the Alt+tab method. Finally, there is what I used to do while performing migrations, 20 years ago, working on BOBJ and migrating from BO3 to BOv4: printing both versions and then overlaying the printed documents in front of a window to try and highlight differences! Again artisanal way. Plus, that was not ecologically-friendly! There are also two last categories: NO TEST and vague similarity test! So let’s get started with a professional way of performing regression testing!
[ulplocker id=”eCtG1LnHCqgB8PPN”]
BI tools, and especially SAP BusinessObjects, are now widely used within organizations to manage and drive businesses! Those products can be used to manage and report very sensitive data such as HR, Healthcare (patient data) or financial records. Thus, it is needed to ensure compliance and conformity, especially dealing with data following regulations such as SOX, Basel, HIPAA, FISMA, etc. We have developed solutions to help clients ensure compliance from a strict security perspective (everything started for us from this need!). We have a famous concrete use case: the US Department of Homeland Security making use of 360Eyes (https://wiiisdom.com/sap-business-objects/audit-impact-analysis-metadata/) and 360View (https://wiiisdom.com/sap-business-objects/security/) to ensure FISMA compliance. The current topic is not related to security, but to data itself or BO report datasets. Indeed, when dealing with conformity, you need to ensure that security is properly defined and that you can provide comprehensive outputs for internal and external auditing (again exactly what 360View and 360Eyes do). When demonstrating 360View, for instance, I am used to saying (real story based on my 20 years of experience as an administrator): “If end users have no access to their own content or cannot use it, they will contact the admin asking for immediate assistance, BUT if the same user can see the payroll of any employee, they won’t contact the admin saying they have access to all the payrolls, including yours!”. You can smile, but this is how it does work in the real world. Speaking more generally about cybersecurity, threats are often perceived as coming from outside (external hacking). However, recent research has marked internal threats (internal malpractice and human error) as equally dangerous to sensitive data. Thinking about Edward Snowden (NSA data breach), access control was not properly setup, and definitely was not under control. He was able to access – and did – more than he should have. No one caught it, obviously. Only very specific users should have the sufficient rights to access all the resources on a system. By the way, if someone gets access to normally restricted resources, there will be nobody to warn the administrator or anyone else from the IT team, just as I explained with the payroll.
But again, I will write something different regarding security management, documentation and audit in SAP BusinessObjects deployments. This document is about in-report data and datasets. Complying with regulations is also a major requirement! From a risk control perspective, regular testing of BI report data or output is mandatory (regression test, that is). As I am always saying, it is a full time job or a very regular, if not daily task! Let me highlight a few concrete examples! An organization’s BI landscape is composed of a BI tool, BOBJ in our case, some data to query and analyze, so, a database (ODB or Hana for example) or BW cubes, an OS (Windows or Linux), a WAS (Tomcat …), an ETL (Data Services…), an Enterprise Job scheduler (Control-M, $U, UC4, TWS, VTOM…), servers, etc. So many components! All these components also evolve (for instance, applying a BusinessObjects FP or SP, upgrading the OS etc.). Other examples could be changes in a universe or at a table level in the data warehouse. That’s why I am saying you need to run regression testing almost every day! This is also the case when performing major upgrades (BusinessObjects migration during months) like the future one to SAP BI4.2.
Within our 10 steps migration to BI4.2 path, step 7 deals with regression testing. I have even witnessed frozen migration projects because of this very sensitive step! Everything was perfectly fine from a pure technical perspective, but not enough time had been given to end users or key users in order to run all those tests. Using 360Bind assists in keeping track of the migration process and, above all, makes it safe! If, in parallel to your migration project, you need to repoint Web Intelligence reports from UNV universe to UNX universe in bulk (Step 3), regression testing is a clear necessity! (360View can bulk repoint Webi reports from UNV to UNX)
To sum up, there is a need to reduce the operational risk and thus, the cost of BI report Risk Control testing. This is exactly where our solution 360bind (https://wiiisdom.com/360bind/) can help, as it provides a way to mitigate those risks, save time and money! 360Bind is a no-brainer just like 360Plus is with mitigating risk around SAP BusinessObjects disaster recovery! (https://wiiisdom.com/sap-business-objects/backup-recycle-restore-deleted/) Let me share some concrete use cases, real life experience … as usual!
Case 1 is a US real-estate company asking for some information about 360Bind. After a webex demo, the client was OK to run a POC. Following our standard process, one of our pre-sales helped the client (wait, at that time, they were still a prospect, but you will soon see why I am saying client) install the product. Indeed, from a pure technical perspective 360 Bind is just a web app deployed on the BOBJ web application server (Tomcat, WebSphere, WebLogic, Netweaver, etc.) in only 5 minutes. 360Bind is only one module of our 360Suite web app, (each module in 360Suite can be purchased individually) including 360View for security management and bulk updates replacing the CMC, 360Plus for backup and promotion, replacing LCM, 360Cast for dynamic report bursting (prompt values and filter values are stored in a database) launched by using your Enterprise job scheduler, as a replacement for Publications; finally, 360Bind to automate regression testing. This is all part of the risk mitigation strategy you should set up for your SAP BusinessObjects platform. But let’s come back to our US company case. After the POC install, the pre-sales helped run a test to show the general workflow. The prospective client said they had just completed a migration to BI4.1 and had done their webi regression test manually. So, it was interesting to give it a try with their most important report dealing with margins (public information are communicated on the basis on this report!). Now it is time to explain how 360Bind works behind the scenes. There are several options but the concept is about generating instances, first using the same prompts or filters at different times or from different platforms (let’s say margin.wid generated in XI3 and BI4.2).
Then 360Bind engine automatically compares the 2 outputs and highlights the differences: data in tab (e.g. figures) or charts (going down to pixel level!). To sum up, 360bind automates webi, deski or crystal report updates for easier comparison of report output in an earlier version of BO, against the same report output in a converted or migrated version of BO or over time (simply because the report itself or a universe was modified).
So, our presales ran the margin report in XI3, then in BI4.1 and then performed the comparison! That’s the exact moment, where we can say they were no longer a prospect but a client. Indeed the margin and revenues were different after the migration! (TO BE 100% CLEAR, the same data coming from the same database returns a $1-billion margin in XI3 and a $1.2-billion in BI4.1!!!!). The client’s comment was: “With 1-2 patches a quarter, this takes 40+ hours to validate, and doing it manually is not as precise as the automated counterpart.” And at that time it was only about BOBJ patches, not the other one (universe change, SO patch, BW upgrade, ETL modification …).
Case 2 is about migration, but there is almost nothing to say. Anybody can easily understand the added value of automating regression testing on webis, crystals, etc. But let me just share concrete figures. A chemical giant running SAP BusinessObjects BI4 on top of BW (exactly where we have a unique expertise) spent 250 man days (another Alt+tab man) running manual regression testing only to find a 5% error rate! With 360Bind, this was reduced to 15 days and 100% accurate!
Case 3 can be linked to case 1, performing regression testing while the BusinessObjects platform is changing. In that case it is a city using large numbers of pre-build reports and universes provided by OEM BOBJ vendors. This is also the case in Healthcare with EPIC for example, in HR with ADP or banking with Metavante. Indeed there is a big community of software vendors providing OEM BO as their reporting tool. Organizations (here, the city) purchasing their software usually go for default BO. Then from time to time vendors deliver new universes clients need to deploy without any real notice.
360vers (https://wiiisdom.com/360vers/), our solution for versioning can help highlight the differences between 2 versions of a same universe (created, modified or deleted object and impact analysis) but 360bind will help ensure the new universe will not impact existing BO reports by automating regression tests! A similar case can be used thinking about cloud computing. SAP BI is going to the cloud; great news for our business… In that scenario, one relies on the cloud provider who may be changing OS, applying SP, etc. every now and then (without any notice).
Case 4: I always like talking about US government agencies. They manage such sensitive data (like public companies in some cases) that they cannot externalise/outsource this part of the migration project (i.e.: to offshore companies). There is no other way than having a federal agent (for example) to perform the regression tests! In that specific scenario having an automated regression test tool is valuable just because it helps to save costs!
The last case is a Bank needed to apply a SAP BusinessObjects SP (BI4.1 SP6). The old process was: the IT team decides to apply a BO SP and then asks end users or key users to take time to test their reports. Thus, busy end users or busier key users need to make sure BO reports produce the exact same content between the old version and the new version of SAP BusinessObjects (consistency). Those end users were probably using the Alt+tab or the window method. 360Bind helps change the way the IT department is working with end users or key users! 360Bind is part of a brand new conformity process: more secure, more reliable and cheaper (any organization is dreaming about those basic ROI for easier, faster, safer and cheaper SAP BusinessObjects upgrades). Indeed they first used 360Bind during this BI4.1 SP6 upgrade and realized that some report came out with mismatching results (data displayed is different before and after applying the SP). Digging into the 360Bind results, they discovered that this SP was buggy with their configuration (360Bind allows to generate, for conformity purposes, logs or comparison summary showing that BO reports produce the exact same content or not between the old version and the new version). The client’s first comment was that 360bind should be sold to SAP to ensure better SP quality! Then they realized that this can be invaluable for key users or end users. In their case, by using 360Bind they can increase the tests scope, and effectiveness, as well as improve report validation accuracy. Among everything, 360Bind free end users or key users from tedious manual testing! The new process is: the IT team runs some tests using 360Bind and then just asks end users to spend their time on regression issues 360Bind highlighted!
There are some regression testing tools on the market, but they are generic. 360Bind is 100% SAP BusinessObjects oriented! For instance, there are several options being 100% linked to the SAP BusinessObjects world for including or excluding style, structure from the comparison, going down to comparing deski output and webi output following report conversion! 360Bind allows users to automatically compare SAP BusinessObjects report outputs between different SAP BusinessObjects installations (major versions or SP, FP) or over time to find out inconsistencies or errors.
These 5 scenarios can happen tomorrow in your organization. That’s why using 360Bind is a nobrainer! It is like for backup disaster recovery in SAP BusinessObjects, both deal with risks and conformity in SAP BusinessObjects.
To conclude you should act now! How can your company invest (in some cases millions) in SAP BO without relying on a non-manual regression testing strategy? This is one of the backbones of your deployments! If you are a SAP Business Analytics administrator, share this with your manager to justify the purchase. If you are a BI manager or a Risk Manager or an Alt-tab user, contact us! It is all about increasing efficiency and reducing risks while ensuring your SAP BusinessObjects reports are producing accurate and consistent output. All those cases/stories are part of understanding how we develop our solutions to help clients streamline and extend their SAP BusinessObjects deployments. We can just state: Those applications constitute a set of solutions for BO admins, created by BO admins.
Find out more there https://wiiisdom.com/360bind/
[/ulplocker]