Health Service Case Study

Rapid Implementation Of Data Transformation Assurance

Learn how IDS delivered a repeatable data migration and transformation assurance solution for a major Health Service digital transformation programme

Get In Touch Book An iData Discovery Call

Follow us

A central player in the UK health sector undergoing a digital transformation of one of its vital internal applications, the delivery of new and updated pLDM. Testing had become inconsistent, and stakeholders were losing confidence in the transformation process. The data migration test team were struggling to keep pace with the volume of data being migrated and transformed. Their approach had become a risk to the project.

IDS were manadated to implement a mechanism to assure that the migration and transformation of data had quality gates and to offer unequivocable certainty that the data transformation rules had been thoroughly tested. The goal was to assure the data was available in the correct format and location in the target database.

Progression From A Manual User Interface And SQL Scripting

The test, release management and domain leads on the programme recognised the importance of completing data migration testing as quickly as possible. But the traditional manual user interface and SQL scripting approach they had adopted was tightly constricting the programme’s progress. It was slow. Too slow.

The requirement from the client was to recover the lost time in testing due to unforeseen blockages, and to accelerate the process around data migration and transformation assurance.

Given the critical nature of the data being transformed it was imperative that a solution was provided to mitigate the inherent risk associated with migrating sensitive and critical data in large volumes.

Automated Data Migration And Assurance With iData

iData is IDS’s market-leading data transformation and migration assurance engine. Based on many hundreds of applications and huge volumes of data, we have developed this unique product to automate the process of validating transformation rules, and efficiently transferring accurate data to a new target database.

iData uses an automation process to generate template scripts that can be easily edited to drive the focus of each test to assure that the transformation process is assured. This addition of templates lends itself to rapid script creation and faster feedback loops for each data migration cycle.

Freeing Resources By Eliminating Manual Testing

The growing complexity of the client’s testing coverage was compounding the inherent challenge of manual testing. The team had an overhead of 20- man days of testing to complete for each and every deployment of the migrated data. This exposed the manual testing approach to human error, which could be catastrophic for the organisation.

Faced with the increasing cost and complexity of their manual testing approach, the pLDM transformation programme benefited from IDS’s end-to-end service to help deliver a leaner, more cost-effective process through our specialist iData migration transformation automated assurance capabilities.

Efficient Automated Data Assurance – Implementing The Solution

One of the biggest concerns to the project was to offer certainty that no patient records would be lost or altered in any way during the migration process.

Given that the project had already been subject to significant delays, it was imperative that testing was completed on time, without any reduction in quality and coverage.

Knowing the data migration strategy and design, we decided that the most efficient approach would be to focus on a staged implementation of iData and to execute our testing at each cycle of migration.

Using iData’s inbuilt learning capabilities, we edited the template scripts in-line with the transformation rules which had been stipulated by the project team.

Our scripts reflected more than 250 transformation rules, customised to meet the programme requirements.

We implemented iData and appropriate scripts at each stage. This provided assurance that each transformation had been assumed to be correct and provided an audit trail for any lost data and at which point the data had been dropped. With a clear understanding of the costs and return on investment opportunities, we began the process of establishing an automated data migration assurance solution for the execution and re-execution of tests that enable the transformation rules to be exercised every time a transform and load took place.

Cutting Testing Times:
From 4 weeks to 30 minutes!

With manual testing, it could take up to 4 weeks to complete. With iData as a solution, we calculated that the entire assurance process could be completed in less than 30 minutes and provide 100% coverage across 330,000 records.

If they had chosen to execute the same level of automation using manual techniques, the cost and time required would deplete the test budget without getting the required assurance on the quality of the product/application.

Having achieved this, the programme has increased the volume of testing as well as improvements in:

  • Transform and load reliability
  • Overall quality was improved on-demand reporting metrics
  • Efficient data transformation testing, improved testing practices and cohesion with the data team
  • Through the specialist implementation of IDS’s iData Migration and Transformation assurance solution, the client now benefits from a considerable saving in staff utilisation, freeing up skilled resources to focus on manual and exploratory testing or other areas of concern.
  • The quality of testing is greatly improved, the client has greater confidence in the application under test as the possibility of human oversight is eliminated.

Eliminating The Testing Bottleneck

  • iData delivered a data migration and transformation assurance solution
  • iData identified and rectified transformation errors across 330,000 records
  • iData captured lost records and assisted in locating the point at which those records had been lost
  • iData provided a mechanism to build test assets ahead of time enabling us to be ready whenever the transform and load cycle took place
  • iData provided a standardised approach to data transformation, migration and assurance
  • Significant time is saved and faster feedback on the state of application quality
  • Reporting was integrated into Jira for tracking, auditing and reporting
  • The client now has the benefit of a fully functioning, repeatable migration assurance test pack working as part of a continuous delivery model
  • Enabling the client’s data teams to build-deploy-test as often as required, resulting in much more robust export, transform and load practices
  • Senior stakeholders were given data certainty – absolute confidence in the data being deployed