How Local Councils Should Avoid Data Migration Failure
12 Aug, 2022 | IDS
Any government or public sector organization is responsible for collecting and storing data on behalf of the public. This responsibility requires institutions, like local councils to periodically review their ERP systems.
They must identify if they’re still fit for purpose, facing end-of-life and plan to migrate their data safely to a new system in response.
Local council data migration can be a long process and it’s not a ’one size fits all’ process. ERP migration can take weeks or months depending on how much data there is to transfer. Also, there’s the golden rule to put data quality first - avoiding data quality issues and catastrophic data migration failure.
Focus on Data with Value
Firstly, teams must decide what type of data they need to achieve their data migration objectives.
Questions to ask for value-mapping exercises, and avoid hoarding data, to inform data migration planning include:
What practical applications will this data have and who will utilize it most often?
How will this data improve our council’s decisions and services?
What legislative requirements or restrictions must we consider?
This way, councils make sure time and effort is not wasted on the wrong data that causes complete data migration failure at the first hurdle.
Data Migration Planning
The data migration planning strategy will be two-fold, depending on what type of ERP legacy and destination data sources are being used. For example, a cloud data warehouse migration may require a phased approach, whereas a traditional migration would need a ’big bang’ approach.
Then, while planning the budget, objectives, time, scope, and resources needed for the migration, it’s time to focus on the data itself.
It’s then crucial for councils to ask themselves the following questions during data migration planning:
Is all the relevant data we need available?
How is the data going to be used post-migration?
What risks should we be prepared for?
How will this migration affect our users and stakeholders?
Council data must also be backed up in the event of unforeseen disaster. If in doubt, consult with an expert or any organization who has handled your data in the past to take back control of any bad data.
Identify Issues Before Migrating
It’s important to identify any issues in the data before migrating it - like poor mapping, blank, missing or whitespace values, duplicates, incomplete entries etc - to prevent sensitive citizen data from becoming lost, damaged, or inaccessible.
To avoid data migration failure, successfully finding and cleansing anomalies in the data before the migration, lets you fix them before risking the worthwhile nature of the migration.
During this phase, rules will be inbuilt to resolve any problems that may come up. Otherwise, if you identify data quality issues too late in the migration process, this delays the project and results in chaotic quick-fixes and unintended breaches in security.
The Right Data Migration Team
A data migration team should include non-technical project managers as well as subject-matter experts. They should be involved in some capacity throughout the process and lead the initial data mapping exercise.
This person is essential for ensuring the success of the council data migration and that any sensitive legal requirements for data are met. They will also accommodate for other stakeholder needs, like the land charges department, that rely on statistics.
This expert in the data migration team should also advise councils to understand their legal data retention requirements and help decide which data ought to be migrated.
These teams should also include an individual with technical knowledge of the source system. This person will have a good understanding of the source database and work closely with the above expert to convert general rules into technical requirements. Equally, they can also read into the underlying data, helping inform mapping decisions.
These teams should include an individual with insider, technical knowledge about the source system. This member works closely with the expert, and any other relevant data migration team members. As a result, general requirements are converted into technical, transformation requirements and prepares data to help inform mapping decisions.
Test Drive the Data
Test data management practices are used to assure data quality after a local council data migration. To avoid data migration failure at the final hurdle, quality assurance practices, like ETL, ensure data is trustworthy and error-free before going live in its new system.
These processes ’test drive’, validate and simulate live data, post-migration, but in a non-production environment. The data’s quality is, therefore, assured, certifying the council has clean, compliant, and accurate information before migrating to the production environment.
Records must be tested at different stages, compliantly, to touch as many mapping points as possible to align with the new system requirements. This sets up council teams for rapid migration and successful testing, keeping citizen’s data safe and secure.
IDS have a team of experts who work on your council's data migration project from start to finish. The Kovenant™ methodology is a repeatable, standardized framework which has managed risk for local council migration projects.
Kovenantä has been applied to public sector migration strategies and helped plan, execute, secure, and assure data migrations with 50% savings in time and cost.
To know more about Kovenant™ and how it can be applied in your council, get in touch with us today.