In our company we have to dismiss a software and introduce a new one. I have received the task to follow and take care of the whole data migration.
We use 3 different data sources (each containing different kind of information), whose records have to be merged into a unique target DB. Since the new DB has a different schema also some mapping steps are required to coincile data from the two environments.
Technically it is everything clear, but I have to describe in a "proof of concept" the migration process. About data migration is there a general pattern that could be used to describe it?
I will describe the involved systems, which data will be moved, the mapping tables used and the planned tests to ensure data quality. However can anyone suggest a different approach? Maybe in this way I can realize that I am missing something in my documentation.
Best Answer
Data migration as you describe it is similar to the Extract, Transform and Load (ETL) process; which is mostly used to load operational data into data warehouses, but can be generalized to tackle any data migrations. A number of strategies related to ETL can be found here.