Data Migration

Migrating data  can be a challenging  task depending on the size of the data, platform
and systems involved with the migration process. The age or the nature of data can also
add to the complexity of the migration process. Legacy data or old data needs to be
re-validated and cleaned to prevent any major issues when the data has been
migrated just like you have with Data Warehouse system.
ETL applications like Talend , Pentaho and Microsoft SSIS can be utilized in this process.
The structure of the data plays a major role in determining the schema and the layer
of the data.Texted based systems and RDBMS have different properties that needs to be
examined and changed. The final properties to be examined is the platform.
Migrating data from SQL Server 2008 to SQL Server 2012 is much easier
than migrating from SQL Server 2008 to Oracle 12c or 11g due to different datatypes,
objects , schema and properties. In some cases the underlying objects like
Stored procedures, functions and cursors has to be manually re-created or
altered depending on the application utilized in the migration. Each database vendor
may have internal or external systems that can be utilized for the migration process
which requires connection to the data source using OLEDB or  ODBC properties.
SQL statements may and various objects like stored procedures may be utilized to
migrate data from one platform to another using linked connection protocols like
linked Server in SQL Sever to extract data into a staging tables for cleaning process.
  1. SQL Server Import and Export Wizard
When I did a Postgres migration to SQL Server using the sample database
DVDRental it is much easier to save file and open it from SSIS. You can view
the underlying tasks and have option to alter the object definition to match to
SQL Server without throwing errors due to datatypes definition. This task can be
also completed using the task wizard.
  1. MySQL Workbench
  2. Oracle SQL Developer(Migration Wizard)
  3. Oracle JDeveloper
  4. Microsoft SSIS Transfer Database Task
  5. Sybase Migrate Database Task (SQL Anywhere)
  6. Microsoft SQL Server Migration Assistant (Oracle, Sybase, Access, DB2)
For more readings

Comments

Post a Comment

Popular posts from this blog

ETL ( Automation vs Manual )

ETL applications like Talend, SSIS and Pentaho can simplify ETL or Data migration by eliminating or reducing the amount of SQL coding needed to complete the process. However same functions performed by ETL applications can be performed by scripting using various dialect of SQL and programming language like C# or Java. The downside of scripting manually without an ETL application is the need of connecting to data source and destination source. However most vendors like Microsoft SQL Server provides developers the option through linked server using SQL Server Management Studio or Oracle MySQL Workbench for MySQL.  Depending on the application a third programming language is need as interface like Java or C++ when performing the process from a developmental application. This will allow both the source and destination sources to be linked.  Another problem with the manual process is the complexity of SQL coding needed which needs to be altered constantly and the length of the codes invol
Read more

SSIS, SSAS & Stored Procedure

Image
In the following series SSIS would be utilized to extract data from Stored Procedure which contain query that is linked with SSAS MDX query via linked Server. Talend requires additional components while SSIS has fewer components but the output is the same. First process in extracting the query through SSAS via Linked Server is to create a linked server. Linked Server Second process is to create the stored procedure to extract the data. Stored Procedure definition with the MDX and regular SQL qeury Third process is to create an SSIS package. In the Control Flow component you need  two task which the Execute SQL Task and Data Flow task.  The fourth step is to create the database connection to the source and destination systems. In this case only one database system is being utilized. Another big difference between SSIS and Talend is that Talend  works well with JDBC but JDBC is allowed in SSIS rather .NET, OLED, ODBC etc which makes the connection
Read more