Extract Transform Load(ETL) vs Extract Load Transform (ELT)

Data Warehouse, Database Migration and Database Development utilize
 Extract Transform Load (ETL) Technologies to clean and restructure data prior to loading into the
final environment. Database Management Systems (DBMS) not 
Relational Database Management Systems (RDBMS) may consist files and XML
 files generated from internal and external systems  with various format.Poorly designed RDBMS may contain errors from internal and external applications makes it hard to utilized the data for reporting purpose.
Same applications utilized for ETL like  Talend or SSIS can also be utilized for Extract Load Transform
(ELT) but the concept varies both implementation and design 


  • ELT can be used when moving data from platform that are manufactured by the same vendor but is a newer version with the same datatype and environment.
  • ELT can also be utilized when working on a large data. The entire data can be extracted from the source like a Data Warehouse or existing RDBMS and dumped into staging tables with corresponding environment.   Afterwards it can be transformed using ELT methodology to clean any errors or restructure the data before it is dumped into the final destination.  
  • An underlying triggers within a RDBMS can be utilized to perform complex ELT task in conjunction with various objects like stored procedures,User-defined functions and ,cursors, packages(Oracle) to perform ELT task instead of the traditional ETL applications.
Popular ETL 
  • Microsoft SSIS
  • Oracle Data Integrator (ODI) & Data Warehouse Builder
  • Talend Open Studio
  • Pentaho  
For additional readings 
ETL VS ELT Difference is in the how

Comments

Post a Comment

Popular posts from this blog

ETL ( Automation vs Manual )

ETL applications like Talend, SSIS and Pentaho can simplify ETL or Data migration by eliminating or reducing the amount of SQL coding needed to complete the process. However same functions performed by ETL applications can be performed by scripting using various dialect of SQL and programming language like C# or Java. The downside of scripting manually without an ETL application is the need of connecting to data source and destination source. However most vendors like Microsoft SQL Server provides developers the option through linked server using SQL Server Management Studio or Oracle MySQL Workbench for MySQL.  Depending on the application a third programming language is need as interface like Java or C++ when performing the process from a developmental application. This will allow both the source and destination sources to be linked.  Another problem with the manual process is the complexity of SQL coding needed which needs to be altered constantly and the length of the co...
Read more

Data Migration

Migrating data  can be a challenging  task depending on the size of the data, platform and systems involved with the migration process. The age or the nature of data can also add to the complexity of the migration process. Legacy data or old data needs to be re-validated and cleaned to prevent any major issues when the data has been migrated just like you have with Data Warehouse system. ETL applications like Talend , Pentaho and Microsoft SSIS can be utilized in this process. The structure of the data plays a major role in determining the schema and the layer of the data.Texted based systems and RDBMS have different properties that needs to be examined and changed. The final properties to be examined is the platform. Migrating data from SQL Server 2008 to SQL Server 2012 is much easier than migrating from SQL Server 2008 to Oracle 12c or 11g due to different datatypes, objects , schema and properties. In some cases the underlying objects like ...
Read more

SSIS, SSAS & Stored Procedure

Image
In the following series SSIS would be utilized to extract data from Stored Procedure which contain query that is linked with SSAS MDX query via linked Server. Talend requires additional components while SSIS has fewer components but the output is the same. First process in extracting the query through SSAS via Linked Server is to create a linked server. Linked Server Second process is to create the stored procedure to extract the data. Stored Procedure definition with the MDX and regular SQL qeury Third process is to create an SSIS package. In the Control Flow component you need  two task which the Execute SQL Task and Data Flow task.  The fourth step is to create the database connection to the source and destination systems. In this case only one database system is being utilized. Another big difference between SSIS and Talend is that Talend  works well with JDBC but JDBC is allowed in SSIS rather .NET, OLED, ODBC etc which makes t...
Read more