Most of modern ETL procedural tools Allow execution in parallel mode, In other words, loading is not done data by data but in blocks.
Types of parallelism
-
Data partitioning. In this circumstance, each node processes a specific number of data (as an example, 1000 rows). The timing and specific type of partitioning depends on the type of database being accessed. In the case of a Deposit, everything is usually partitioned, because on the disks we find chunks of tables with data.
-
Pipelining. This type of parallelism allows, once the data is divided, to also partition the operations performed on it, such as adding a field “Y” to each row. In other words, allows structural-level modifications.
-
Components (edit) (Edit). Components are defined that assign a reference to a field, for example, a letter, and they are reused each time that same field needs to be changed again in the other rows. In other words, this type of parallelism procedure allows you to create logic and reuse it.
Advantages of parallelism
-
Reduces processing times.
-
Positive business impact.
-
Better use of resources.
Sometimes it is not feasible to use it.
It is always desirable, and even more so when working with large volumes of data, design ELT processes so that they can run in parallel mode. The thing is that certain processes do not allow it, since they necessarily require sequential execution (as an example, a procedure “A” that depends on the completion of a procedure “B”).
(function(d, s, id) {
var js, fjs = d.getElementsByTagName(s)[0];
if (d.getElementById(id)) return;
js = d.createElement(s); js.id = id;
js.src = “//connect.facebook.net/es_ES/all.js#xfbml=1&status=0”;
fjs.parentNode.insertBefore(js, fjs);
}(document, ‘script’, 'facebook-jssdk'));
