Simple Info apps

ETL Editor

Overview

The ETL class, implementing the IETL interface, is responsible for managing the Extract, Transform, and Load (ETL) processes within the application. It facilitates the movement and transformation of data between different sources, handling entity structures, and executing data scripts.

Features

  • Script Management: Handles the creation, execution, and management of ETL scripts.
  • Data Copying: Facilitates the copying of data between different data sources and entity structures.
  • Error Handling: Provides robust error handling and logging during ETL processes.
  • Progress Reporting: Supports progress reporting for long-running ETL operations.

Properties

  • DMEEditor: Reference to the IDMEEditor instance for accessing various data management functionalities.
  • RulesEditor: Instance of IRulesEditor for managing data rules.
  • Passedargs: Stores arguments passed during ETL operations.
  • ScriptCount: The count of scripts to be executed.
  • CurrentScriptRecord: Keeps track of the current record being processed in the script.
  • StopErrorCount: The threshold for stopping the ETL process on encountering errors.
  • LoadDataLogs: Logs related to data loading.
  • Script: Stores the ETL script header information.

Methods

Script Creation

  • CreateScriptHeader: Creates a header for an ETL script.
  • GetCreateEntityScript: Generates ETL scripts for creating entities.
  • GetCopyDataEntityScript: Generates scripts for copying data between entities.

Data Copying

  • CopyEntitiesStructure: Copies the structure of entities between data sources.
  • CopyEntityStructure: Copies the structure of a single entity between data sources.
  • CopyDatasourceData: Copies data for all entities in a data source.
  • CopyEntitiesData: Copies data for specified entities between data sources.
  • CopyEntityData: Copies data for a single entity between data sources.

Script Execution

  • RunChildScriptAsync: Runs child scripts asynchronously as part of a parent script.
  • RunCreateScript: Executes the script for creating entities.
  • RunCopyEntityScript: Executes a script to copy data from one entity to another.

Import Methods

  • CreateImportScript: Creates a script for importing data.
  • RunImportScript: Runs the import script.

Utility Methods

  • InsertEntity: Inserts an entity into a data source.
  • LoadETL: Loads an ETL script from storage.
  • SaveETL: Saves the current ETL script to storage.

Events

  • PassEvent: An event handler for passing events within the ETL process.

Usage

The ETL class is used in scenarios where data needs to be extracted from one source, potentially transformed, and then loaded into another source. It’s vital in situations involving data migration, synchronization, or integration across different systems or databases.

Note

This documentation provides a summary of the ETL class functionalities. Detailed implementation and usage should be referred to in the source code and additional documentation for specific methods and properties.