Beep Data Management Engine

Overview

The Beep Data Management Engine is a comprehensive, multi-target framework that provides powerful data access patterns, unit of work implementations, change tracking, and enterprise-level data management capabilities for modern .NET applications.

Built with extensibility in mind, the engine supports a wide variety of data sources from traditional relational databases to modern vector databases, file formats, and custom data providers through its plugin-based architecture.

Key Features

Engine Statistics

Installation

Install the package using one of the following methods:

Install-Package TheTechIdea.Beep.DataManagementEngine

dotnet add package TheTechIdea.Beep.DataManagementEngine

<PackageReference Include="TheTechIdea.Beep.DataManagementEngine" Version="2.0.32" />

Basic Usage

Here's a simple example to get you started:

using Autofac;
using TheTechIdea.Beep;
using TheTechIdea.Beep.ConfigUtil;
using TheTechIdea.Beep.Logger;
using TheTechIdea.Util;

// Initialize DME Editor using Autofac
var builder = new ContainerBuilder();
builder.RegisterType<DMEEditor>().As<IDMEEditor>().SingleInstance();
builder.RegisterType<ConfigEditor>().As<IConfigEditor>().SingleInstance();
builder.RegisterType<DMLogger>().As<IDMLogger>().SingleInstance();
builder.RegisterType<Util>().As<IUtil>().SingleInstance();
builder.RegisterType<ErrorsInfo>().As<IErrorsInfo>().SingleInstance();
builder.RegisterType<JsonLoader>().As<IJsonLoader>().SingleInstance();
builder.RegisterType<AssemblyHandler>().As<IAssemblyHandler>().SingleInstance();

var container = builder.Build();
var dmeEditor = container.Resolve<IDMEEditor>();

// Initialize required configurations for IDataSource support
InitializeBeepDefaults(dmeEditor);

// Create a data connection
ConnectionProperties connectionProps = new ConnectionProperties
{
    ConnectionName = "MyDatabase",
    DatabaseType = DataSourceType.SqlServer,
    ConnectionString = "Server=localhost;Database=MyDB;Integrated Security=true;"
};

// Add connection to configuration
dmeEditor.ConfigEditor.AddDataConnection(connectionProps);

// Get data source and open connection
IDataSource dataSource = dmeEditor.GetDataSource("MyDatabase");
dataSource.Openconnection();

// Create UnitOfWork for your entity
var unitOfWork = new UnitofWork<Customer>(
    dmeEditor, 
    "MyDatabase", 
    "Customers", 
    "CustomerId"
);

// Load data
await unitOfWork.Get();

// Work with your data
unitOfWork.New(); // Create new customer
var newCustomer = unitOfWork.CurrentItem;
newCustomer.Name = "John Doe";
newCustomer.Email = "john@example.com";

// Save changes
await unitOfWork.Commit();

Supported Data Sources

The Beep Data Management Engine supports a wide variety of data sources:

Documentation Sections

Latest Updates

Complete Documentation Index

Getting Started

Core Concepts

Data Management

Editor Classes

Advanced Topics

Tools & Reference

Examples & Best Practices

🛠️ Data Management Editor Classes

The Beep Data Management Engine includes a comprehensive suite of editor classes that provide specialized functionality for different aspects of data management:

🔧 Helper Classes & Utilities

BeepDM includes a comprehensive collection of helper classes that extend core functionality and simplify common data management tasks:

// Get all 39 predefined connection configurations
var allConfigs = ConnectionHelper.GetAllConnectionConfigs();

// Filter by category
var vectorDatabases = allConfigs.Where(c => c.DatasourceCategory == DatasourceCategory.VectorDB).ToList();
var rdbmsSystems = allConfigs.Where(c => c.DatasourceCategory == DatasourceCategory.RDBMS).ToList();
var fileSources = allConfigs.Where(c => c.DatasourceCategory == DatasourceCategory.FILE).ToList();

Console.WriteLine($"Found {vectorDatabases.Count} vector databases");
Console.WriteLine($"Found {rdbmsSystems.Count} RDBMS systems");
Console.WriteLine($"Found {fileSources.Count} file formats");

                        // Add all configurations to your DME setup
                        foreach (var config in allConfigs)
{
    if (!dmeEditor.ConfigEditor.DataDriversClasses.Any(d => d.GuidID == config.GuidID))
    {
        dmeEditor.ConfigEditor.DataDriversClasses.Add(config);
    }
}

🏗️ Core Infrastructure Classes

Essential infrastructure components that provide foundational services across the engine:

🔄 UnitOfWork Ecosystem

Complete suite of classes supporting the Unit of Work pattern implementation:

🔄 ETL & Data Processing

Comprehensive Extract, Transform, Load capabilities and data processing tools:

📁 File Management System

Comprehensive file processing and data source capabilities:

⚙️ Workflow & Rules Engine

Powerful workflow orchestration and business rules processing:

🎛️ Specialized Manager Classes

ConfigEditor's specialized managers providing focused functionality:

Overview

The Beep Data Management Engine follows a modular, layered architecture designed for scalability, maintainability, and extensibility: