Automation Data Sources

Bridging workflow nodes to real data through a single provider seam

DataSourceAutomationNode reads from configured BeepDataSources (databases, files, APIs and other providers) during workflow execution. The framework does not depend on a concrete data stack: the node resolves an ambient provider, so the same diagram can run against a test source in staging and production in release.

Table of Contents

Provider Contract

IAutomationDataSourceProvider is deliberately small — query and connectivity, both async, both cancellation-aware:

public interface IAutomationDataSourceProvider
{
    Task<IReadOnlyList<Dictionary<string, object>>> QueryAsync(
        string dataSourceName,
        string entityName,
        string query,
        int maxRows,
        CancellationToken cancellationToken = default);

    Task<bool> TestConnectionAsync(
        string dataSourceName,
        CancellationToken cancellationToken = default);
}

Either entityName or query may be supplied: the node passes both through and the provider decides how to interpret them (a raw SQL source typically uses query; an entity-oriented source uses entityName).

Registry

AutomationDataSourceRegistry is an ambient single-provider slot:

MemberDescription
ProviderGets or sets the active provider. Setting null disables real data access.
HasProviderTrue when a provider is registered.
Clear()Removes the registered provider (tests use this between cases).

When a node executes and no provider is registered, it fails with a clear error rather than silently returning no rows.

DataSourceAutomationNode

PropertyTypeDescription
DataSourceNamestringRegistered data source to read from.
EntityNamestringEntity/table to read when not using a raw query.
QueryStringstringRaw query text passed to the provider.
QueryParametersDictionary<string, object>Parameter values for the query.
OutputFormatstringShape of the emitted data (for example rows as dictionaries).
MaxRowsintRow cap applied at the provider.
UseCache / CacheTimeoutbool / TimeSpanOptional result caching for repeated reads.

The node also exposes SetDataSourceManager(IDMEEditor, IDMLogger, IErrorsInfo) for hosts that already run a BeepDM editor, so logging and error reporting flow into the same pipeline as the rest of the application.

Wiring the BeepDM Provider

Implement the interface over your BeepDM editor (or any other stack) once at startup; every datasource-backed node in every workflow then works:

public sealed class BeepDmProvider : IAutomationDataSourceProvider
{
    private readonly IDMEEditor _editor;
    public BeepDmProvider(IDMEEditor editor) => _editor = editor;

    public async Task<IReadOnlyList<Dictionary<string, object>>> QueryAsync(
        string dataSourceName, string entityName, string query, int maxRows,
        CancellationToken cancellationToken = default)
    {
        var ds = _editor.GetDataSource(dataSourceName);
        var result = ds.RunQuery(entityName, query, maxRows);
        return result.Rows.Select(r => new Dictionary<string, object>(r)).ToList();
    }

    public async Task<bool> TestConnectionAsync(string dataSourceName,
        CancellationToken cancellationToken = default)
        => _editor.GetDataSource(dataSourceName)?.ConnectionStatus == ConnectionState.Open;
}

// Register once at startup
AutomationDataSourceRegistry.Provider = new BeepDmProvider(editor);

// Configure a node (in code or through the property grid)
var node = new DataSourceAutomationNode
{
    Name = "Load Orders",
    DataSourceName = "Sales",
    QueryString = "SELECT * FROM Orders WHERE Status = @status",
    MaxRows = 1000
};
node.QueryParameters["status"] = "Open";

Secrets Stay Out of Diagrams

Nodes reference a data source name; credentials live in the credential vault or in the BeepDM connection configuration, never in the diagram file.