Rules Engine & Parsers

Overview

The Rules Engine lets you attach named business-logic units to any data pipeline without coupling rule code to controllers or service classes. Every rule implements one interface, returns a typed output bag, and is auto-discovered from assemblies — no manual wiring required.

Folder map

Rules/
  RulesEngine.cs                   core engine (register, solve, audit)
  RulesEngine.ExpressionEvaluation.cs   shunting-yard + RPN evaluator
  RuleParserFactory.cs             pluggable parser registry
  RuleCatalog.cs                   lifecycle-managed rule-structure store
  RuleRegistry.cs                  assembly-scanning auto-wiring
  RulesParser.cs                   default tokenizer-based parser
  Tokenizer.cs                     stateless lexer
  InMemoryRuleStateStore.cs        ConcurrentDictionary state for stateful rules
  BuiltinParsers/
    CsvColumnParser.cs             col[N] / row.col[N] references
    FormulaParser.cs               Excel-style SUM(A,B), A1 cell refs
    SqlWhereParser.cs              field = 'v', IN (...), BETWEEN, IS NULL
    JsonRuleTreeParser.cs          n8n-style JSON node tree
    NfelParser.cs                  Natural Formula Expression Language
  BuiltinRules/
    Compute/       arithmetic, IfElse, MapValueTable
    Date/          DateDiff, AddDatePart, business hours
    DQ/            IsNullOrEmpty, IsInRange, MatchesRegex, IsUnique...
    Enrich/        AddMetadataFields, LookupEnrich, MergeFields
    Flow/          CircuitBreaker, RetryWithBackoff, DeadLetterQueue
    Notify/        EmitAuditTrail, LogMessage, SendWebhook
    Numeric/       Clamp, RoundNumeric, GetEntityAggregate
    Query/         RunScalarQuery, RunParameterizedLookup, SetFromQuery
    Record/        SetFieldValue, CopyField, IncrementField, ConditionalSetField
    Route/         RouteOnCondition, SplitIntoBuckets, ThrottleRule
    Schema/        EnforceNotNull, ValidateSchemaFields
    Security/      hashing, masking, tokenisation
    Transform/     string cleaning and reshaping
    Validate/      email, phone, URL, credit-card, SSN

Common parameter conventions

Quick-Start — Three lines to run a rule

// 1. Build registry once at startup
var registry = new RuleRegistry(editor);
registry.Discover();   // scans loaded assemblies automatically

// 2. Solve a named built-in rule
var (outputs, result) = registry.Engine.SolveRule(
    "DQ.IsNullOrEmpty",
    new Dictionary<string, object> { ["Value"] = "alice@example.com" });

bool isEmpty = (bool)result;   // false

Subscribing to audit events

registry.Engine.RuleEvaluated += (_, e) =>
    Console.WriteLine($"{e.RuleKey} | ok={e.Success} | {e.Elapsed.TotalMilliseconds:F1} ms");

RuleEngine API

public partial class RuleEngine : IRuleEngine
{
    public RuleEngine(IRuleParser parser);

    // Fires after every SolveRule call (success or fail)
    public event EventHandler<RuleAuditEventArgs> RuleEvaluated;

    // Registration
    public void              RegisterRule(IRule rule);       // throws on duplicate key
    public bool              UnregisterRule(string ruleKey);
    public bool              HasRule(string ruleKey);
    public IEnumerable<string> GetRegisteredKeys();

    // Execution
    public (Dictionary<string, object> outputs, object result)
        SolveRule(string ruleKey, Dictionary<string, object> parameters);
    public (Dictionary<string, object> outputs, object result)
        SolveRule(string ruleKey, Dictionary<string, object> parameters, RuleExecutionPolicy policy);

    // Direct expression evaluation
    public object EvaluateExpression(IList<Token> tokens, Dictionary<string, object> parameters);
    public object EvaluateExpression(IList<Token> tokens, Dictionary<string, object> parameters, RuleExecutionPolicy policy);
}

RuleAuditEventArgs

RuleExecutionPolicy

var policy = new RuleExecutionPolicy
{
    MaxDepth                 = 10,      // max recursive rule-reference depth
    AllowDeprecatedExecution = false,   // reject Deprecated rules
    MinimumLifecycleState    = RuleLifecycleState.Approved,
    AllowedTokenTypes        = null     // null = allow all types
};
var (_, result) = engine.SolveRule("MyRule", parameters, policy);

Exceptions

Expression Evaluation

Rule expressions use a shunting-yard → RPN pipeline: infix token stream → Reverse Polish Notation → stack-based evaluator with automatic type coercion.

Operator precedence (high → low)

Token types and syntax

Type coercion rules

• Any arithmetic/comparison with null propagates null, except =/!= which compare nullity.
• + with a string operand performs string concatenation.
• Comparison operators attempt numeric comparison first; fall back to StringComparer.OrdinalIgnoreCase.
• AND/OR coerce both sides to bool via Convert.ToBoolean.

Rule references (recursive)

engine.RegisterRule(new SimpleRule("DQ.Base", ":score > 0"));
engine.RegisterRule(new SimpleRule("DQ.Full", "@DQ.Base AND :score < 100"));

var (_, ok) = engine.SolveRule("DQ.Full",
    new Dictionary<string, object> { [":score"] = 55.0 });  // true

Tokenizer

Tokenizer is a stateless, deterministic lexer — never throws for malformed input, always returns structured diagnostics.

var result = new Tokenizer(":price * 1.1 > 50").Tokenize();
// result.Tokens     → [ EntityField(":price"), Multiply, NumericLiteral("1.1"), GreaterThan, NumericLiteral("50") ]
// result.Diagnostics → []

ParseDiagnostic fields

Parsers

RuleParserFactory is a case-insensitive registry of IRuleParser implementations. The default parser handles the standard Beep expression language; five additional built-in parsers cover common domain syntaxes.

public sealed class RuleParserFactory : IRuleParserFactory
{
    void                   RegisterParser(string ruleType, IRuleParser parser);  // throws on duplicate
    IRuleParser            GetParser(string ruleType);     // throws if not found
    bool                   HasParser(string ruleType);
    IEnumerable<string>   GetRegisteredTypes();
}

Built-in parsers

Parser examples

// SQL WHERE
var sqlResult = new SqlWhereParser().ParseRule("Status IN ('Active','Pending') AND Score >= 50");

// NFEL
var nfelResult = new NfelParser().ParseRule("order.total > 100 && !isCancelled");

// CSV Column
var csvResult = new CsvColumnParser().ParseRule("row.Amount[3]");
// Token[0]: Identifier "Amount", Token[1]: NumericLiteral "3"

// JSON tree
string json = @"{""rule"":""DQ.IsInRange"",""params"":{""Min"":""0"",""Max"":""100""},""children"":[]}";
var jsonResult = new JsonRuleTreeParser().ParseRule(json);

Registering a custom parser

[RuleParser(parserKey: "MyDslParser")]
public sealed class MyDslParser : IRuleParser
{
    List<IRuleStructure> IRuleParser.RuleStructures { get; } = new();
    public ParseResult ParseRule(string expression) { /* tokenise, return ParseResult */ }
    public ParseResult ParseRule(IRule rule) => ParseRule(rule.RuleText);
    public void Clear() { }
}

// Manual registration or auto-discovered via RuleRegistry.Discover():
factory.RegisterParser("MyDslParser", new MyDslParser());

RuleCatalog

In-memory store for IRuleStructure instances with lifecycle-state management. Use it to enumerate, filter, and promote rules independently of the execution engine.

public sealed class RuleCatalog : IRuleCatalog
{
    void                           Register(IRuleStructure structure);   // throws on duplicate GuidID or name
    void                           Unregister(string guidId);
    IRuleStructure                 GetByGuid(string guidId);
    IRuleStructure                 GetByName(string ruleName);
    IEnumerable<IRuleStructure>   GetAll();
    IEnumerable<IRuleStructure>   GetByModule(string module);
    IEnumerable<IRuleStructure>   GetByTag(string tag);           // comma-separated Tags field
    IEnumerable<IRuleStructure>   GetByLifecycleState(RuleLifecycleState state);
    void                           Promote(string guidId, RuleLifecycleState newState);  // forward-only
}

RuleLifecycleState

catalog.Promote(guid, RuleLifecycleState.Approved);  // forward only — downgrade throws

RuleRegistry — Auto-wiring

public sealed class RuleRegistry
{
    public IRuleEngine        Engine        { get; }   // populated by Discover()
    public IRuleParserFactory ParserFactory { get; }

    public RuleRegistry(IDMEEditor editor);
    public RuleRegistry(IDMEEditor editor, IRuleEngine engine, IRuleParserFactory parserFactory);

    // Scans ConfigEditor.RuleParserClasses + ConfigEditor.Rules
    // Safe to call multiple times — already-registered keys are skipped
    public void Discover();
}

var registry = new RuleRegistry(editor);
registry.Discover();

var engine  = registry.Engine;
var factory = registry.ParserFactory;

Rule State Store

Stateful rules (CircuitBreaker, RetryWithBackoff, DeadLetterQueue, ThrottleRule) use IRuleStateStore to persist counters between calls. Pass one via the "IRuleStateStore" parameter key.

public interface IRuleStateStore
{
    object?  Get(string stateKey);
    void     Set(string stateKey, object value);
    int      Increment(string stateKey, int delta = 1);
    void     Remove(string stateKey);
    bool     Contains(string stateKey);
    void     Clear();
}

Built-in Rules Reference

Group: DQ — Data Quality

var (_, inRange) = engine.SolveRule("DQ.IsInRange",
    new Dictionary<string, object> { ["Value"] = 75.0, ["Min"] = 0.0, ["Max"] = 100.0 });
// inRange → true

Group: Validate

Group: Record

// {Token} interpolation in SetFieldValue
var (outputs, _) = engine.SolveRule("Record.SetFieldValue",
    new Dictionary<string, object>
    {
        ["FieldName"] = "FullName",
        ["Value"]     = "{FirstName} {LastName}",
        ["Record"]    = new Dictionary<string, object> { ["FirstName"] = "Alice", ["LastName"] = "Smith" }
    });
// outputs["Record"]["FullName"] → "Alice Smith"

Group: Compute

Group: Date

Group: Numeric

Group: Enrich

Group: Flow — Resilience

var store = new InMemoryRuleStateStore();
var (outputs, allowed) = engine.SolveRule("Flow.CircuitBreaker",
    new Dictionary<string, object>
    {
        ["StateKey"]         = "PaymentGateway",
        ["FailureThreshold"] = "3",
        ["ResetSeconds"]     = "60",
        ["CallSucceeded"]    = lastCallOk.ToString(),
        ["IRuleStateStore"]  = store
    });
// (bool)allowed  → true  = Closed/HalfOpen (proceed)
// outputs["CircuitState"] → "Closed" | "Open" | "HalfOpen"

Group: Route

Group: Notify

Group: Query

Group: Schema

Creating a Custom Rule

Implement IRule, annotate with [Rule], deploy the assembly — RuleRegistry.Discover() picks it up.

[Rule(ruleKey: "Pricing.ApplyDiscount",
      ParserKey = "RulesParser",
      RuleName  = "ApplyDiscount",
      RuleType  = "Transform")]
public sealed class ApplyDiscount : IRule
{
    public string         RuleText  { get; set; } = "Pricing.ApplyDiscount";
    public IRuleStructure Structure { get; set; } = new RuleStructure();

    public (Dictionary<string, object> outputs, object result)
        SolveRule(Dictionary<string, object> parameters = null)
    {
        var output = new Dictionary<string, object>();
        if (!parameters.TryGetValue("Price",    out var priceRaw) ||
            !parameters.TryGetValue("Discount", out var discRaw))
        { output["Error"] = "Missing parameters"; return (output, null); }

        double price    = Convert.ToDouble(priceRaw);
        double discount = Convert.ToDouble(discRaw);       // 0–100 %
        double final    = price * (1 - discount / 100.0);

        output["FinalPrice"]  = final;
        output["SavedAmount"] = price - final;
        return (output, final);
    }
}

// Usage:
var (outputs, finalPrice) = engine.SolveRule("Pricing.ApplyDiscount",
    new Dictionary<string, object> { ["Price"] = 120.00, ["Discount"] = 15.0 });
// finalPrice → 102.0 | outputs["SavedAmount"] → 18.0

Rule attribute fields

End-to-End Examples

1. Validate and enrich an order record

var registry = new RuleRegistry(editor);
registry.Discover();
var engine = registry.Engine;

var record = new Dictionary<string, object>
    { ["CustomerEmail"] = "bob@example.com", ["OrderTotal"] = 250.0, ["Status"] = "New" };

// Validate email
var (_, emailOk) = engine.SolveRule("Validate.ValidEmail",
    new Dictionary<string, object> { ["Value"] = record["CustomerEmail"] });
if (!(bool)emailOk) throw new Exception("Invalid email");

// Check total in range
var (_, rangeOk) = engine.SolveRule("DQ.IsInRange",
    new Dictionary<string, object> { ["Value"] = record["OrderTotal"], ["Min"] = 1.0, ["Max"] = 9999.0 });
if (!(bool)rangeOk) throw new Exception("Total out of range");

// Add metadata timestamps
var (enrichOut, _) = engine.SolveRule("Enrich.AddMetadataFields",
    new Dictionary<string, object> { ["Record"] = record });
var enriched = (IDictionary<string, object>)enrichOut["Record"];
// enriched now has CreatedAt, UpdatedAt, RecordId

2. Expression evaluation without a stored rule

var parseResult = new RuleParser().ParseRule(":discount > 10 AND :total >= 200");
if (parseResult.Success)
{
    object ok = engine.EvaluateExpression(
        parseResult.Structure.Tokens,
        new Dictionary<string, object> { [":discount"] = 15.0, [":total"] = 350.0 });
    // ok → true
}

3. DQ pipeline with audit log

engine.RuleEvaluated += (_, e) =>
    logger.LogInformation("[{Key}] ok={Ok} {Ms:F1}ms", e.RuleKey, e.Success, e.Elapsed.TotalMilliseconds);

string[] dqKeys = { "DQ.IsNullOrEmpty", "DQ.MatchesRegex", "Schema.EnforceNotNull" };
var bag = new Dictionary<string, object>
{
    ["Value"]   = record["CustomerEmail"],
    ["Pattern"] = @"^[\w.+-]+@[\w-]+\.[a-z]{2,}$",
    ["Record"]  = record,
    ["Fields"]  = "CustomerEmail,OrderTotal,Status"
};
bool allPassed = dqKeys.All(key => engine.SolveRule(key, bag).result is bool b && b);