Cloud warehouses & analytics engines
Cores that register with DatasourceCategory.CLOUD in this repo: managed SQL warehouses, federated query engines, and hosted analytics APIs. Some inherit RDBSource (Snowflake, Spanner, Presto); others implement IDataSource directly (BigQuery, Kusto).
Authentication vs target
Keep identity (OAuth client, service account JSON path, personal access token, API key) separate in your head from warehouse routing (project/dataset, account/region/warehouse/database, cluster URI). In Beep, both are usually captured on ConnectionProperties but map to different fields—mirror vendor wizards when in doubt, then validate with a read-only probe.
Never commit live secrets. Examples in docs use placeholders such as <ACCOUNT>, <TOKEN>.
Billing, latency, staging
- Large scans and long-running queries can incur cost; prefer filtered queries and limits when exploring metadata.
- Network RTT dominates; cache connection objects where the datasource supports it.
- For ETL, stage extracts in a cheaper store when the platform recommends it (vendor-specific).
Vendor authentication references
Use official guides for credential rotation and least-privilege roles (links open vendor / Microsoft documentation):
| Engine | Documentation |
|---|---|
| Google BigQuery | Authentication overview |
| Snowflake | Authentication |
| Google Cloud Spanner | Authentication |
| Azure Data Explorer (Kusto) | Application authentication |
| Presto / Trino | Trino security: authentication |
Documented providers
- Google BigQuery —
GoogleBigQuery/ - Snowflake —
SnowFlakeDataSource/ - Google Cloud Spanner —
SpannerDataSourceCore/ - Azure Data Explorer (Kusto) —
KustoDataSource/ - Presto —
PrestoDatasource/
More CLOUD cores in this repo
Additional projects under DataSourcesPluginsCore use DatasourceCategory.CLOUD (e.g. Rockset, Firebolt, Firebase, Amazon S3/Dynamo wrappers, Azure Cosmos/Document, Google Sheets, RealIM). Add Help/providers/cloud-*.html pages when you document them.
Contrast: CRM / marketing REST in Connectors/
This family targets warehouses and analytics (DatasourceCategory.CLOUD). Product APIs such as Salesforce, HubSpot, or Slack are separate projects under Connectors/<Category>/ with WebAPIDataSource and CommandAttribute-driven operations. Overview: REST / SaaS connectors; command catalogs: flagship conn-* index.
BeepDM skills
BeepDM/.cursor/connectionproperties/SKILL.mdBeepDM/.cursor/idatasource/SKILL.md
Next implementation family: Messaging & vector.