DuckDB, DuckLake and the data work around an AI agent
Understand the query engine, table format and operational responsibilities behind a managed business-data layer for agents.
DuckDB provides SQL execution; DuckLake organizes lake data with a SQL catalog and Parquet storage. Those components can support an agent data architecture, but an application still needs ingestion, credentials, permissions, freshness and query controls. Combined manages those surrounding responsibilities for its supported business-data workflows.
Separate the engine, table format and service
The names are related, but describe different responsibilities. DuckLake’s specification defines a catalog database and data storage. The catalog records metadata in SQL tables, while the stored table data uses Parquet. DuckDB can query DuckLake through its extension.
| Component | Responsibility | Question for your design |
|---|---|---|
| DuckDB | SQL execution | Which queries, resources and concurrent workloads must the application admit? |
| DuckLake | Catalog and table representation over stored data | Who operates the catalog, storage and maintenance? |
| Ingestion | Acquire and commit source changes | Who handles authorization, checkpoints, schema changes and retries? |
| Agent access service | Authenticate, discover permitted data and enforce query bounds | What can each identity read, and how is access revoked? |
Consult the DuckLake specification for format details and the DuckDB extension documentation for direct usage. A local experiment is useful for learning these components; it does not by itself establish a production service’s operating characteristics.
What remains after a SQL query works?
Connecting an agent to a query engine is one step. A business-data product must also keep the inputs reliable and ensure that each query is permitted. Write down the owner for each responsibility before choosing whether to operate the stack yourself.
- Source lifecycle: initial authorization, expired credentials, changed schemas and connector-specific deletion behavior.
- Commit correctness: distinguish staged data from committed data and recover interrupted ingestion without treating every retry as new business activity.
- Access: map the requesting identity to an account and explicit dataset grants.
- Query bounds: admit an allowed statement and enforce input, output and execution limits.
- Evidence: expose freshness and receipts so the agent can explain what it used.
- Retention: revoke access and carry out the applicable deletion policy when a source or account is removed.
A stored copy of source data is also a freshness choice. It can support queries across committed datasets, but it should not be described as an instantaneous view of every upstream change.
How Combined uses this architecture
Combined’s documented SQL surface queries granted DuckLake datasets through bounded, parameterized, read-only DuckDB SQL. Source ingestion produces committed data and provenance. External agents discover permitted logical relations through the catalog or MCP tools.
The agent receives logical relation names, not a general-purpose storage credential. Direct file paths, object-storage URLs and DuckLake metadata are outside the public SQL surface. Use the exact relation names returned by discovery; do not construct names from an assumed physical layout.
Combined’s documented query defaults include a 256 MiB estimated-input ceiling per query and a 100 GiB monthly estimated query input budget per account. A grant can impose a lower limit. These are material workload constraints to test during an evaluation, not a claim of unlimited warehouse-scale execution.
MCP, the API and typed SDKs provide access paths to the same governed data workflow. Combined supplies this layer for the agents you operate; it does not replace your agent runtime or application logic.
Decide what you want to own
Operating the components yourself can make sense when you need specialized ingestion, deployment control or a workload outside a managed product’s limits. Budget for connector upkeep, identity mapping, query admission, recovery and on-call ownership alongside storage and compute.
A managed approach is worth testing when your required sources are supported and your team wants to focus on the agent’s business task. In Combined, start with a Ready connector, confirm the first committed records, scope the agent grant, and run a representative query with a receipt.
Use the ingestion evaluation checklist to compare responsibility and evidence. For a small reproducible SQL exercise, try the synthetic invoice example. Neither a technology choice nor a successful toy query substitutes for checking your actual workload.
Sources and further reading
Explore the documentation behind this guide. Product details checked on September 9, 2026.