-
|
I’m interested in the case of using pgduckpipe to build an HTAP solution. For one of my data transformation use cases, I construct queries that are quite complex, containing a chain of CTEs, some of them SELECTs some INSERTs and some DELETEs. All of the tables involved would be synced to DuckLake. I’m wondering how the pgduckpipe query planner operates in such a case. Imagine CTE1 and CTE2, both being SELECT commands and CTE2 joins on CTE1. Is each CTE handled individually or is it possible for the query planner to combine multiple dependent CTEs (or subselects in general) and push them together to the DuckDB query executor? Thanks! |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
The SQL will be passed to DuckDB for execution in its entirety, and this behavior is inherited from pg_duckdb. pg_duckdb was planning to support "subquery dispatch", where parts of a SQL query are executed by Postgres and others by DuckDB. pg_duckdb would then merge the intermediate results and return the final result to the user. This is a more advanced feature that has not yet been implemented. |
Beta Was this translation helpful? Give feedback.
The SQL will be passed to DuckDB for execution in its entirety, and this behavior is inherited from pg_duckdb.
pg_duckdb was planning to support "subquery dispatch", where parts of a SQL query are executed by Postgres and others by DuckDB. pg_duckdb would then merge the intermediate results and return the final result to the user. This is a more advanced feature that has not yet been implemented.