Break Up the Pipeline Structure to Reach a Nearly Optimal End-to-End Latency

Query optimization is still problematic in the commercial database system because database optimizers sometimes choose a bad execution plan with several-fold latency differences. In this paper, we de-sign a new dynamic optimization strategy called query split, which takes advantage of run-time statistics. Integrating query split into PostgreSQL, we have a 2x speedup in total end-to-end latency on Join Order Benchmark and achieve near-optimal latency by comparing with the optimal execution plan. Our finding reveals that breaking up the static pipeline between database optimizer and executor can benefit the query processing and greatly reduce end-to-end latency.