d5963d81d0
This is based on Jeff's most excellent work to identify why non-recursive listing under postgreskv was phenomenally slow. It turns out PostgreSQL's query planner was actually using two sequential scans of the pathdata table to do its job. It's unclear for how long that has been happening, but obviously it won't scale any further. The main change is propagating bucket association with pathnames through the CTE so that the query planner lets itself use the pathdata index on (bucket, fullpath) for the skipping-forward part. Jeff also had some changes to the range ends to keep NULL from being used- I believe with the intent of making sure the query planner was able to use the pathdata index. My tests on postgres 9.6 and 11 indicate that those changes don't make any appreciable difference in performance or query plan, so I'm going to leave them off for now to avoid a careful audit of the semantic differences. There is a test included here, which only serves to check that the new version of the function is indeed active. To actually ensure that no sequential scans are being used in the query plan anymore, our tests would need to be run against a test db with lots of data already loaded in it, and that isn't feasible for now. Change-Id: Iffe9a1f411c54a2f742a4abb8f2df0c64fd662cb |
||
---|---|---|
.. | ||
2018092201_initial-tables.down.sql | ||
2018092201_initial-tables.up.sql | ||
2019111301_fix_seq_scans_in_list.down.sql | ||
2019111301_fix_seq_scans_in_list.up.sql | ||
data.go | ||
migrate.go |