JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Jenkinsfile: collect full table scan queries from tests

Browse Source 
This change is first attempt to detect queries which are introducing
bad performance with full table scans. CRDB have special query to list
all queries which performs FTS.

With this change we are collecting all full table scan queries executed while unit tests. Results are combined from all started CRDB nodes into single results file. Such output can be later reviewed. This approach is not perfect but it's just one piece what we would like to build to detect FTS before pushing it into production.

part of https://github.com/storj/storj/issues/5471

Change-Id: I615fd9e624a136bb0870f9ff9c10e4eb5918339c
This commit is contained in:
Michal Niewrzal 2023-01-19 11:03:30 +01:00 committed by Storj Robot
parent 4cbb1ed296
commit 984da95543
1 changed files with 8 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
Jenkinsfile.verify
View File

@ -162,11 +162,19 @@ pipeline {
sh 'use-ports -from 1024 -to 10000 &'
sh 'go test -tags noembed -parallel 4 -p 6 -vet=off $COVERFLAGS -timeout 32m -json -race ./... 2>&1 | tee .build/tests.json | xunit -out .build/tests.xml'
sh 'cockroach sql --insecure --host=localhost:26256 -e \'WITH full_scans AS (SHOW FULL TABLE SCANS) SELECT query FROM full_scans GROUP BY query\' > .build/fts_node_256256.txt'
sh 'cockroach sql --insecure --host=localhost:26257 -e \'WITH full_scans AS (SHOW FULL TABLE SCANS) SELECT query FROM full_scans GROUP BY query\' > .build/fts_node_256257.txt'
sh 'cockroach sql --insecure --host=localhost:26258 -e \'WITH full_scans AS (SHOW FULL TABLE SCANS) SELECT query FROM full_scans GROUP BY query\' > .build/fts_node_256258.txt'
sh 'cockroach sql --insecure --host=localhost:26259 -e \'WITH full_scans AS (SHOW FULL TABLE SCANS) SELECT query FROM full_scans GROUP BY query\' > .build/fts_node_256259.txt'
sh 'cockroach sql --insecure --host=localhost:26260 -e \'WITH full_scans AS (SHOW FULL TABLE SCANS) SELECT query FROM full_scans GROUP BY query\' > .build/fts_node_256260.txt'
sh 'cat .build/fts_node_256256.txt .build/fts_node_256257.txt .build/fts_node_256258.txt .build/fts_node_256259.txt .build/fts_node_256260.txt| sort | uniq > .build/full_table_scan.txt'
}
post {
always {
archiveArtifacts artifacts: '.build/tests.json'
archiveArtifacts artifacts: '.build/full_table_scan.txt'
sh script: 'cat .build/tests.json | tparse -all -top -slow 100', returnStatus: true
junit '.build/tests.xml'