Skip to content

Migrate from lukewarm to true cold runs #793

@rschu1ze

Description

@rschu1ze

This ticket is a tracking ticket for the migration from lukewarm to true cold runs. Details about this can be found here (original issue which triggered this) and in the benchmark rules about caching.

I suggest that we crowd-source this effort. Submissions are asked to

  • change their scripts (usually run.sh) to stop or kill the database before each first run of each query. This ensures all databases caches are cleared.
  • remove tag lukewarm-cold-run from their template.json file and from their result json files,
  • mark the submission as done in below list.

Affected submissions:

  • byconity
  • chdb
  • chdb-parquet
  • citus
  • clickhouse
  • clickhouse-datalake
  • clickhouse-datalake-partitioned
  • clickhouse-parquet
  • clickhouse-parquet-partitioned
  • clickhouse-tencent
  • clickhouse-web
  • cloudberry
  • cockroachdb
  • cratedb
  • daft-parquet
  • daft-parquet-partitioned
  • databend
  • datafusion
  • datafusion-partitioned
  • datafusion-vortex
  • datafusion-vortex-partitioned
  • doris
  • doris-parquet
  • drill
  • druid
  • duckdb
  • duckdb-dataframe
  • duckdb-datalake
  • duckdb-datalake-partitioned
  • duckdb-memory
  • duckdb-parquet
  • duckdb-parquet-partitioned
  • duckdb-vortex-partitioned
  • elasticsearch
  • glaredb
  • glaredb-partitioned
  • greenplum
  • heavyai
  • infobright
  • kinetica
  • mariadb
  • mariadb-columnstore
  • monetdb
  • mongodb
  • mysql
  • mysql-myisam
  • octosql
  • opteryx
  • oxla
  • pandas
  • paradedb
  • paradedb-partitioned
  • parseable
  • pg_clickhouse
  • pg_duckdb
  • pg_duckdb-indexed
  • pg_duckdb-parquet
  • pg_mooncake
  • pgpro_tam
  • pinot
  • polars
  • polars-dataframe
  • postgresql
  • postgresql-indexed
  • postgresql-orioledb
  • questdb
  • selectdb
  • siglens
  • singlestore
  • spark
  • spark-auron
  • spark-comet
  • spark-gluten
  • sqlite
  • starrocks
  • tidb
  • timescaledb
  • timescaledb-no-columnstore
  • ursa
  • victorialogs
  • yugabytedb

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions