Timeline

Timeline#

Authors of full-papers that are accepted to the Cambridge University Press (CUP) journal proceedings will be encouraged (but not required) to submit supporting materials for Artifact Evaluation following the Climate Informatics conference. Reviewing will take place over 5 weeks.

These are the key dates (all dates are in the Anywhere on Earth (AOE / UTC-12) timezone:

  • Expression of interest (Authors): Mon 28 June

  • Expression of interest (Reviewers): Sun 1 September

  • Artifact platform opening: Mon 2 September

  • Artifact submission (extended deadline): Thu 3 October

  • Start of review phase: Mon 7 October

  • Further clarifications until Thu 7 November

  • Final decisions sent to authors: Fri 8 November

Process#

The evaluation process uses an optional, lightweight single-blind system. Authors and Reviewers will be added to the HotCRP system of the CI2024 Artifact Evaluation, and the authors will submit their artifacts to the system.

Authors#

Authors may iteratively improve their artifacts during the process to overcome small technical problems, but may not submit new material that would require substantially more reviewing effort.

The detailed submission process is as follows:

  1. Register your interest to submit an artifact.

  2. Read the Submission Guidelines page for details on artifact preparation.

  3. Upload your artifact to Zenodo (recommended), or otherwise make it available via a stable URL (i.e. the URL should not change if you later make updates to the artifact; and ideally, the URL has a good chance of continuing to exist well into the future).

  4. Finalize your submission on HotCRP with a link to your artifact.

Reviewers#

A committee of reviewers, the Artifact Evaluation Committee (AEC), will review the submitted artifacts against three criteria: is the software available? Is it functional, and can it be used to reproduce the (central) claims or thesis of the paper? The reviewers will interact with the authors to suggest how to improve the reproducibility if any issues are discovered.

The detailed review process is as follows:

  1. Register your interest to review artifacts.

  2. Follow the instructions to access the assigned artifacts on HotCRP.

  3. Review the artifacts and provide feedback to the authors.

  4. Score the artifacts based on the criteria provided.

Contents