experiment_pipeline
ExperimentPipeline
A class for the experiment pipeline process.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
settings
|
Settings
|
Settings for the pipeline which includes the data folder locations, the maximum number of queries to send per minute, the maximum number of attempts when retrying, and whether to run the experiment in parallel |
required |
Source code in src/prompto/experiment_pipeline.py
14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 |
|
log_estimate
Function to log the estimated time of completion of the next experiment.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
experiment
|
Experiment
|
The experiment that is being processed |
required |
Source code in src/prompto/experiment_pipeline.py
log_progress
Function to log the progress of the queue of experiments.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
experiment
|
Experiment
|
The experiment that was just processed |
required |
Source code in src/prompto/experiment_pipeline.py
run
Run the pipeline process of continually by checking for new experiment files and running the experiments sequentially in the order that the files were created.
The process will continue to run until the program is stopped.
Source code in src/prompto/experiment_pipeline.py
update_experiment_files
Function to update the list of experiment files by sorting
the files by creation/change time (using os.path.getctime
).