1 clim_recal.resample

clim_recal.resample

Resample UKHADS data and UKCP18 data.

  • UKCP18 is resampled temporally from a 360 day calendar to a standard (365/366 day) calendar and projected to British National Grid (BNG) EPSG:27700 from its original rotated polar grid.
  • UKHADS is resampled spatially from 1km to 2.2km in BNG aligned with the projected UKCP18

1.1 Classes

Name Description
CPMResampler CPM specific changes to HADsResampler.
CPMResamplerManager Class to manage processing CPM resampling.
HADsResampler Manage resampling HADs datafiles for modelling.
HADsResamplerManager Class to manage processing HADs resampling.
ResamblerBase Base class to inherit for HADs and CPM.
ResamblerManagerBase Base class to inherit for HADs and CPM resampler managers.

1.1.1 CPMResampler

clim_recal.resample.CPMResampler(self, *, input_path=RAW_CPM_TASMAX_PATH, output_path=RESAMPLING_OUTPUT_PATH / CPM_OUTPUT_LOCAL_PATH, variable_name=VariableOptions.default(), input_files=None, cpus=None, crop_region=RegionOptions.GLASGOW, crop_path=RESAMPLING_OUTPUT_PATH / CPM_CROP_OUTPUT_LOCAL_PATH, final_crs=BRITISH_NATIONAL_GRID_EPSG, input_file_extension=NETCDF_EXTENSION_STR, export_file_extension=NETCDF_EXTENSION_STR, input_file_x_column_name=CPM_RAW_X_COLUMN_NAME, input_file_y_column_name=CPM_RAW_Y_COLUMN_NAME, start_index=0, stop_index=None, prior_time_series=None, _resample_func=cpm_reproject_with_standard_calendar)

CPM specific changes to HADsResampler.

1.1.1.1 Attributes

Name Type Description
input_path PathLike | None Path to CPM files to process.
output Path to save processed CPM files.
input_files Path or Paths of NCF files to reproject.
cpus Number of cpu cores to use during multiprocessing.
resampling_func Function to call on self.input_files.
crop Path or file to spatially crop input_files with.
final_crs Coordinate Reference System (CRS) to return final format in.
input_file_x_column_name str Column name in input_files or input for x coordinates.
input_file_y_column_name str Column name in input_files or input for y coordinates.
input_file_extension File extensions to glob input_files with.

1.1.1.2 Examples

>>> if not is_data_mounted:
...     pytest.skip(mount_doctest_skip_message)
>>> resample_test_cpm_output_path: Path = getfixture(
...         'resample_test_cpm_output_path')
>>> cpm_resampler: CPMResampler = CPMResampler(
...     input_path=RAW_CPM_TASMAX_PATH,
...     output_path=resample_test_cpm_output_path,
...     input_file_extension=TIF_EXTENSION_STR,
... )
>>> cpm_resampler
<CPMResampler(...count=100,...
    ...input_path='.../tasmax/05/latest',...
    ...output_path='.../test-run-results_..._.../cpm')>
>>> pprint(cpm_resampler.input_files)
(...Path('.../tasmax/05/latest/tasmax_...-cpm_uk_2.2km_05_day_19801201-19811130.tif'),
 ...Path('.../tasmax/05/latest/tasmax_...-cpm_uk_2.2km_05_day_19811201-19821130.tif'),
 ...
 ...Path('.../tasmax/05/latest/tasmax_...-cpm_uk_2.2km_05_day_20791201-20801130.tif'))

1.1.2 CPMResamplerManager

clim_recal.resample.CPMResamplerManager(self, *, input_paths=RAW_CPM_PATH, resample_paths=RESAMPLING_OUTPUT_PATH / CPM_OUTPUT_LOCAL_PATH, variables=(VariableOptions.default()), crop_regions=RegionOptions.all(), crop_paths=Path(), sub_path=CPM_SUB_PATH, start_index=0, stop_index=None, start_date=CPM_START_DATE, end_date=CPM_END_DATE, configs=list(), config_default_kwargs=dict(), resampler_class=CPMResampler, cpus=None, _input_path_dict=dict(), _resampled_path_dict=dict(), _cropped_path_dict=dict(), _strict_fail_if_var_in_input_path=True, _allow_check_fail=False, runs=RunOptions.preferred())

Class to manage processing CPM resampling.

1.1.2.1 Attributes

Name Type Description
input_paths PathLike | Sequence[PathLike] Path or Paths to CPM files to process. If Path, will be propegated with files matching
resample_paths PathLike | Sequence[PathLike] Path or Paths to to save processed CPM files to. If Path will be propagated to match input_paths.
variables Which VariableOptions to include.
runs Sequence[RunOptions | str] Which RunOptions to include.
crop_regions RegionOptions (like Manchester, Scotland etc.) to crop results to.
crop_paths Where to save region crop files.
sub_path Path Path to include at the stem of input_paths.
start_index Index to begin iterating input files for resampling or cropping.
stop_index Index to to run from start_index to when resampling or cropping. If None, iterate full list of paths.
start_date date Not yet implemented, but in future from what date to generate start index from.
end_date date Not yet implemented, but in future from what date to generate stop index from.
configs list[CPMResampler] List of HADsResampler instances to iterate resampling or cropping.
config_default_kwargs Parameters passed to all running self.configs.
resampler_class type[CPMResampler] class to construct all self.configs instances with.
cpus Number of cpu cores to use during multiprocessing.

1.1.2.2 Examples

>>> if not is_data_mounted:
...     pytest.skip(mount_doctest_skip_message)
>>> resample_test_cpm_output_path: Path = getfixture(
...         'resample_test_cpm_output_path')
>>> cpm_resampler_manager: CPMResamplerManager = CPMResamplerManager(
...     stop_index=9,
...     resample_paths=resample_test_cpm_output_path,
...     crop_paths=resample_test_cpm_output_path,
...     )
>>> cpm_resampler_manager
<CPMResamplerManager(variables_count=1, runs_count=4,
                     input_paths_count=4)>
>>> configs: tuple[CPMResampler, ...] = tuple(
...     cpm_resampler_manager.yield_configs())
>>> pprint(configs)
(<CPMResampler(count=9, max_count=100,
               input_path='.../tasmax/05/latest',
               output_path='.../cpm/tasmax/05')>,
 <CPMResampler(count=9, max_count=100,
               input_path='.../tasmax/06/latest',
               output_path='.../cpm/tasmax/06')>,
 <CPMResampler(count=9, max_count=100,
               input_path='.../tasmax/07/latest',
               output_path='.../cpm/tasmax/07')>,
 <CPMResampler(count=9, max_count=100,
               input_path='.../tasmax/08/latest',
               output_path='.../cpm/tasmax/08')>)

1.1.3 HADsResampler

clim_recal.resample.HADsResampler(self, *, input_path=RAW_HADS_TASMAX_PATH, output_path=RESAMPLING_OUTPUT_PATH / HADS_OUTPUT_LOCAL_PATH, variable_name=VariableOptions.default(), input_files=None, cpus=None, crop_region=RegionOptions.GLASGOW, crop_path=RESAMPLING_OUTPUT_PATH / HADS_CROP_OUTPUT_LOCAL_PATH, final_crs=BRITISH_NATIONAL_GRID_EPSG, input_file_extension=NETCDF_EXTENSION_STR, export_file_extension=NETCDF_EXTENSION_STR, input_file_x_column_name=HADS_XDIM, input_file_y_column_name=HADS_YDIM, start_index=0, stop_index=None, cpm_for_coord_alignment=RAW_CPM_TASMAX_PATH, cpm_for_coord_alignment_path_converted=False, _resample_func=hads_resample_and_reproject)

Manage resampling HADs datafiles for modelling.

1.1.3.1 Attributes

Name Type Description
input_path PathLike | None Path to HADs files to process.
output Path to save processed HADS files.
input_files Iterable[PathLike] | None Path or Paths of NCF files to resample.
resampling_func Function to call on self.input_files.
crop Path or file to spatially crop input_files with.
final_crs str Coordinate Reference System (CRS) to return final format in.
input_file_x_column_name str Column name in input_files or input for x coordinates.
input_file_y_column_name str Column name in input_files or input for y coordinates.
input_file_extension NETCDF_OR_TIF File extensions to glob input_files with.
start_index First index of file to iterate processing from.
stop_index Last index of files to iterate processing from as a count from start_index. If None, this will simply iterate over all available files.
cpm_for_coord_alignment T_Dataset | PathLike | None CPM Path or Dataset to match alignment with.
cpm_for_coord_alignment_path_converted bool Whether a Path passed to cpm_for_coord_alignment should be processed.

1.1.3.2 Notes

1.1.3.3 Examples

>>> if not is_data_mounted:
...     pytest.skip(mount_doctest_skip_message)
>>> resample_test_hads_output_path: Path = getfixture(
...         'resample_test_hads_output_path')
>>> hads_resampler: HADsResampler = HADsResampler(
...     output_path=resample_test_hads_output_path,
... )
>>> hads_resampler
<HADsResampler(...count=504,...
    ...input_path='.../tasmax/day',...
    ...output_path='...run-results_..._.../hads')>
>>> pprint(hads_resampler.input_files)
(...Path('.../tasmax/day/tasmax_hadukgrid_uk_1km_day_19800101-19800131.nc'),
 ...Path('.../tasmax/day/tasmax_hadukgrid_uk_1km_day_19800201-19800229.nc'),
 ...,
 ...Path('.../tasmax/day/tasmax_hadukgrid_uk_1km_day_20211201-20211231.nc'))

1.1.3.4 Methods

Name Description
set_cpm_for_coord_alignment Check if cpm_for_coord_alignment is a Dataset, process if a Path.
1.1.3.4.1 set_cpm_for_coord_alignment

clim_recal.resample.HADsResampler.set_cpm_for_coord_alignment()

Check if cpm_for_coord_alignment is a Dataset, process if a Path.

1.1.4 HADsResamplerManager

clim_recal.resample.HADsResamplerManager(self, *, input_paths=RAW_HADS_PATH, resample_paths=RESAMPLING_OUTPUT_PATH / HADS_OUTPUT_LOCAL_PATH, variables=(VariableOptions.default()), crop_regions=RegionOptions.all(), crop_paths=RESAMPLING_OUTPUT_PATH / HADS_CROP_OUTPUT_LOCAL_PATH, sub_path=HADS_SUB_PATH, start_index=0, stop_index=None, start_date=HADS_START_DATE, end_date=HADS_END_DATE, configs=list(), config_default_kwargs=dict(), resampler_class=HADsResampler, cpus=None, _input_path_dict=dict(), _resampled_path_dict=dict(), _cropped_path_dict=dict(), _strict_fail_if_var_in_input_path=True, _allow_check_fail=False, cpm_for_coord_alignment=RAW_CPM_TASMAX_PATH, cpm_for_coord_alignment_path_converted=False)

Class to manage processing HADs resampling.

1.1.4.1 Attributes

Name Type Description
input_paths PathLike | Sequence[PathLike] Path or Paths to CPM files to process. If Path, will be propegated with files matching
resample_paths PathLike | Sequence[PathLike] Path or Paths to to save processed CPM files to. If Path will be propagated to match input_paths.
variables Which VariableOptions to include.
crop_regions RegionOptions (like Manchester, Scotland etc.) to crop results to.
crop_paths Sequence[PathLike] | PathLike Where to save region crop files.
sub_path Path Path to include at the stem of input_paths.
start_index Index to begin iterating input files for resampling or cropping.
stop_index Index to to run from start_index to when resampling or cropping. If None, iterate full list of paths.
start_date date Not yet implemented, but in future from what date to generate start index from.
end_date date Not yet implemented, but in future from what date to generate stop index from.
configs list[HADsResampler] List of HADsResampler instances to iterate resampling or cropping.
config_default_kwargs dict[str, Any] Parameters passed to all running self.configs.
resampler_class type[HADsResampler] class to construct all self.configs instances with.
cpus Number of cpu cores to use during multiprocessing.
cpm_for_coord_alignment T_Dataset | PathLike CPM Path or Dataset to match alignment with.
cpm_for_coord_alignment_path_converted bool Whether a Path passed to cpm_for_coord_alignment should be processed.

1.1.4.2 Examples

>>> if not is_data_mounted:
...     pytest.skip(mount_doctest_skip_message)
>>> resample_test_hads_output_path: Path = getfixture(
...         'resample_test_hads_output_path')
>>> hads_resampler_manager: HADsResamplerManager = HADsResamplerManager(
...     variables=VariableOptions.all(),
...     resample_paths=resample_test_hads_output_path,
...     )
>>> hads_resampler_manager
<HADsResamplerManager(variables_count=3, input_paths_count=3)>

1.1.4.3 Methods

Name Description
set_cpm_for_coord_alignment Check if cpm_for_coord_alignment is a Dataset, process if a Path.
yield_configs Generate a CPMResampler or HADsResampler for self.input_paths.
1.1.4.3.1 set_cpm_for_coord_alignment

clim_recal.resample.HADsResamplerManager.set_cpm_for_coord_alignment()

Check if cpm_for_coord_alignment is a Dataset, process if a Path.

1.1.4.3.2 yield_configs

clim_recal.resample.HADsResamplerManager.yield_configs()

Generate a CPMResampler or HADsResampler for self.input_paths.

1.1.5 ResamblerBase

clim_recal.resample.ResamblerBase(self, *, input_path=Path(), output_path=RESAMPLING_OUTPUT_PATH, variable_name=VariableOptions.default(), input_files=None, cpus=None, crop_region=RegionOptions.GLASGOW, crop_path=RESAMPLING_OUTPUT_PATH, final_crs=BRITISH_NATIONAL_GRID_EPSG, input_file_extension=NETCDF_EXTENSION_STR, export_file_extension=NETCDF_EXTENSION_STR, input_file_x_column_name='', input_file_y_column_name='', start_index=0, stop_index=None)

Base class to inherit for HADs and CPM.

1.1.5.1 Attributes

Name Description
max_count Maximum length of self.input_files ignoring start_index and start_index.

1.1.5.2 Methods

Name Description
crop_projection Crop a projection to region geometry.
execute Run all steps for processing
execute_crops Run all specified crops.
set_input_files Replace self.input and process self.input_files.
1.1.5.2.1 crop_projection

clim_recal.resample.ResamblerBase.crop_projection(index=0, override_export_path=None, return_results=False, sync_reprojection_paths=True, **kwargs)

Crop a projection to region geometry.

1.1.5.2.2 execute

clim_recal.resample.ResamblerBase.execute(skip_spatial=False, **kwargs)

Run all steps for processing

1.1.5.2.3 execute_crops

clim_recal.resample.ResamblerBase.execute_crops(skip_crop=False, **kwargs)

Run all specified crops.

1.1.5.2.4 set_input_files

clim_recal.resample.ResamblerBase.set_input_files(new_input_path=None)

Replace self.input and process self.input_files.

1.1.6 ResamblerManagerBase

clim_recal.resample.ResamblerManagerBase(self, *, input_paths=Path(), resample_paths=Path(), variables=(VariableOptions.default()), crop_regions=RegionOptions.all(), crop_paths=Path(), sub_path=Path(), start_index=0, stop_index=None, start_date=None, end_date=None, configs=list(), config_default_kwargs=dict(), resampler_class=None, cpus=None, _input_path_dict=dict(), _resampled_path_dict=dict(), _cropped_path_dict=dict(), _strict_fail_if_var_in_input_path=True, _allow_check_fail=False)

Base class to inherit for HADs and CPM resampler managers.

1.1.6.1 Attributes

Name Description
crop_folder Return self._output_path set by set_resample_paths().
input_folder Return self._input_path set by set_input_paths().
max_count Maximum length of self.input_files ignoring start_index and start_index.
resample_folder Return self._output_path set by set_resample_paths().

1.1.6.2 Classes

Name Description
VarirableInBaseImportPathError Checking import path validity for self.variables.
1.1.6.2.1 VarirableInBaseImportPathError

clim_recal.resample.ResamblerManagerBase.VarirableInBaseImportPathError()

Checking import path validity for self.variables.

1.1.6.3 Methods

Name Description
check_paths Check and set input, resample and crop paths.
execute_crop_configs Run all resampler configurations
execute_resample_configs Run all resampler configurations
set_crop_paths Propagate self.resample_paths if needed.
set_resample_paths Propagate self.resample_paths if needed.
yield_configs Generate a CPMResampler or HADsResampler for self.input_paths.
yield_crop_configs Generate a CPMResampler or HADsResampler for self.input_paths.
1.1.6.3.1 check_paths

clim_recal.resample.ResamblerManagerBase.check_paths(run_set_data_paths=True, run_set_crop_paths=True)

Check and set input, resample and crop paths.

1.1.6.3.2 execute_crop_configs

clim_recal.resample.ResamblerManagerBase.execute_crop_configs(multiprocess=False, cpus=None)

Run all resampler configurations

1.1.6.3.2.1 Parameters
Name Type Description Default
multiprocess bool If True run parameters in resample_configs with multiprocess_execute. False
cpus int | None Number of cpus to pass to multiprocess_execute. None
1.1.6.3.3 execute_resample_configs

clim_recal.resample.ResamblerManagerBase.execute_resample_configs(multiprocess=False, cpus=None)

Run all resampler configurations

1.1.6.3.3.1 Parameters
Name Type Description Default
multiprocess bool If True run parameters in resample_configs with multiprocess_execute. False
cpus int | None Number of cpus to pass to multiprocess_execute. None
1.1.6.3.4 set_crop_paths

clim_recal.resample.ResamblerManagerBase.set_crop_paths()

Propagate self.resample_paths if needed.

1.1.6.3.5 set_resample_paths

clim_recal.resample.ResamblerManagerBase.set_resample_paths()

Propagate self.resample_paths if needed.

1.1.6.3.6 yield_configs

clim_recal.resample.ResamblerManagerBase.yield_configs()

Generate a CPMResampler or HADsResampler for self.input_paths.

1.1.6.3.7 yield_crop_configs

clim_recal.resample.ResamblerManagerBase.yield_crop_configs()

Generate a CPMResampler or HADsResampler for self.input_paths.

1.2 Functions

Name Description
reproject_2_2km_filename Return tweaked path to indicate standard day projection.
reproject_standard_calendar_filename Return tweaked path to indicate standard day projection.

1.2.1 reproject_2_2km_filename

clim_recal.resample.reproject_2_2km_filename(path)

Return tweaked path to indicate standard day projection.

1.2.2 reproject_standard_calendar_filename

clim_recal.resample.reproject_standard_calendar_filename(path)

Return tweaked path to indicate standard day projection.

Back to top