1 clim_recal.utils.server

clim_recal.utils.server

Utility functions.

1.1 Classes

Name	Description
CondaLockFileManager	Run conda_lock install to generate conda yml.

1.1.1 CondaLockFileManager

clim_recal.utils.server.CondaLockFileManager(self, conda_file_path=DEFAULT_CONDA_LOCK_PATH, env_paths=DEFAULT_ENV_PATHS, replace_file_path=False, legacy_arch=GITHUB_ACTIONS_ARCHITECTURE, legacy_name_prefix=CONDA_LEGACY_PREFIX, default_kwargs=lambda: DEFAULT_CONDA_LOCK_KWARGS())

Run conda_lock install to generate conda yml.

1.1.1.1 Attributes

Name	Type	Description
conda_file_path	PathLike	Path to write conda-lock file to.
env_paths	Sequence[PathLike]	Paths of configs to combine. For supported formats see: https://conda.github.io/conda-lock/
replace_file_path	bool	Whether to replace file_path if it already exists.
legacy_arch	str | None	What archeticture to use for legacy export.
legacy_name_prefix	PathLike | str	str to precede legacy_arch export file if run_legacy_mv() is run.
default_kwargs	dict[str, Any]	kwargs to pass to self.run_conda_lock().

1.1.1.2 Notes

This is derived from automating, with the -p osx-64 etc. components now specified in pyproject.toml and environment.yml, the following command:

conda-lock -f environment.yml -f python/pyproject.toml -p osx-64 -p linux-64 -p linux-aarch64

A full exmaple with options matching saved defaults:

conda-lock -f environment.yml -f python/pyproject.toml -p osx-64 -p linux-64 -p linux-aarch64 --check-input-hash

1.1.1.3 Examples

>>> conda_lock = CondaLockFileManager()
>>> conda_lock
<CondaLockFileManager(conda_file_path='../conda-lock.yml', env_paths=('../environment.yml', 'pyproject.toml'), legacy_arch='linux-64')>
>>> conda_lock.run()
['conda-lock lock --lockfile ../conda-lock.yml -f ../environment.yml -f pyproject.toml']
>>> conda_lock.run(as_str=True, use_default_kwargs=True)
'conda-lock lock --lockfile ../conda-lock.yml -f ../environment.yml -f pyproject.toml --check-input-hash'

1.1.1.4 Methods

Name	Description
conda_lock_cmd_str	Return configured conda-lock command.
legacy_export_cmd_str	Command to export legacy conda_lock file from self.conda_file_path.
run	Return self configurations, optionally execute as subprocess.
run_conda_lock	Check and optionally execute self.conda_lock_cmd_str().
run_legacy_export	Run self.legacy_export_cmd_str().
run_legacy_mv	Run self.legacy_export_cmd_str.

1.1.1.4.1 conda_lock_cmd_str

clim_recal.utils.server.CondaLockFileManager.conda_lock_cmd_str(use_default_kwargs=False, **kwargs)

Return configured conda-lock command.

1.1.1.4.2 legacy_export_cmd_str

clim_recal.utils.server.CondaLockFileManager.legacy_export_cmd_str(**kwargs)

Command to export legacy conda_lock file from self.conda_file_path.

1.1.1.4.3 run

clim_recal.utils.server.CondaLockFileManager.run(as_str=False, include_all=False, execute_all=False, conda_lock=True, execute_conda_lock=False, use_default_kwargs=False, legacy_export=False, execute_legacy_export=False, legacy_move=False, execute_legacy_move=False, cmds_list=None, execute_priors=False, cmds_post_list=None, execute_cmds_post=False, parent_dir_after_lock=False, **kwargs)

Return self configurations, optionally execute as subprocess.

1.1.1.4.3.1 Parameters

Name	Type	Description	Default
as_str	bool	Whether to return as a str, if not as a list[str].	False
include_all	bool	Include all commands, overriding individual parameters like conda_lock etc. Combine with execute_all to also run.	False
execute_all	bool	Run all included commands, overriding individual parameters like execute_conda_lock etc. Combine with include_all to run all commands.	False
conda_lock	bool	Whether to include self.run_conda_lock().	True
execute_conda_lock	bool	Whether to run the generated commands via subprocess.run().	False
use_default_kwargs	bool	Whether to use self.default_kwargs params to run self.run_conda_lock().	False
legacy_export	bool	Whether to add the self.legacy_export_cmd_str command.	False
execute_legacy_export	bool	Whether to run the self.legacy_export_cmd_str().	False
legacy_move	bool	Whether to add the self.legacy_mv_cmd_str() command.	False
execute_legacy_move	bool	Whether to run the self.legacy_mv_cmd_str().	False
cmds_list	list[str] | None	A list of commands to execute. If passed, these are executed prior.	None
execute_priors	bool	Execute commands passed in cmds_list prior to any others.	False
cmds_post_list	list[str] | None	A list of commands to run after all others.	None
execute_cmds_post	bool	Execute commands passed in cmds_post_list after all others.	False
parent_dir_after_lock	bool	Whether to return to parent dir after lock command.	False

1.1.1.4.3.2 Returns

Type	Description
list[str] | str	A list of commands generated, or a str of each command separated by a newline character (\n).

1.1.1.4.3.3 Examples

>>> conda_lock_file_manager.run()
['conda-lock lock --lockfile ../conda-lock.yml -f ../environment.yml -f pyproject.toml']
>>> print(conda_lock_file_manager.run(
...     as_str=True, legacy_export=True, legacy_move=True))
conda-lock lock --lockfile ../conda-lock.yml -f ../environment.yml -f pyproject.toml
conda-lock render --kind explicit --platform linux-64
mv conda-linux-64.lock .conda-linux-64.lock

1.1.1.4.4 run_conda_lock

clim_recal.utils.server.CondaLockFileManager.run_conda_lock(execute=False, use_default_kwargs=False, parent_dir_after_lock=False, **kwargs)

Check and optionally execute self.conda_lock_cmd_str().

1.1.1.4.4.1 Parameters

Name	Type	Description	Default
execute	bool	Whether to run self.conda_lock_cmd_str() as a subprocess.	False
use_default_kwargs	bool	Whether to include the self.default_kwargs in run.	False
kwargs		Any additional parameters to pass to self.conda_lock_cmd_str().	{}

1.1.1.4.4.2 Returns

Type	Description
str	Final generated command str, whether excuted or not.

1.1.1.4.4.3 Examples

>>> conda_lock_file_manager.run_conda_lock()
'conda-lock lock --lockfile ../conda-lock.yml -f ../environment.yml -f pyproject.toml'
>>> conda_lock_file_manager.run_conda_lock(use_default_kwargs=True)
'conda-lock lock --lockfile ../conda-lock.yml -f ../environment.yml -f pyproject.toml --check-input-hash'
>>> conda_lock_file_manager.run_conda_lock(pdb=True)
'conda-lock lock --lockfile ../conda-lock.yml -f ../environment.yml -f pyproject.toml --pdb'

1.1.1.4.5 run_legacy_export

clim_recal.utils.server.CondaLockFileManager.run_legacy_export(execute=False, **kwargs)

Run self.legacy_export_cmd_str().

1.1.1.4.5.1 Parameters

Name	Type	Description	Default
execute	bool	Whether to run self.conda_lock_cmd_str() as a subprocess.	False
kwargs		Any additional parameters to pass to self.legacy_export_cmd_str().	{}

1.1.1.4.5.2 Returns

Type	Description
str	Final generated command str, whether excuted or not.

1.1.1.4.5.3 Examples

>>> conda_lock_file_manager.run_legacy_export()
'conda-lock render --kind explicit --platform linux-64'
>>> conda_lock_file_manager.run_legacy_export(pdb=True)
'conda-lock render --kind explicit --platform linux-64 --pdb'

1.1.1.4.6 run_legacy_mv

clim_recal.utils.server.CondaLockFileManager.run_legacy_mv(execute=False)

Run self.legacy_export_cmd_str.

1.1.1.4.6.1 Parameters

Name	Type	Description	Default
execute	bool	Whether to run self.conda_lock_cmd_str() as a subprocess.	False

1.1.1.4.6.2 Returns

Type	Description
str	Final generated command str, whether excuted or not.

1.1.1.4.6.3 Examples

>>> conda_lock_file_manager.run_legacy_mv()
'mv conda-linux-64.lock .conda-linux-64.lock'
>>> conda_lock_file_manager.legacy_name_prefix = '../.'
>>> conda_lock_file_manager.run_legacy_mv()
'mv conda-linux-64.lock ../.conda-linux-64.lock'

1.2 Functions

Name	Description
kwargs_to_cli_str	Convert kwargs into a cli str.
make_user	Make user account and copy code to that environment.
make_users	Load a file of usernames and passwords and pass each line to make_user.
rm_user	Remove user and user home folder.
set_and_pop_attr_kwargs	Extract any key: val pairs from kwargs to modify instance.

1.2.1 kwargs_to_cli_str

clim_recal.utils.server.kwargs_to_cli_str(space_prefix=True, **kwargs)

Convert kwargs into a cli str.

1.2.1.1 Parameters

Name	Type	Description	Default
kwargs		key=val parameters to concatenate as str.	{}

1.2.1.2 Returns

Type	Description
str	A final str of concatenated **kwargs in command line form.

1.2.1.3 Examples

>>> kwargs_to_cli_str(cat=4, in_a="hat", fun=False)
' --cat 4 --in-a hat --not-fun'
>>> kwargs_to_cli_str(space_prefix=False, cat=4, fun=True)
'--cat 4 --fun'
>>> kwargs_to_cli_str()
''

1.2.2 make_user

clim_recal.utils.server.make_user(user, password, code_path=RSTUDIO_DOCKER_USER_PATH, user_home_path=DEBIAN_HOME_PATH)

Make user account and copy code to that environment.

1.2.2.1 Parameters

Name	Type	Description	Default
user	str	Name for user and home folder name to append to user_home_path.	required
password	str	Login password.	required
code_path	PathLike	Path to copy code from to user home directory.	RSTUDIO_DOCKER_USER_PATH
user_home_path	PathLike	Path that user folder will be in, often Path('/home') in linux.	DEBIAN_HOME_PATH

1.2.2.2 Returns

Type	Description
Path	Full path to generated user home folder.

1.2.2.3 Examples

>>> import os
>>> if os.geteuid() != 0:
...     pytest.skip('requires root permission to run')
>>> user_name: str = 'an_unlinkely_test_user'
>>> password: str = 'test_pass'
>>> code_path: Path = JUPYTER_DOCKER_USER_PATH
>>> make_user(user_name, password, code_path=code_path)
PosixPath('/home/an_unlinkely_test_user')
>>> Path(f'/home/{user_name}/python/conftest.py').is_file()
True
>>> rm_user(user_name)
'an_unlinkely_test_user'

1.2.3 make_users

clim_recal.utils.server.make_users(file_path, user_col, password_col, file_reader, **kwargs)

Load a file of usernames and passwords and pass each line to make_user.

1.2.3.1 Parameters

Name	Type	Description	Default
file_path	PathLike	Path to collumned file including user names and passwords per row.	required
user_col	str	str of column name for user names.	required
password_col	str	str of column name for passwords.	required
file_reader	Callable	Callable (function) to read file_path.	required
**kwargs		Additional parameters for to pass to file_reader function.	{}

1.2.3.2 Examples

>>> if is_platform_darwin:
...     pytest.skip('test designed for docker jupyter')
>>> import os
>>> if os.geteuid() != 0:
...     pytest.skip('requires root permission to run')
>>> tmp_data_path = getfixture('data_fixtures_path')
>>> from pandas import read_excel
>>> def excel_row_iter(path: Path, **kwargs) -> dict:
...     df: DataFrame = read_excel(path, **kwargs)
...     return df.to_dict(orient="records")
>>> test_accounts_path: Path = tmp_data_path / 'test_user_accounts.xlsx'
>>> assert test_accounts_path.exists()
>>> user_paths: tuple[Path, ...] = tuple(make_users(
...     file_path=test_accounts_path,
...     user_col="User Name",
...     password_col="Password",
...     file_reader=excel_row_iter,
...     code_path=JUPYTER_DOCKER_USER_PATH,
... ))
>>> [(path / 'python' / 'conftest.py').is_file()
...  for path in user_paths]
[True, True, True, True, True]
>>> [rm_user(user_path.name) for user_path in user_paths]
['sally', 'george', 'jean', 'felicity', 'frank']

1.2.4 rm_user

clim_recal.utils.server.rm_user(user, user_home_path=DEBIAN_HOME_PATH)

Remove user and user home folder.

1.2.4.1 Parameters

Name	Type	Description	Default
user	str	User home folder name (usually the same as the user login name).	required
user_home_path	PathLike	Parent path of user folder name.	DEBIAN_HOME_PATH

1.2.4.2 Returns

Type	Description
str	user name of account and home folder deleted.

1.2.4.3 Examples

>>> import os
>>> if os.geteuid() != 0:
...     pytest.skip('requires root permission to run')
>>> if is_platform_darwin:
...     pytest.skip('test designed for docker jupyter')
>>> user_name: str = 'very_unlinkely_test_user'
>>> password: str = 'test_pass'
>>> make_user(user_name, password, code_path=JUPYTER_DOCKER_USER_PATH)
PosixPath('/home/very_unlinkely_test_user')
>>> rm_user(user_name)
'very_unlinkely_test_user'

1.2.5 set_and_pop_attr_kwargs

clim_recal.utils.server.set_and_pop_attr_kwargs(instance, **kwargs)

Extract any key: val pairs from kwargs to modify instance.

1.2.5.1 Parameters

Name	Type	Description	Default
instance	Any	An object to modify.	required
kwargs		key: val parameters to potentially modify instance attributes.	{}

1.2.5.2 Returns

Type	Description
dict[str, Any]	Any remaining kwargs not used to modify instance.

1.2.5.3 Examples

>>> kwrgs = set_and_pop_attr_kwargs(
...    conda_lock_file_manager, env_paths=['pyproject.toml'], cat=4)
>>> conda_lock_file_manager.env_paths
['pyproject.toml']
>>> kwrgs
{'cat': 4}