Skip to content

Introduce $CLUSTERSHELL_CFGDIR #483

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 8, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion doc/examples/defaults.conf-rsh
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
# Example defaults.conf file for clusters using rsh instead of ssh.
#
# To enable this file, install it in one of the following locations:
# /etc/clustershell/defaults.conf (system-wide)
# $CLUSTERSHELL_CFGDIR/defaults.conf (global configuration, default to
# /etc/clustershell/defaults.conf)
# $XDG_CONFIG_HOME/clustershell/defaults.conf (per-user)
# $HOME/.local/etc/clustershell/defaults.conf (per-user)
#
Expand Down
31 changes: 20 additions & 11 deletions doc/sphinx/config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,16 @@ Configuration
clush
-----

The following configuration file defines system-wide default values for
The following configuration file defines global default values for
several *clush* tool parameters::

/etc/clustershell/clush.conf
$CLUSTERSHELL_CFGDIR/clush.conf

If *$CLUSTERSHELL_CFGDIR* is not defined, */etc/clustershell/clush.conf* will
be used,

*clush* settings might then be overridden per user if one of the following
files is found, in priority order::
*clush* settings might then be overridden (globally, or per user) if one of the
following files is found, in priority order::

$XDG_CONFIG_HOME/clustershell/clush.conf
$HOME/.config/clustershell/clush.conf (only if $XDG_CONFIG_HOME is not defined)
Expand Down Expand Up @@ -123,13 +126,16 @@ ClusterShell loads *groups.conf* configuration files that define how to
obtain node groups configuration, ie. the way the library should access
file-based or external node group **sources**.

The following configuration file defines system-wide default values for
The following configuration file defines global default values for
*groups.conf*::

/etc/clustershell/groups.conf
$CLUSTERSHELL_CFGDIR/groups.conf

If *$CLUSTERSHELL_CFGDIR* is not defined, */etc/clustershell/groups.conf* will
be used,

*groups.conf* settings might then be overridden per user if one of the
following files is found, in priority order::
*groups.conf* settings might then be overridden (globally, or per user) if one
of the following files is found, in priority order::

$XDG_CONFIG_HOME/clustershell/groups.conf
$HOME/.config/clustershell/groups.conf (only if $XDG_CONFIG_HOME is not defined)
Expand Down Expand Up @@ -542,10 +548,13 @@ in *defaults.conf*.

The following configuration file defines ClusterShell system-wide defaults::

/etc/clustershell/defaults.conf
$CLUSTERSHELL_CFGDIR/defaults.conf

*defaults.conf* settings might then be overridden per user if one of the
following files is found, in priority order::
If *$CLUSTERSHELL_CFGDIR* is not defined, */etc/clustershell/defaults.conf*
will be used,

*defaults.conf* settings might then be overridden (globally, or per user) if
one of the following files is found, in priority order::

$XDG_CONFIG_HOME/clustershell/defaults.conf
$HOME/.config/clustershell/defaults.conf (only if $XDG_CONFIG_HOME is not defined)
Expand Down
7 changes: 5 additions & 2 deletions doc/sphinx/release.rst
Original file line number Diff line number Diff line change
Expand Up @@ -564,8 +564,11 @@ versions.
Configuration files
"""""""""""""""""""

When ``$XDG_CONFIG_HOME`` is defined, ClusterShell will use it to search for
additional configuration files.
When ``$CLUSTERSHELL_CFGDIR`` or ``$XDG_CONFIG_HOME`` are defined,
ClusterShell will use them to search for additional configuration files.

If ``$CLUSTERSHELL_CFGDIR`` is not defined, the global configuration files will
be searched for in `/etc/clustershell`

PIP user installation support
"""""""""""""""""""""""""""""
Expand Down
8 changes: 5 additions & 3 deletions doc/txt/clush.conf.txt
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ following order:
1. command-line options
2. user configuration file (*$XDG_CONFIG_HOME/clustershell/clush.conf*)
3. local pip user installation (*$HOME/.local/etc/clustershell/clush.conf*)
4. system-wide configuration file (*/etc/clustershell/clush.conf*)
4. global configuration file (*$CLUSTERSHELL_CFGDIR/clush.conf*, defaults to
*/etc/clustershell/clush.conf*)

For each parameter, the first obtained value will be used.

Expand Down Expand Up @@ -122,8 +123,9 @@ Simple configuration file.
FILES
=====

*/etc/clustershell/clush.conf*
System-wide clush configuration file.
*$CLUSTERSHELL_CFGDIR/clush.conf*
Global clush configuration file. If $CLUSTERSHELL_CFGDIR is not defined,
*/etc/slutershell/clush.conf* is used instead.

*$XDG_CONFIG_HOME/clustershell/clush.conf*
User configuration file for clush. If $XDG_CONFIG_HOME is not defined,
Expand Down
5 changes: 3 additions & 2 deletions doc/txt/clush.txt
Original file line number Diff line number Diff line change
Expand Up @@ -278,8 +278,9 @@ Copy files
FILES
=====

*/etc/clustershell/clush.conf*
System-wide clush configuration file.
*$CLUSTERSHELL_CFGDIR/clush.conf*
Global clush configuration file. If $CLUSTERSHELL_CFGDIR is not defined,
*/etc/clustershell/clush.conf* is used instead.

*$XDG_CONFIG_HOME/clustershell/clush.conf*
User configuration file for clush. If $XDG_CONFIG_HOME is not defined,
Expand Down
12 changes: 7 additions & 5 deletions doc/txt/groups.conf.txt
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ following sources in the following order:

1. user configuration file (*$XDG_CONFIG_HOME/clustershell/groups.conf*)
2. local pip user installation (*$HOME/.local/etc/clustershell/groups.conf*)
3. system-wide configuration file (*/etc/clustershell/groups.conf*)
3. Global configuration file (*$CLUSTERSHELL_CFGDIR/groups.conf*, defaults to
*/etc/clustershell/groups.conf*)

If no *groups.conf* is found, group support will be disabled.

Expand Down Expand Up @@ -159,13 +160,14 @@ Simple configuration file for local groups and slurm partitions binding.
FILES
=====

*/etc/clustershell/groups.conf*
System-wide node groups configuration file.
*$CLUSTERSHELL_CFGDIR/groups.conf* (defaults to */etc/clustershell/groups.conf*)
Global node groups configuration file.

*/etc/clustershell/groups.conf.d/*
*$CLUSTERSHELL_CFGDIR/groups.conf.d/* (defaults to
*/etc/clustershell/groups.conf.d/*)
Recommended directory for additional configuration files.

*/etc/clustershell/groups.d/*
*$CLUSTERSHELL_CFGDIR/groups.d/* (defaults to */etc/clustershell/groups.d/*)
Recommended directory for *autodir*, where native group definition files
(.yaml files) are found.

Expand Down
4 changes: 3 additions & 1 deletion lib/ClusterShell/Defaults.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,9 @@ def _distant_workerclass(defaults):

def config_paths(config_name):
"""Return default path list for a ClusterShell config file name."""
return ['/etc/clustershell/%s' % config_name, # system-wide config file
return [os.path.join(os.environ.get('CLUSTERSHELL_CFGDIR',
'/etc/clustershell/%s'),
config_name), # global config file
# default pip --user config file
os.path.expanduser('~/.local/etc/clustershell/%s' % config_name),
# per-user config (top override)
Expand Down
53 changes: 53 additions & 0 deletions tests/CLIConfigTest.py
Original file line number Diff line number Diff line change
Expand Up @@ -319,6 +319,59 @@ def testClushConfigWithInstalledConfig(self):
options, _ = parser.parse_args([])
config = ClushConfig(options)

def testClushConfigCustomGlobal(self):
"""test CLI.Config.ClushConfig (CLUSTERSHELL_CFGDIR global custom
config)
"""

# Save existing environment variable, if it's defined
custom_config_save = os.environ.get('CLUSTERSHELL_CFGDIR')

# Create fake CLUSTERSHELL_CFGDIR
custom_cfg_dir = make_temp_dir()

try:
os.environ['CLUSTERSHELL_CFGDIR'] = custom_cfg_dir

cfgfile = open(os.path.join(custom_cfg_dir, 'clush.conf'), 'w')
cfgfile.write(dedent("""
[Main]
fanout: 42
connect_timeout: 14
command_timeout: 0
history_size: 100
color: never
verbosity: 2
ssh_user: joebar
ssh_path: ~/bin/ssh
ssh_options: -oSomeDummyUserOption=yes
"""))

cfgfile.flush()
parser = OptionParser("dummy")
parser.install_clush_config_options()
parser.install_display_options(verbose_options=True)
parser.install_connector_options()
options, _ = parser.parse_args([])
config = ClushConfig(options) # filename=None to use defaults!
self.assertEqual(config.color, THREE_CHOICES[1])
self.assertEqual(config.verbosity, VERB_VERB) # takes biggest
self.assertEqual(config.fanout, 42)
self.assertEqual(config.connect_timeout, 14)
self.assertEqual(config.command_timeout, 0)
self.assertEqual(config.ssh_user, 'joebar')
self.assertEqual(config.ssh_path, '~/bin/ssh')
self.assertEqual(config.ssh_options, '-oSomeDummyUserOption=yes')
cfgfile.close()

finally:
if custom_config_save:
os.environ['CLUSTERSHELL_CFGDIR'] = custom_config_save
else:
del os.environ['CLUSTERSHELL_CFGDIR']
shutil.rmtree(custom_cfg_dir, ignore_errors=True)


def testClushConfigUserOverride(self):
"""test CLI.Config.ClushConfig (XDG_CONFIG_HOME user config)"""

Expand Down