Integrating configuration file handling

[madduck]

Hello,

I recently wrote a CLI and used click, and I am loving it. I also wanted to provide the means for users to be able to create a configuration file in a default configuration, and even to provide a CLI option to override this default location. When data are read from the configuration file, they should actually override defaults, but not explicitly specified CLI options. I think this is what you'd normally expect, i.e. in the context of:

@click.option(
  "--foo",
  default=False,
  is_flag=True
)

1. if I call mycmd sub --foo, then foo should be true;
2. if I call mycmd sub, but a configuration file ~/.config/mycmd/config.toml exists with:

   [sub]
   foo = true

   then foo should be true
3. else, foo should default to false.

Initially, I found click-configfile and was really excited, but three things kept me from falling in love:

1. I needed to duplicate the schemata already specified using @click.option for the config file parsing to work;
2. The plugin/library uses configfile, i.e. "INI files", which are dreadful;
3. It didn't seem particularly maintained.

So I set out to do it differently, with decorators. Let the code speak:

@click.group
@cconfig_file_option("--cfgfile")
@click.option("--no-act", is_flag=True)
@merge_config
def mycmd(no_act):
  # do something

@mycmd.command
@merge_config
@click.option("--foo", is_flag=True)
def sub(foo):
  # do something

This would yield:

Usage: mycmd [OPTIONS] COMMAND [ARGS]...

Options:
  --cfgfile PATH  Config file to read
  --no-act        Do not actually act, just show what would be done
  --help          Show this message and exit.

Commands:
  sub

and you could create a file ~/.config/mycmd/config.toml with the following content:

no_act = true

[sub]
foo = false

and that would be considered before CLI defaults are.

How do you like it?

You can see the implementation at https://github.com/madduck/pngx/blob/main/pngx/config.py

Is this something worth that might make it into click mainline?

There are a few things that would need to be done on integration:

1. ParameterSource needs a new member, e.g. CONFIGFILE (see here);
2. An exception like BadConfig would be better than just a ClickException (see here).
3. Testing and test coverage, there are probably a hundred corner cases I have not foreseen…

Where to take things from here?

[Rowlando13]

In general, I am open to new community sourced ideas, and I appreciate the effort you put into this. However, Click is so widely used, that I would like to have more comprehensive docs and better test coverage before anything major is added, unless there is very clear community support. I will continue to keep an eye on this to see if it gathers community support.

[Rowlando13]

@kdeldycke Thoughts?

[kdeldycke]

Ahah thanks @Rowlando13 for invoking me! 😁

@madduck I stumbled upon the very same limitations of click-configfile a couple of years ago, and to solve this I built Click Extra.

Click Extra implements a ready-to-use --config parameter that is introspecting your already-defined Click CLI and map the configuration file to its options.

Bonus features:

• Supports TOML, YAML, JSON, INI and XML
• Automatic inference of the configuration file structure from your CLI's options
• Remote loading of configuration files from URLs
• Respect of CLI > Configuration > Environment > Defaults precedence

And lots of other stuff.

So go try Click Extra and send me feedback there! 🤗

[madduck]

@kdeldycke oh this is fantastic, I've migrated my project to your work. I wish I had found it earlier! Thank you!

[kdeldycke]

Since that discussion, @madduck switch to my own Click Extra's --config machinery and seems happy with it. Even contributed some good ideas! :)

As for the suggested new CONFIGFILE in ParameterSource, I think DEFAULT_MAP is perfect to play that role, and that's exactly what I am using in Click Extra's --config. See:

• https://github.com/kdeldycke/click-extra/blob/96719fdcd7916acb03ac2e13f0885aeb0c07d76c/click_extra/config.py#L1100-L1128
• Reorder ParameterSource #2879

So I guess this thread can be closed.