allow per-package configuration options in config file

[bos]

(Imported from Trac #223, reported by @dcoutts on 2008-01-29)

Currently only a subset of the configure command line flags can be stored in cabal-install's config file. There is no particularly good reason for this. It should be reasonably straightforward to do it generically and allow all configure options to be kept in the config file.

This may require some unification of the API for managing command line flags (the Simple.Command module) and the API for loading and saving from config flags (in ParseUtils?).

Once this is done, it should be straightforward to keep per-package config options as well as global options in the cabal-install config file.

These could be used to augment the options used when configuring a package. The order of overriding in cabal-install is currently:

savedConfig `mappend` configFlags

That is, the configure options from the config file are overridden by the ones supplied on the command line. With per-package saved config we would extend this to:

globalSavedConfig `mappend` packageSavedConfig pkg `mappend` configFlags

ie per-package saved, overrides global saved, but command line still overrides everything.

The slightly tricky bit is in how the api for config files and command line flags should be made to work better with each other, so that one set of information can be read/written from either source. The rest should be straightforward.

[bos]

(Imported comment by mnislaih on 2008-01-29)

I am working on this, wish me luck :)

[bos]

(Imported comment by mnislaih on 2008-03-02)

Since this is not a trivial refactoring and I am not a usual committer, here is a plan write-up.

Problem

We have `ConfigFlags?`, a data structure in typical database fashion, a dictionary of plain values composed of several fields. There are two main goals

a) Be able to serialize it (done now via `ParseUtils?`.`FieldDescrs?`)

b) Be able to manipulate it via command line flags (done via Command.Option)

For a), we want a format that is human readable and modifiable.

For b), we want to be able to specify the flags, with an expressivity comparable to the `GetOpt?` language.

Overall, we want to have a coherence and be able to reuse code. We may want to see these as two facets of the same problem and be able to specify them in one single place.

(and in Haskell 98 of course)

Solution 1

Specify b) via Distribution.Command.Option, and then convert those options to
ParseUtils?.FieldDescrs? automatically, which can be serialized for b).
The problem is that NoArg/OptArg? options cannot really be converted to FieldDescrs?.
We are missing a proper name for the field in the serialized file, and since
there are usually several Options of this kind associated to one field, we want to see them as defining the possible values of the field. But we cannot collect them,
as the option description is missing a declarative reference to which record they modify in the data structure.

option :: [Char] -> [String] -> String -> c -> d
       -> (c -> d -> ArgDescr a) -> Option a
option ss ls d get set arg = Option ss ls d (arg get set)

There are getters and setters, but those are opaque functions.

Hence, we can only automatically infer FieldDescrs? for ReqArg? options. The others must be inputed manually.

This seems error prone and tricky to maintain to me, although it has the advantage of being the simplest refactoring among all the options.

Solution 2

If we reify the record, then options can be given a more declarative type.

option :: Record data field -> [Char] -> String -> String -> ArgDescr field -> Option field
data Record data field =
  { get  :: data -> field
  , set  :: field -> data -> data

This is good enough to enable us to collect all the `NoArg?` options related to

one field, and condense a single FieldDescr? out of it.

Solution 3

But why stop there? We can put more metadata in the Record descriptor.

 interface :: OptInterface,
 desc      :: String,
 shape     :: Shape,
 ppr       :: field -> Doc
}
data OptInterface {
                  shortFlags :: [String],
              longFlags  :: [String]  }
data Shape field = StringValued (String -> ReadS field)
                 | Choice       (Map OptInterface field)

So now it is possible to extract an Option directly from this structure.

mkOption :: Record data field -> [Option data]

The definition of mkOption should be trivial. If the Record is `StringValued?`

then we get a single Option, otherwise we get an Option for every choice.

And the same goes for FieldDescrs?.

mkFieldDescr :: Record data field -> FieldDescr data

For the row name in the file we just take the first longFlag,

and if it has a Choice shape then we build the trivial parser for it.
Otherwise we just use the parser provided in StringValued?.

Finally, we could just extend FieldDescr? to accomodate the role of Record above.
But this poses the minor issue that FieldDescr? 'instances' are provided in CabalInstall, whereas they would be required by Cabal already in Distribution.Setup.hs
So they will need to be moved there.

[bos]

(Imported comment by @dcoutts on 2008-03-04)

Thanks for keeping us posted, it sounds like a very sensible plan.

[bos]

(Imported comment by guest on 2008-03-04)

It would be useful to be able to specify flags in the config file for all the commands and not only for configure e.g. install's --root-cmd.

This poses the question if it should be possible to specify different defaults for the same flag for two different commands.

[bos]

(Imported comment by @dcoutts on 2008-04-07)

The config file now holds the options for the global command (things like repos, cache dir etc), the install/upgrade command and the configure and upload commands.

Thu Aug 21 22:05:55 BST 2008  Duncan Coutts <duncan@haskell.org>
- Rearrange config settings and global command line settings
  The intention here is to have the config file content and parser
  be derivied from the command line types and flags. Now instead
  of having a saved config type that contains additional
  information to that kept in command line parameters we now put
  all the information in the command line settings type and make
  the saved config just the aggregation of the settings for
  various key commands like configure, install, upload. There's
  also an extended GlobalFlags type with the things like repos,
  cache dir etc. When we generate the initial config file we put
  in commented out versions of all (non-deprecated) valid fields,
  along with their default values. The aim is to make the config
  file self-documenting.
  

As before it also has two sets of saved install dirs.

Fri Aug 22 02:57:02 BST 2008  Duncan Coutts <duncan@haskell.org>
- Add back separate user and global sets of install dirs
  The user install setting selects between the two sets of dirs.
  At the moment it's just the "user-" and "global-" prefixed field
  names to support existing config files.
  Fri Aug 22 13:00:59 BST 2008  Duncan Coutts <duncan@haskell.org>
- Add sections for user/global install-dirs to the config file
  So it looks like:
  install-dirs user
    prefix: /home/username/.cabal
    ...
  Rather than using user-prefix, global-prefix, etc etc for each
  field. The old field names are still recognised but not added
  into the initial config file.
  

The default config file now looks like:

-- version:
-- numeric-version:
-- config-file:
remote-repo: hackage.haskell.org:http://hackage.haskell.org/packages/archive
remote-repo-cache: /home/username/.cabal/packages
-- local-repo:
-- documentation: False
-- dry-run:
-- reinstall:
-- root-cmd:
-- cabal-lib-version:
-- log-builds:
-- build-reports:
-- only:
-- verbose: 1
-- distpref: dist
-- compiler: ghc
-- with-compiler:
-- with-hc-pkg:
-- scratchdir:
-- program-prefix:
-- program-suffix:
-- library-vanilla: True
-- library-profiling: False
-- shared: False
-- executable-profiling: False
-- optimization: True
-- library-for-ghci: True
-- split-objs: False
-- executable-stripping: True
-- configure-option:
user-install: True
-- package-db:
-- flags:
-- extra-include-dirs:
-- extra-lib-dirs:
-- constraint:
-- verbose: 1
-- check:
-- username:
-- password:
install-dirs user
  prefix: /home/username/.cabal
  -- bindir: $prefix/bin
  -- libdir: $prefix/lib
  -- libsubdir: $pkgid/$compiler
  -- libexecdir: $prefix/libexec
  -- datadir: $prefix/share
  -- datasubdir: $pkgid
  -- docdir: $datadir/doc/$pkgid
  -- htmldir: $docdir/html
  -- haddockdir: $htmldir
install-dirs global
  -- prefix: /usr/local
  -- bindir: $prefix/bin
  -- libdir: $prefix/lib
  -- libsubdir: $pkgid/$compiler
  -- libexecdir: $prefix/libexec
  -- datadir: $prefix/share
  -- datasubdir: $pkgid
  -- docdir: $datadir/doc/$pkgid
  -- htmldir: $docdir/html
  -- haddockdir: $htmldir

There are some fields there that come from command line flags that do not really apply to the config file and so should be filtered out.

This syntax will allow us to to per-package config options, like:

package foo
  extra-lib-dirs: /some/thing

But actually adding that feature is not the top priority. Having a more self-documenting config file that contains all the options that can be specified on the command line is top priority.

[bos]

(Imported comment by @dcoutts on 2008-08-22)

The new config seems to be working. The fields that do not apply in the config file are now excluded.

The remaining issue can be punted to a later milestone.

[ezyang]

This is obsoleted by cabal.project in nix-local-build.