Forward compat scheme

[hvr]

Generally, a given cabal-install release is only expected to understand (cabal-)spec(ification)-versions that were known at the time of release.

The Cabal-1.12 release, and more recently the Cabal-2.0 release have made it apparent that we don't have a good forward-compatibility story in place, thereby limiting our ability to evolve the .cabal format. Concretely, cabal-install-1.10 is not expected to understand package meta-data declared using the cabal-version:1.12 spec(ification)-version; also, upon encountering such a future .cabal file in the package index, cabal-install-1.10 ought to gracefully recover (instead of failing fatally to parse the package index). A similar situation occurred with the Cabal-2.0 release, which also introduced new syntax which previous parsers would fail to recognise, again resulting in fatal parsing failures.

To prevent this from happening again, the following forward compatibility has been devised, which is intended to make it possible to evolve the lexical and syntactical structure of the .cabal format in future, while not breaking legacy clients.

---

New-style Cabal-Specification Version Declaration

A new-style spec-version declaration at the beginning of a the .cabal file (without any preceding whitespace) and follow the following case-insensitive grammar (expressed in RFC5234 ABNF):

newstyle-spec-version-decl = "cabal-version" *WS ":" *WS newstyle-spec-version *WS

newstyle-spec-version      = NUM "." NUM [ "." NUM ]

NUM    = DIGIT0 / DIGITP 1*DIGIT0
DIGIT0 = %x30-39
DIGITP = %x31-39
WS = %20

The use of a new-style spec-version is

• invalid before spec-version 1.12,
• optional starting with spec-version 1.12,
• recommended starting with spec-version 2.0, and
• mandatory starting with spec-version 2.1.

It's also assumed that the following invariant holds:

• If a new-style spec-version declaration is present, its value must match the spec-version as determined by a full .cabal parser.

Scanning the Cabal-Specification Version

1. If a .cabal file contains a valid new-style spec-version declaration, it is authoritative;
2. otherwise, the spec-version must be below 2.1 (and either a full .cabal parser or a heuristic cabal-version-scanner (tbd) must be used to determine the exact spec-version).

The new-style spec-version declaration is designed to be simple to parse by means of common string operations. A simple implementation is shown below

scanSpecVersion :: ByteString -> Maybe Version
scanSpecVersion bs = do
    fstline':_ <- pure (BS.Char8.lines bs)

    -- parse <newstyle-spec-version-decl>
    -- normalise: remove all whitespace, convert to lower-case
    let fstline = BSW8.map toLowerW8 $ BSW8.filter (/= 0x20) $ BS.toStrict fstline'
    ["cabal-version",vers] <- pure (BSS.split ':' fstline)

    -- parse <spec-version> tolerantly
    ver <- simpleParse (BSS.unpack vers)
    guard $ case versionNumbers ver of
              [_,_]   -> True
              [_,_,_] -> True
              _       -> False

    pure ver
  where
    -- | Translate ['A'..'Z'] to ['a'..'z']
    toLowerW8 :: Word8 -> Word8
    toLowerW8 w | 0x40 < w && w < 0x5b = w+0x20
                | otherwise            = w

Appendix: Compatiblity with clients prior to cabal 2.0

Since this new scheme is only understood properly starting with cabal 2.0, older clients need to avoid being exposed to spec-versions 2.0 and newer.

To this end we exploit that cabal 2.0 started using the incremental secure 01-index.tar package index by default, while cabal versions prior to cabal 2.0 use the (non-incremental/non-secure) 00-index.tar package index by default (NB: cabal 1.24 was the first release that added experimental non-default/opt-in support for the secure 01-index.tar), by establishing the following hackage-side invariant:

• the legacy index 00-index.tar.gz contains only .cabal files with a spec version below 2

This way, the huge install-base of legacy cabal clients prior to cabal 2.0 keep working without requiring modifications, as they won't be exposed to incompatible .cabal files; with the unfortunate exception that the (hopefully uncommon) cabal clients prior to cabal 1.12 may still be exposed to incompatible cabal versions using the >=-less spec-version declarations.

[23Skidoo]

Related: #4448.

[gbaz]

This seems like a good way forward.

[phadej]

@hvr, should the ABNF contain end-of-line character? '\n' ?

[phadej]

merged in #5046

[typesanitizer]

Can the docs be updated to reflect this change? I just encountered this issue and am having a hard time understanding what I should do if I want to express >= 2.2 (I'm using common stanzas). Nevermind, the docs have been updated in #5131.

[Aster89]

I use GHCup to manage my Haskell environment. Based on that, I know my currently only installed and in use version of Cabal is 3.10.2.1 (also cabal --version says the same).

Why does cabal build fail if foo.cabal contains this?

cabal-version:   3.10
name:            foo
version:         1.0

executable foo
    main-is:          main.hs
    build-depends:    base

The error is

Errors encountered when parsing cabal file ./foo.cabal:

foo.cabal:0:0: error:
Unsupported cabal-version 3.10. See https://github.com/haskell/cabal/issues/4899.

which brought me here.

Changing the first line of foo.cabal to

cabal-version:   3.0

solves the problem, but... what's wrong with 3.10, if that is the version I have installed?

---

The content of main.hs is irrelevant, so it can as well just be

main :: IO ()
main = print 0

[tbidne]

@Aster89 The cabal file's cabal-version (file format spec) is not the same as the executable's version (cabal --version). See the docs for more details.

Perhaps the error message could be improved e.g. list the supported versions.

[Aster89]

... or cabal-version could have been cabal-file-version or cabal-format-version. But I guess that's too much of a breaking change today? :D

[gbaz]

I think rather than link to this ticket we should link to https://cabal.readthedocs.io/en/stable/file-format-changelog.html

and maybe the message could be unsupported specification of cabal format version in cabal-version field. current cabal-version values are listed at https://cabal.readthedocs.io/en/stable/file-format-changelog.html

[flip111]

@gbaz you're so right. Not being able use the cabal --version for cabal-version: is so confusing