Code loading might be better just fully parsing TOML files #27414
Description
Right now, code loading uses some regexes that can parse the Project files in the format that Pkg writes them.
This has a few negative side effects:
- We do not actually support TOML file, only the subset that the specialized regexes manage to parse.
- There is no way to cache any results, instead, for everything getting loaded we start parsing from scratch.
- Giving good error messages seems hard. After we found that for some reason we cannot load a package we cannot look up things that we have already parsed, since we have thrown away all the parsed information.
- The code is kinda hard to read when parsing of TOML files is blended together with the code loading logic. There are currently ~6 functions that parse TOML files in different ways depending on what step in code loading we are doing.
I would argue that just parsing the full TOML files, cache the results and use that in code loading would lead to faster and more simple code. The drawback is that we need a full TOML parser ni Base, but since we have decided to use TOML for code loading that seems inevitable, should have called it some other file extension if we didn't intend to support TOML fully.