Skip to content

Suboptimal parseInstalledPackageInfo performance #2276

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
blackgnezdo opened this issue Dec 15, 2014 · 4 comments
Closed

Suboptimal parseInstalledPackageInfo performance #2276

blackgnezdo opened this issue Dec 15, 2014 · 4 comments

Comments

@blackgnezdo
Copy link
Contributor

My search for ghc-pkg memory consumption described in https://ghc.haskell.org/trac/ghc/ticket/9885 led me to parseInstalledPackageInfo. The following simple program will eat up 8G of memory:

import Distribution.InstalledPackageInfo

main = print $ length $ show $ parseInstalledPackageInfo x
where x = "ld-options: x" ++ concat (take 20000 $ repeat ","x" ")

The memory profile looks something like this:
(953)>>=...(...)
>>=..
>>=.
>>=
many1
many
sepBy1
sepBy
parseSepList
parseOptCommaList
listField
installedFieldDescrs
accumFields.setField
accumFields
parseFieldsFlat.
parseFieldsFlat
parseInstalledPackageInfo
parsePackageInfo
registerPackage
runit
main 794023536

@23Skidoo
Copy link
Member

Bad performance of the parser is a known issue. @dcoutts has written a new Parsec-based parser, but it hasn't been merged yet.

@blackgnezdo
Copy link
Contributor Author

@dcoutts, I wouldn't mind applying the patch locally if it's easy enough to backport to ghc 7.8.3.

@blackgnezdo
Copy link
Contributor Author

My workaround for this was to split the long comma separated list of ld-options into multiple ld-options with one item per line as the following program doesn't exhibit the pathology:

import Distribution.InstalledPackageInfo

main = print $ length $ show $ parseInstalledPackageInfo $ unlines $ take 20000 (repeat "ld-options: x")

@ttuegel ttuegel added this to the Cabal-1.24 milestone Apr 23, 2015
@23Skidoo 23Skidoo modified the milestones: Cabal 1.24, Cabal 1.26 Feb 21, 2016
@ezyang ezyang modified the milestone: Cabal 2.0 Sep 6, 2016
@phadej phadej self-assigned this Dec 16, 2017
@phadej phadej added this to the 2.2 milestone Dec 16, 2017
phadej added a commit to phadej/cabal that referenced this issue Jan 11, 2018
phadej added a commit to phadej/cabal that referenced this issue Jan 11, 2018
phadej added a commit to phadej/cabal that referenced this issue Jan 11, 2018
phadej added a commit to phadej/cabal that referenced this issue Jan 11, 2018
phadej added a commit to phadej/cabal that referenced this issue Jan 12, 2018
phadej added a commit to phadej/cabal that referenced this issue Jan 12, 2018
@phadej
Copy link
Collaborator

phadej commented Jan 17, 2018

Fixed in #5017

@phadej phadej closed this as completed Jan 17, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants