Company mode hangs Emacs when looking up for candidates.

[geraldus]

I'm facing hangs when company tries to complete something while I typing with big projects. This hangs varies in duration: sometimes hangs are very short, sometimes them last for several seconds. If I type something after company popup appeared I'm facing hang again. When I say hang I mean that if type something I see nothing to happen for some time, chars are being inserted only after completions shown. It makes company totally useless with haskell-mode.

During my investigation I've found that completions for company mode are provided by haskell-process-completions-at-point function [1] by default. To complete things it calls haskell-process-get-repl-completions [2], the latter calls haskell-process-queue-sync-request [3]. Word sync in the function name confuses me a lot. I guess this could be the crux of the matter. I'm not sure, but I suppose haskell-process-queue-command is async and could be used as replacement.

My current knowledge does not allow me to go any further, but I would be glad to provide a pull request if I'll have some guidance.

Any thoughts?

[1]

haskell-mode/haskell.el

Line 59 in 6f2c6c7

(defun haskell-process-completions-at-point ()

[2]

haskell-mode/haskell.el

Line 76 in 6f2c6c7

(haskell-process-get-repl-completions process text))))

[3]

haskell-mode/haskell-process.el

Line 270 in d6921e8

(rawstr (haskell-process-queue-sync-request process reqstr)))

P.S. I have no good way to test how company performs with other prog-modes and large files, however, I've tried to open 10k LOC jQuery source and tested company with it, having company-idle-delay set to 0 and company-minimum-prefix-length set to 2. In this case completions does not appear instantly, but popup delay was really short, and more important there were no hangs, i.e. I easily can continue typing and see feedback immediately.

[geraldus]

Okay, I've written very first version of asynchronous company backend, though it still need some minor tweaks it provides required minimal functionality and makes company to work properly without hangs.
Also I want to introduce some features in nearest future like:

• annotations with module information like company-ghc backend does
• type information in echo area (though I'm not using it myself)
• completions for language pragmas (this will be helpful when structured mode is not used)
• completions for module names
• include some default completions (:complete repl does not provide completions for things defined in let statements etc.)

So, the source of evil is this: (add-hook 'completion-at-point-functions 'haskell-process-completions-at-point nil t). If we remove this hook we'll end up with completions from tags-completion-at-point-function. I haven't detailed look at how useful is it yet. Also I need to figure out when TAGS are generated if haskell-tags-on-save is not set.

Also, as the thing I wrote is a backend, it brings company dependency and I have no strong opinion how should I provide this backend, should it be a separate package?

[gracjan]

Can we see the code somewhere?

[geraldus]

Surely, will upload it in about a hour. Indeed it was very simple!

[lukehoersten]

So if I understand correctly, you're saying you're basically disabling repl-based completion and forcing company to just complete from the TAGS?

[geraldus]

Here is it https://github.com/geraldus/company-interactive-haskell-backend.
@lukehoersten currently I did not disable anything, because when I use backend I've wrote company does not uses any other sources of completions. But I definitely will try to remove that hook and test TAGS based completions

[gracjan]

Nice! And it seems pretty non-invasive.

Is there an example in the internet somewhere what is the proper way of handling 'optional dependencies' in Emacs?

[lukehoersten]

Is there a way to decouple the async completion hook from the company-mode dependency? I think we'd like haskell-mode not to depend on external modes if we can help it.

[geraldus]

I can't be certain right now, but I already though of such option, because this will ease auto-complete support.

[lukehoersten]

Right that seems like the way to go: improve the hook by adding an async version so auto-complete and company can both use that. Sounds good @geraldus

[geraldus]

@lukehoersten are you familiar with autocomplete? Is there async support for completions?

To support async completions company need a fetcher function which takes exactly one argument — a callback to be executed with list of completions when ready. If there similar approach in autocomplete I think it will be very easy. But there is small caveat, as fetcher should take exact one argument it should take a prefix (a partial text to be completed) somehow at run time, in case of company I use company-grab-symbol currently, but likely it will be changed to haskell-ident-at-point or something else more haskell specific. So all we need is to provide function which takes a prefix-grabbing-function, and builds and returns fetcher function.

[lukehoersten]

I'm not familiar with autocomplete but an async complete hook is generic enough that any autocomplete functionality could use it.

[gracjan]

Note that the feature or async hooks is provided by company and if autocomplete expects a synchronously and immediately delivered list of completions then we will not be able to work around the missing functionality. So we will need to use our async functionality by explicitly waiting for it like now. It might hang, but then the advice would be to switch to company-mode.

[geraldus]

I'm trying to write a function which will return a completion prefix for GHC options pragma and need help with this stuff.

Let me describe expected behaviour first (| indicates point position).

Nothing to prefix:
{-# OPTIONS_GHC| ⟹ nil
{-# OPTIONS_GHC|#-} ⟹ nil
{-# OPTIONS_GHC |#-} ⟹ nil

Single -:
{-# OPTIONS_GHC -| ⟹ -
{-# OPTIONS_GHC -| #-} ⟹ -

Double --:
{-# OPTIONS_GHC --| ⟹ --
{-# OPTIONS_GHC --| #-} ⟹ --
(1) I'm not sure should it support options prefixed with two dashes, likely they are nonsense in pragma context.

Valid options:
{-# OPTIONS_GHC -fn| ⟹ -fno
{-# OPTIONS_GHC -f| #-} ⟹ -f
{-# OPTIONS_GHC -user-| #-} ⟹ -user-
{-# OPTIONS_GHC --num| ⟹ --num
{-# OPTIONS_GHC --show-| #-} ⟹ --show-
{-# OPTIONS_GHC -XOv| #-} ⟹ -XOv

Multiple options support:
{-# OPTIONS_GHC -fwarn-implicit-prelude -f| ⟹ -f
{-# OPTIONS_GHC -fwarn-implicit-prelude -f| #-} ⟹ -f
{-# OPTIONS_GHC -fwarn-implicit-prelude -f| -fwarn-missing-import-lists ⟹ -f
{-# OPTIONS_GHC -fwarn-implicit-prelude -f| -fwarn-missing-import-lists #-} ⟹ -f

Return nil if in the middle of character sequence:
{-# OPTIONS_GHC -fwa|rn ⟹ nil
{-# OPTIONS_GHC -|fw #-} ⟹ nil

(2) Also looks like some options should be followed by value, e.g. -odir, but I'm not sure is it makes sense in pragma context. I have very little usage experience of this pragma.

I've took this existing code as starting point. Initially I've decided to replace looking-back with looking-at in sake of performance. But in this case I need to remember point position and make some complex comparison of match bounds against it, especially for multiple option cases.

So there are already two questions. The third question if it is possible to have some sane regular expression for looking-at to handle all this complex stuff (the idea is to be able easily grab prefix using match-string-no-properties function)?

One last note: it is also needed prefix bounds to be returned, i.e. the positions in buffer where prefix starts and ends, and I guess the easiest way it to get everything (prefix and bound) with a single regexp using match- functions. For now I'm leaning to implantation using looking-back with line beginning position as limit.

Pinging @gracjan