How to limit the number of values shown for autocomplete field?

[nicolazilio]

First of all, I'd like to say good job on creating DjangoQL! It's a great app.

I have a quick question about your autocomplete function. I use it to show suggestions for a field that has many possible values (>1000), like so

class SearchFieldOptPartDescription(StrField):
    """Create a list of unique part descriptions for search"""

    model = order_management_Order
    name = 'part_description'
    suggest_options = True

    def get_options(self):
        return super(SearchFieldOptPartDescription, self).\
        get_options().all().distinct()

I wonder if you could point me in the right direction as to how I could reduce the number of shown options to, let's say, the top 10 after a person has started typing 3 letters. Right now, it shows all available values right from the start.

For example for jquery autocomplete function I would use slice(), like so

$('#id_part_description').autocomplete({
            source: function(request, response) {
                    var results = $.ui.autocomplete.filter(product_names, request.term);
                    response(results.slice(0, 10));

I suspect I would have to change DjangoQL's javascript to accomplish this. But I don't know where to find the relevant lines of code that I need to modify.

Thanks a lot in advance!

[vkrizan]

Hi,

It makes sense to me to limit the number of suggested values, if there are lot of possible values to choose from. Shortening the list of suggested values can be done by matching the with the type value and/or with a fuzzy search, plus limiting the maximum values that are shown.

I guess you are looking for generateSuggestions JS method. I see that there could be two options here:

1. Filter while generating suggested options

   djangoql/djangoql/static/djangoql/js/completion.js

   Line 810 in 339f40f

   this.suggestions = field.options.map(function (f) {

2. Enhance filter function

   djangoql/djangoql/static/djangoql/js/completion.js

   Line 733 in 339f40f

   searchFilter = function (item) {

   .

The first option should create less memory footprint.

[nicolazilio]

Thanks for pointing me in the right direction.

I tried to implement the following changes:

1. I changed

   djangoql/djangoql/static/djangoql/js/completion.js

   Line 806 in 339f40f

   .indexOf(this.prefix.toLowerCase()) >= 0;

   
   to .indexOf(this.prefix.toLowerCase()) >= 3;,

2. I added slice(0,10) to
   

   djangoql/djangoql/static/djangoql/js/completion.js

   Line 810 in 339f40f

   this.suggestions = field.options.map(function (f) {

Although these changes theoretically accomplish what I want, for some reason they also result in the autocomplete function not reporting all possible options, when implemented either individually or together. For example, if I search for an item that I know is in the database, such as SspI, by typing ssp, I don't get it as a option. While others possible results are reported. The original js code does report SspI when I search for ssp.

[vkrizan]

Check out the case-insensitive search, which is there for "value" type fields at

djangoql/djangoql/static/djangoql/js/completion.js

Line 806 in 339f40f

.indexOf(this.prefix.toLowerCase()) >= 0;

In other words, you need to normalize both input value and suggested value (e.g. to lower-case) before comparing it.

[stebunovd]

hey @nicolazilio,

thank you for bringing this issue to the discussion. The current implementation of suggestion options feature wasn't designed to deal with large recordsets. It loads all of the suggestion options synchronously together with introspection schema and performs search through them on the frontend. It works well with a relatively small number of suggestion options, but as a number of records grow, it becomes less and less usable.

I suppose the better solution for large recordsets would perform the search asynchronously, i.e. send the search request to the backend and get a limited set of search results from there. This would keep the initial page load times low and at the same time support multiple suggestion fields with an arbitrarily large number of records. In theory, it could also make it easier to implement more complex suggestion strategies (like fuzzy search or typos correction).

If we decide to follow this approach, it would require modifications to the introspection schema format (introducing async mode for suggestion options), and also an API endpoint on the backend for suggestions search and support for it on the frontend. This is probably not a quick fix, it may require time to build and test everything, but I guess it should be a more universal solution covering almost all of the use cases that people might have for suggestion options.

I'm open to other ideas of course - please feel free to suggest. If you think that an easier solution might exist please let me know.

thanks,

Denis

[nicolazilio]

Dear @stebunovd,

Thanks for the very thorough explanation.

I'm actually surprised by how well the current implementation works: populating 1000s of suggestions takes literally (and I mean it in the real sense of the word) a split second.

For me, the main issue was really more about formatting than performance per se. If you think you could combine the two things, that'd be awesome. Unfortunately, my knowledge of djangoql and, in fact, Django, is probably not good enough to suggest anything better.

Thank you again!
Nicola

[stebunovd]

alright, it's already in master and should be in the next release v0.14.0