Integrate search design

[dstufft]

We need a page that will allow users to search via the web UI for projects to aid in the discovery of projects.

[dstufft]

#739 implements a very basic version of this, but I know the design will require more features.

[nlhkabu]

@dstufft I am going to remove the design tag for this and change the title to 'integrate search design' to indicate that my part is done

[dstufft]

This will need things like:

• Faceting
• Sorting
• Pagination
• Number of total results

[di]

@dstufft Looks like we already have pagination and total # of results, and #1004 adds sorting by release date.

Are there additional types of sorting we might want to support? Number of downloads might be useful.

For faceting, what else besides "Development Status"?

[dstufft]

Number of downloads is a bit interesting, though it'll take some thought maybe. Right now we're regenerating the search index every 15 minutes or so, but in the future I plan to make a full regeneration a rare thing and just have it add/remove stuff from the current index based on what is actually changing. That means there won't be a great "hook" to trigger updating the download counts as downloads happen. That's not an insurmountable thing though, we could either just periodically regenerate the entire index, or have a task that looks for the N oldest projects which were last indexed at least X time ago and reindex those.

I would guess probably ordering by name wouldn't be the worst thing to do.

For faceting, I'm not sure! I guess maybe we want to make it possible to facet on any of the classifier values? That would effectively replace the old browse page then as well as add some real functionality to classifiers.

[dstufft]

FWIW, you can see a list of all available classifiers here: https://pypi.python.org/pypi?%3Aaction=list_classifiers

[nlhkabu]

The original design specified that each classifier should work as a facet, with only one group being open at a time (#755)

On the HTML side, the checkboxes should be nested (see #754).

For the ordering, no. of downloads was definitely a consideration when I put together the design...

Another idea is to calculate package 'popularity' by working out how many other packages depend on that package. Unless this is already how 'relevance' works??

[dstufft]

Relevance is a full text search, it doesn't take into account anything special about the fact these are packages which depend on each other.

We can do ordering by download count, it'll just require a bit of extra work to keep that data reasonably refreshed.

The popularity idea will take a bit more effort, in general most packages don't have that information in a way that PyPI can read it, so I'm not sure if it'd be worth it to implement it until that is the case or not.