Partial search updates #3693
Partial search updates #3693
Conversation
warehouse/search/__init__.py
Outdated
| def store_projects_for_project_reindex(config, session, flush_context): | ||
| # We'll (ab)use the session.info dictionary to store a list of pending | ||
| # Project updates to the session. | ||
| purges = session.info.setdefault("warehouse.search.project_updates", set()) |
There was a problem hiding this comment.
Probably want to call this something other than purges... reindexable? changed_projects? updated_projects?
warehouse/search/tasks.py
Outdated
| @@ -87,6 +87,9 @@ def _project_docs(db): | |||
| .order_by(Release.name) | |||
| ) | |||
|
|
|||
| if project_name: | |||
| release_data = release_data.filter(Release.name == project_name) | |||
There was a problem hiding this comment.
Probably want to add this filter earlier on releases_list instead of here? Maybe it makes no difference...
There was a problem hiding this comment.
it's damn fast either way, but you're right!
There was a problem hiding this comment.
so, since releases_list is a subquery, it seems it is not easily modified after the fact. probably best to let this sleeping dog lay.
There was a problem hiding this comment.
You could just .subquery it later, and do releases_list = releases_list.filter(...) earlier. Again, might not really matter.
warehouse/search/__init__.py
Outdated
| # a Project to reindex for when the session has been committed. | ||
| for obj in (session.new | session.dirty | session.deleted): | ||
| if obj.__class__ == Project: | ||
| purges.add(obj) |
There was a problem hiding this comment.
If the project has been deleted, I'm not sure it makes sense to add it to the set here, because the .filter below will not return any releases at all. (I guess a deleted project can just fall out of the index when we reindex nightly instead of trying to do something special for it here.)
However if a release has been deleted, it does make sense.
warehouse/search/__init__.py
Outdated
|
|
||
| @db.listens_for(db.Session, "after_commit") | ||
| def execute_project_reindex(config, session): | ||
| purges = session.info.pop("warehouse.search.project_updates", set()) |
A fools attempt at #701
Incrementally reindexes new documents or unindexes whenever a Project or Release is created, updated, or deleted.