Enhance mypy support.

[roberthoenig]

This

• runs mypy and lint in a single travis job (so we have five jobs again, the maximum that can be parallely executed)

• adds support for adding directories to the lister exclude list (needed for excluding zulip_bots/zulip_bots/bots

• runs mypy for each package fully and separately

• excludes all files that need to be fixed.

Btw I noticed that we have a lister duplicate; one in tools and one in tools/server_lib. Which one should we keep? I vote for the server_lib one and move pep8 and custom_check.py there.

[timabbott]

Are you sure this didn't already work? This should work:

        if any(absfpath == expath or absfpath.startswith(expath + '/')                               
               for expath in exclude_abspaths):                                                      
            continue                                                                                 

[roberthoenig]

From my tests and the comment in the code, excludes_paths didn't exclude directories - we might want to make the same change to zulip?

[timabbott]

I think the comments are inaccurate. Looking at the history, we used to have contrib_bots/bots in the exclude paths in the server repo.

[roberthoenig]

Oh right - I see the issue then, it's Windows paths:

if any(absfpath == expath or absfpath.startswith(expath + '/')
               for expath in exclude_abspaths):
            continue

hardcodes the delimiter, so it didn't work for the Windows files. Will update that snippet instead.

[timabbott]

Cool, yeah, and that fix we should contribute to the main repo too.

[timabbott]

maybe TEST_SUITE=static-analysis? I feel like we probably want this to be clearer than "test-static" as to what it is.

[timabbott]

I think we want it to check all 3 packages even if there are errors in the first one; so we should stored a failed variable instead of existing immediately on the first error.

[roberthoenig]

Okay, I was not sure about that bit.

[timabbott]

I posted a few comments; thanks for working on this @derAnfaenger!

[roberthoenig]

Updated. Includes a new commit to remove the duplicate lister.py

[timabbott]

This is good enough; I made one change to the last commit to remove the |= behavior (it's too hard to read and means the wrong thing, since it's a bitwise or!), and merged.

I'm not totally convinced by the use of pathlib.PurePath; why do we need that? It feels like use of overly complex Python features for something simple :). But we can address that in a follow-up commit.

[roberthoenig]

I used pathlib.PurePath since it makes extracting the first segment of a path very easy. pathlib.Path has the same functionality but is more heavy. os.path doesn't offer any comparable method. The easiest non-path alternative I can think of is extracting the beginning of the string until the first separator.

[timabbott]

OK, I guess that's fine.

[neiljp]

pathlib is 3.4+; are tools generally 3+ only?

[roberthoenig]

The mypy package. is python3 only.