Full yield support, including yield expressions

[rockneurotiko]

In python 3.X yield is a statement (currently supported) but also an expression.
You can do something like this:

from typing import Iterator

def test_yield() -> Iterator[int]:
    value = 1
    try:
        value = yield value  # Here is the yield expression
        yield value
        yield value + 1
    except TypeError as e:
        print(e)

gen = test_yield()
x1 = next(gen)
x2 = gen.send(2)
print(x1)  # 1
print(x2)  # 2
gen.throw(TypeError, 'ERROR')
gen.close()

Right now, mypy complains in the expression:

_program.py: In function "test_yield":
_program.py, line 6: Parse error before "value"

And, also, the Iterator type need the functions send, throw and close, but I think this go in typing project:

_program.py, line 14: Iterator[int] has no attribute "send" 
_program.py, line 17: Iterator[int] has no attribute "throw" 
_program.py, line 18: Iterator[int] has no attribute "close"

The reference is here: https://docs.python.org/3.4/reference/expressions.html#yieldexpr

[rockneurotiko]

Also, you can do a return inside a Iterator, and get in with the yield from expression.

This works:

from typing import List, Iterator, Any
trace = []  # type: List[Any]

def inner() -> Iterator[int]:
    try:
        yield 1
    except ValueError:
        trace.append("inner caught ValueError")
    return 2

def outer():
    v = yield from inner()
    trace.append("inner returned %r to outer" % v)
    yield v

g = outer()
trace.append(next(g))
trace.append(g.throw(ValueError))
assert(trace == [
    1,
    "inner caught ValueError",
    "inner returned 2 to outer",
    2,
])

[o11c]

Ugh, I just hit this bug while reimplementing the parser.

Obviously the new parser supports it (though I'm not touching semanal), but I need the tests to pass before switching everything over to actually use the new parser.