Travis is unhappy (again)

[twiecki]

======================================================================
ERROR: pymc3.tests.test_plots.test_plots
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/tests/test_plots.py", line 23, in test_plots
    trace = sample(3000, step=step, start=start)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 182, in sample
    return sample_func(**sample_args)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 193, in _sample
    for strace in sampling:
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/tqdm/_tqdm.py", line 833, in __iter__
    for obj in iterable:
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 282, in _iter_sample
    point, states = step.step(point)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/arraystep.py", line 148, in step
    apoint, stats = self.astep(bij.map(point))
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/metropolis.py", line 132, in astep
    accept = self.delta_logp(q, q0)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/compile/function_module.py", line 898, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/compile/function_module.py", line 884, in __call__
    self.fn() if output_subset is None else\
NotImplementedError: input nd
Apply node that caused the error: InplaceDimShuffle{}(inarray1)
Toposort index: 0
Inputs types: [TensorType(float64, (True,))]
Inputs shapes: [(1, 1)]
Inputs strides: [(8, 8)]
Inputs values: [array([[-0.21887536]])]
Outputs clients: [[Elemwise{exp,no_inplace}(InplaceDimShuffle{}.0)]]
HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
-------------------- >> begin captured logging << --------------------
pymc3: DEBUG: Applied log-transform to lam and added transformed lam_log_ to model.
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_iter_sample (pymc3.tests.test_sampling.TestSample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/pymc-devs/pymc3/pymc3/tests/test_sampling.py", line 75, in test_iter_sample
    for i, trace in enumerate(samps):
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 244, in iter_sample
    for i, strace in enumerate(sampling):
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 282, in _iter_sample
    point, states = step.step(point)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/arraystep.py", line 148, in step
    apoint, stats = self.astep(bij.map(point))
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/metropolis.py", line 132, in astep
    accept = self.delta_logp(q, q0)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/compile/function_module.py", line 898, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/compile/function_module.py", line 884, in __call__
    self.fn() if output_subset is None else\
ValueError: Expected 1 dimensions input
Apply node that caused the error: Subtensor{int64:int64:}(inarray1, Constant{0}, Constant{2})
Toposort index: 1
Inputs types: [TensorType(float64, vector), Scalar(int64), Scalar(int64)]
Inputs shapes: [(1, 2), (), ()]
Inputs strides: [(16, 8), (), ()]
Inputs values: [array([[ 0.20649799,  0.20649799]]), 0, 2]
Outputs clients: [[Elemwise{Composite{(i0 + (i1 * sqr((i2 + i3))))}}(TensorConstant{(1,) of -1..7551280194}, TensorConstant{(1,) of -1.3}, TensorConstant{(1,) of 2.1}, Subtensor{int64:int64:}.0)]]
Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/unittest/case.py", line 393, in __call__
    return self.run(*args, **kwds)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/unittest/case.py", line 320, in run
    self.setUp()
  File "/home/travis/build/pymc-devs/pymc3/pymc3/tests/test_sampling.py", line 26, in setUp
    self.model, self.start, self.step, _ = simple_init()
  File "/home/travis/build/pymc-devs/pymc3/pymc3/tests/models.py", line 53, in simple_init
    step = Metropolis(model.vars, np.diag([1.]), model=model)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/metropolis.py", line 106, in __init__
    self.delta_logp = delta_logp(model.logpt, vars, shared)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/metropolis.py", line 453, in delta_logp
    [logp0], inarray0 = pm.join_nonshared_inputs([logp], vars, shared)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/theanof.py", line 207, in join_nonshared_inputs
    for var, slc, shp, dtyp in ordering.vmap}
  File "/home/travis/build/pymc-devs/pymc3/pymc3/theanof.py", line 207, in <dictcomp>
    for var, slc, shp, dtyp in ordering.vmap}
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
======================================================================
ERROR: test_sample (pymc3.tests.test_sampling.TestSample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/pymc-devs/pymc3/pymc3/tests/test_sampling.py", line 62, in test_sample
    pm.sample(steps, step=self.step, njobs=njobs, random_seed=self.random_seed)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 182, in sample
    return sample_func(**sample_args)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 193, in _sample
    for strace in sampling:
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/tqdm/_tqdm.py", line 833, in __iter__
    for obj in iterable:
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 282, in _iter_sample
    point, states = step.step(point)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/arraystep.py", line 148, in step
    apoint, stats = self.astep(bij.map(point))
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/metropolis.py", line 132, in astep
    accept = self.delta_logp(q, q0)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/compile/function_module.py", line 898, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/compile/function_module.py", line 884, in __call__
    self.fn() if output_subset is None else\
ValueError: Expected 1 dimensions input
Apply node that caused the error: Subtensor{int64:int64:}(inarray1, Constant{0}, Constant{2})
Toposort index: 1
Inputs types: [TensorType(float64, vector), Scalar(int64), Scalar(int64)]
Inputs shapes: [(1, 2), (), ()]
Inputs strides: [(16, 8), (), ()]
Inputs values: [array([[ 0.20649799,  0.20649799]]), 0, 2]
Outputs clients: [[Elemwise{Composite{(i0 + (i1 * sqr((i2 + i3))))}}(TensorConstant{(1,) of -1..7551280194}, TensorConstant{(1,) of -1.3}, TensorConstant{(1,) of 2.1}, Subtensor{int64:int64:}.0)]]
Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/unittest/case.py", line 393, in __call__
    return self.run(*args, **kwds)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/unittest/case.py", line 320, in run
    self.setUp()
  File "/home/travis/build/pymc-devs/pymc3/pymc3/tests/test_sampling.py", line 26, in setUp
    self.model, self.start, self.step, _ = simple_init()
  File "/home/travis/build/pymc-devs/pymc3/pymc3/tests/models.py", line 53, in simple_init
    step = Metropolis(model.vars, np.diag([1.]), model=model)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/metropolis.py", line 106, in __init__
    self.delta_logp = delta_logp(model.logpt, vars, shared)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/metropolis.py", line 453, in delta_logp
    [logp0], inarray0 = pm.join_nonshared_inputs([logp], vars, shared)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/theanof.py", line 207, in join_nonshared_inputs
    for var, slc, shp, dtyp in ordering.vmap}
  File "/home/travis/build/pymc-devs/pymc3/pymc3/theanof.py", line 207, in <dictcomp>
    for var, slc, shp, dtyp in ordering.vmap}
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
======================================================================
ERROR: test_summary_0d_variable_model (pymc3.tests.test_stats.TestStats)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/pymc-devs/pymc3/pymc3/tests/test_stats.py", line 121, in test_summary_0d_variable_model
    trace = pm.sample(100, step=step)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 182, in sample
    return sample_func(**sample_args)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 193, in _sample
    for strace in sampling:
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/tqdm/_tqdm.py", line 833, in __iter__
    for obj in iterable:
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 282, in _iter_sample
    point, states = step.step(point)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/arraystep.py", line 148, in step
    apoint, stats = self.astep(bij.map(point))
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/metropolis.py", line 132, in astep
    accept = self.delta_logp(q, q0)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/compile/function_module.py", line 898, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/compile/function_module.py", line 884, in __call__
    self.fn() if output_subset is None else\
NotImplementedError: input nd
Apply node that caused the error: InplaceDimShuffle{}(inarray1)
Toposort index: 1
Inputs types: [TensorType(float64, (True,))]
Inputs shapes: [(1, 1)]
Inputs strides: [(8, 8)]
Inputs values: [array([[ 0.20649799]])]
Outputs clients: [[Elemwise{Composite{((i0 * (i1 + (i2 * sqr((i3 + i4))))) - (i0 * (i1 + (i2 * sqr((i3 + i5))))))}}(TensorConstant{0.5}, TensorConstant{-2.36260559534}, TensorConstant{-0.591715976331}, TensorConstant{2.1}, InplaceDimShuffle{}.0, InplaceDimShuffle{}.0)]]
HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
======================================================================
ERROR: test_summary_1d_variable_model (pymc3.tests.test_stats.TestStats)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/pymc-devs/pymc3/pymc3/tests/test_stats.py", line 130, in test_summary_1d_variable_model
    trace = pm.sample(100, step=step)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 182, in sample
    return sample_func(**sample_args)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 193, in _sample
    for strace in sampling:
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/tqdm/_tqdm.py", line 833, in __iter__
    for obj in iterable:
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 282, in _iter_sample
    point, states = step.step(point)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/arraystep.py", line 148, in step
    apoint, stats = self.astep(bij.map(point))
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/metropolis.py", line 132, in astep
    accept = self.delta_logp(q, q0)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/compile/function_module.py", line 898, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/compile/function_module.py", line 884, in __call__
    self.fn() if output_subset is None else\
ValueError: Expected 1 dimensions input
Apply node that caused the error: Subtensor{int64:int64:}(inarray1, Constant{0}, Constant{2})
Toposort index: 1
Inputs types: [TensorType(float64, vector), Scalar(int64), Scalar(int64)]
Inputs shapes: [(1, 2), (), ()]
Inputs strides: [(16, 8), (), ()]
Inputs values: [array([[ 0.20649799,  0.20649799]]), 0, 2]
Outputs clients: [[Elemwise{Composite{(i0 + (i1 * sqr((i2 + i3))))}}(TensorConstant{(1,) of -2..6260559534}, TensorConstant{(1,) of -0..1715976331}, TensorConstant{(1,) of 2.1}, Subtensor{int64:int64:}.0)]]
Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/nose/case.py", line 151, in runTest
    test(result)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/unittest/case.py", line 393, in __call__
    return self.run(*args, **kwds)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/home/travis/build/pymc-devs/pymc3/pymc3/tests/test_stats.py", line 129, in test_summary_1d_variable_model
    step = Metropolis(model.vars, np.diag([1.]), blocked=True)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/metropolis.py", line 106, in __init__
    self.delta_logp = delta_logp(model.logpt, vars, shared)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/metropolis.py", line 453, in delta_logp
    [logp0], inarray0 = pm.join_nonshared_inputs([logp], vars, shared)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/theanof.py", line 207, in join_nonshared_inputs
    for var, slc, shp, dtyp in ordering.vmap}
  File "/home/travis/build/pymc-devs/pymc3/pymc3/theanof.py", line 207, in <dictcomp>
    for var, slc, shp, dtyp in ordering.vmap}
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
======================================================================
ERROR: test_summary_2d_variable_model (pymc3.tests.test_stats.TestStats)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/pymc-devs/pymc3/pymc3/tests/test_stats.py", line 140, in test_summary_2d_variable_model
    trace = pm.sample(100, step=step)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 182, in sample
    return sample_func(**sample_args)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 193, in _sample
    for strace in sampling:
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/tqdm/_tqdm.py", line 833, in __iter__
    for obj in iterable:
  File "/home/travis/build/pymc-devs/pymc3/pymc3/sampling.py", line 282, in _iter_sample
    point, states = step.step(point)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/arraystep.py", line 148, in step
    apoint, stats = self.astep(bij.map(point))
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/metropolis.py", line 132, in astep
    accept = self.delta_logp(q, q0)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/compile/function_module.py", line 898, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/theano/compile/function_module.py", line 884, in __call__
    self.fn() if output_subset is None else\
ValueError: Expected 1 dimensions input
Apply node that caused the error: Subtensor{int64:int64:}(inarray1, Constant{0}, Constant{4})
Toposort index: 1
Inputs types: [TensorType(float64, vector), Scalar(int64), Scalar(int64)]
Inputs shapes: [(1, 4), (), ()]
Inputs strides: [(32, 8), (), ()]
Inputs values: [array([[ 0.20649799,  0.20649799,  0.20649799,  0.20649799]]), 0, 4]
Outputs clients: [[Reshape{2}(Subtensor{int64:int64:}.0, TensorConstant{(2,) of 2})]]
Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/site-packages/nose/case.py", line 151, in runTest
    test(result)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/unittest/case.py", line 393, in __call__
    return self.run(*args, **kwds)
  File "/home/travis/miniconda2/envs/testenv/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/home/travis/build/pymc-devs/pymc3/pymc3/tests/test_stats.py", line 139, in test_summary_2d_variable_model
    step = Metropolis(model.vars, np.diag([1.]), blocked=True)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/metropolis.py", line 106, in __init__
    self.delta_logp = delta_logp(model.logpt, vars, shared)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/step_methods/metropolis.py", line 453, in delta_logp
    [logp0], inarray0 = pm.join_nonshared_inputs([logp], vars, shared)
  File "/home/travis/build/pymc-devs/pymc3/pymc3/theanof.py", line 207, in join_nonshared_inputs
    for var, slc, shp, dtyp in ordering.vmap}
  File "/home/travis/build/pymc-devs/pymc3/pymc3/theanof.py", line 207, in <dictcomp>
    for var, slc, shp, dtyp in ordering.vmap}

Probably a theano master problem?

[aseyboldt]

Isn't that the failure because of #1770 in combination with multinomial sampling? Do you mean this one? https://travis-ci.org/pymc-devs/pymc3/jobs/200522344

[twiecki]

Wrong copy&paste.

[aloctavodia]

My last PR (#1775) also fails with these errors. Anyway I can help?

[twiecki]

@aloctavodia Not sure what's behind this. Probably a theano update.

[twiecki]

@nouiz any idea what recent theano change might have triggered that?

[nouiz]

I'm not able to reproduce the first error with the master of Theano. Which Theano flags get used?

[twiecki]

https://github.com/pymc-devs/pymc3/blob/master/scripts/test.sh#L5

[twiecki]

Also can't reproduce locally. Seems like we need the docker. @ColCarroll do you have that set up?

[ColCarroll]

My docker is still on 3.5.2, but has newest theano. That particular test passes fine (new warning, though!): ``` Check effective sample size is equal to number of samples when initializing with MAP ... Optimization terminated successfully. Current function value: 4.594693 Iterations: 0 Function evaluations: 1 Gradient evaluations: 1 WARNING (theano.tensor.blas): We did not found a dynamic library into the library_dir of the library we use for blas. If you use ATLAS, make sure to compile it with dynamics library. 100%|█████| 100/100 [00:00<00:00, 628.66it/s] ok ```

…

On Mon, Feb 13, 2017 at 4:13 PM Thomas Wiecki ***@***.***> wrote: Also can't reproduce locally. Seems like we need the docker. @ColCarroll <https://github.com/ColCarroll> do you have that set up? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1773 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACMHEJmj8up-7oc9886CCRpU4ZEQtVPNks5rcMdygaJpZM4L-KLh> .

[twiecki]

I'm totally stumped by this. No one can reproduce it, even in the docker clone of the travis environment. @ColCarroll Did you mention you tried to roll back to an older theano version but it didn't help?

[ColCarroll]

Sigh, I'm an idiot and was running a test that was non-problematic. Can now reproduce -- I'm looking for a theano SHA where it works (if someone wants to share git-bisect magic, I gather that would help!)

[ColCarroll]

Fails at least back to 0.8.2 -- I don't think this is a theano problem. I can debug more later, but this reproduces the error pretty quick if someone wants to take a look:

./scripts/test.sh -vsx pymc3/tests/test_stats.py:TestStats.test_summary_0d_variable_model