Notebook validation failed: Non-unique cell id

[kohlerjl]

I have recently started receiving this popup error frequently when saving notebooks:

Title: Notebook validation failed

The save operation succeeded, but the notebook does not appear to be valid. The validation error was:
-------------------------
Notebook validation failed: Non-unique cell id 'waiting-opening' detected. Corrected to 'noted-romania'.:
"<UNKNOWN>"

It does not occur in new notebooks, but seems to be triggered after copying and pasting cells from other notebooks. But it is now occurring frequently on two different systems, editing different sets of notebooks. Once it appears within a notebook, it reappears on every subsequent save, with the same 'non-unique cell id' detected, but a different 'corrected to ' value.

I've attached the full output of conda list, but the versions of some particularly relevant packages are:

jupyter 1.0.0 py37h03978a9_6 conda-forge
jupyter_client 6.1.11 pyhd8ed1ab_1 conda-forge
jupyter_console 6.2.0 py_0 conda-forge
jupyter_contrib_core 0.3.3 py_2 conda-forge
jupyter_contrib_nbextensions 0.5.1 py37hc8dfbb8_1 conda-forge
jupyter_core 4.7.1 py37h03978a9_0 conda-forge
jupyter_highlight_selected_word 0.2.0 py37h03978a9_1002 conda-forge
jupyter_latex_envs 1.4.6 py37hc8dfbb8_1001 conda-forge
jupyter_nbextensions_configurator 0.4.1 py37h03978a9_2 conda-forge
jupyterlab_widgets 1.0.0 pyhd8ed1ab_1 conda-forge
nbconvert 5.6.1 py37hc8dfbb8_1 conda-forge
nbformat 5.1.2 pyhd8ed1ab_1 conda-forge
notebook 6.2.0 py37h03978a9_0 conda-forge

[kohlerjl]

conda_list.txt

[kevin-bates]

This message is related to some fairly recent changes to nbformat that introduce cell id metadata for each notebook cell. In this case, the validation logic is encountering a duplicate cell-id that was previously, and randomly, generated 'waiting-opening' and encountering that same cell-id later in the notebook. It's strange that this occurs for the same cell-id value and is as if the corrected-to value is not getting persisted. I'm unable to reproduce this, but I suspect there are a few factors in play here.

I'm cc-ing @MSeal for comment as to what might be going on and how best to proceed (as I'd rather not recommend downgrading).

[MSeal]

Sorry I accidentally hit the close button. 👀 on this now

[MSeal]

I also am struggling to reproduce the event with a very similar dependency list. Even if I manually force cell ids to be invalid / equal the notebook server corrects it with version 6.2.0 before it ever get to the validation error reported here.

@kohlerjl could you post a notebook exhibiting the behavior? I am wondering if something in the file structure will lend a clue as to why the replaced cell-id is not being fixed.

[kohlerjl]

Thanks for the quick follow up.

I spent some time trying to isolate a simple notebook that would reproduce the event. What I've found is that this behavior does not persist between closing and reopening the notebook, or even just refreshing the browser tab (leaving the kernel still running).

However, I've been able to consistently reproduce this behavior by following these steps:

1. Create a new notebook and enter some code in a cell (i.e. 'a = 1')
2. Save the notebook, then refresh the tab
3. Copy the cell and paste a duplicate within the same notebook
4. Save the notebook again, triggering the error

It appears that this behavior does not occur if you duplicate a cell created in the same 'session' (i.e. while the notebook is open in the tab). But If I copy and paste a cell created prior to opening/refreshing the notebook, either from the same notebook or a different notebook, then I get errors about duplicate ids. But the notebook changes do save, and I can simply refresh the tab to mitigate the errors.

I have attached a notebook I produced this way, which gave me the errors prior to reloading it. However, I can't see any difference in the file structure. test.ipynb.txt

I also cannot reproduce this behavior on another system, running Python 3.9.2 and:
ipython 7.19.0
jupyter-client 6.1.7
jupyter-console 6.2.0
jupyter-core 4.6.3
jupyter-server 1.4.1
nbconvert 6.0.7
nbformat 5.0.8
notebook 6.2.0

[kevin-bates]

Thanks @kohlerjl. I'm still unable to reproduce this (sorry). Are you using the Notebook classic or Juptyer Lab interface? (Neither reproduces the issue for me however.)

I also cannot reproduce this behavior on another system, running Python 3.9.2 and:
...
nbformat 5.0.8

This would be because that version of nbformat doesn't contain this consistency check. I'm using nbformat 5.1.2, but, again, no luck. Really curious what else could be going on.

Btw, we started displaying the notebook server version in the console when the server is started in the 6.x timeframe. Could you please confirm the displayed value? The log entry should look similar to the following:

[I 10:06:42.433 NotebookApp] Jupyter Notebook 6.2.0 is running at:

[kohlerjl]

I can confirm that the notebook server reports "Jupyter Notebook 6.2.0 is running at:" on startup.

I've been using Notebook classic. I installed Jupyter Lab and tried the same procedure, but cannot reproduce the behavior there.

I might just transition to using Jupyter Lab going forward, since that seems to be more actively developed now.

[kevin-bates]

I can confirm that the notebook server reports "Jupyter Notebook 6.2.0 is running at:" on startup.

Thank you. I'll have to defer to @MSeal on this one.

I've been using Notebook classic. I installed Jupyter Lab and tried the same procedure, but cannot reproduce the behavior there.

Just a point of clarity, Jupyter Lab >= 3 uses a different server (jupyter_server) whereas Lab < 3 still uses notebook as its server - although I believe the issue here lies in the front-end where cells are manipulated. As a result, it might be helpful to know which version of Lab did not reproduce this behavior and whether you've tried this with Lab < 3.

... Jupyter Lab ... seems to be more actively developed now.

Yes, that is absolutely the case.

[njohnsson]

I get a similar error saying cell ID was corrected to 'domestic-communist'. From comments above it seems cell-id names are randomly generated. Maybe that randomization algorithm should be changed a bit. In this case I thought I was dealing with a virus...

Title: Notebook validation failed
The save operation succeeded, but the notebook does not appear to be valid. The validation error was:
Notebook validation failed: Non-unique cell id 'moving-ultimate' detected. Corrected to 'domestic-communist'.:
""

I am running:
Classic Notebook (Not Jupyter Lab), v 6.2.0; Ubuntu; Chrome Browser
Python 3.8.6 | packaged by conda-forge | (default, Oct 7 2020, 19:08:05)

[MSeal]

@njohnsson make sure you have the latest package versions in your environment. We changed the id algorithm to use hashes and invalidated some of the older packages with name based id generation because it was creating problematic ids and marked the older nbformat packages as deprecated.

Similar to @kevin-bates I've struggled to reproduce the issue in classic. That being said classic is not being actively developed. If you're looking for the same simple look-n-feel I'd suggest using Retrolab or NBClassic with lab instead as new package capabilities will slowly be less supported in classic over time.

[njohnsson]

@MSeal: OK, I will update package versions, but just FYI: I the latest message I got ended with "....Non-unique cell id 'civil-spring' detected. Corrected to 'lesbian-voluntary'."