[Bug] Unable to run in WASM (marimo) - cannot install `ujson`

[aadityabhatia]

What happened?

DSPy relies on ujson, which was put in maintenance mode. According to ujson's homepage-

Users are encouraged to migrate to orjson which is both much faster and less likely to introduce a surprise buffer overflow vulnerability in the future.

marimo runs on pyodide and does not support ujson, but orjson works as expected. I am planning to use DSPy within marimo WASM environment and hoping that switching from ujson to orjson will allow that.

Steps to reproduce

marimo snippet that attempts to import dspy and fails

Within marimo, click on "Manage packages" in the left menu bar and attempt to install dspy or ujson. The following error appears-

Can't find a pure Python 3 wheel for 'ujson'.
See: https://pyodide.org/en/stable/usage/faq.html#why-can-t-micropip-find-a-pure-python-wheel-for-a-package

DSPy version

2.6.27

[okhat]

Oh wow. Sad for ujson to go. I guess we should switch to orjson then :/

[aadya940]

@okhat Can I work on this?

[0xEval]

+1
Thanks for looking into this. It prevents deploying WASM notebooks that use DSPy.

[0xEval]

Unfortunately, it seems ujson is not the only culprit here. A trickier problem would be Huggingface's tokenizer lib (see: pyodide/pyodide-recipes#199)