Validate DataSources more strictly when reading from disk

[fm3]

full datasource validation now also runs when reading datasource-properties.jsons from disk. The result is shown in the dashboard on “show error”.

The same validation also runs in the reserveManualUpload route

This also fixes a regression from #8844 where you could no longer hit reload on unusable datasets.

It has, however, been slightly relaxed (no more path checks as those have been moved elsewhere in #8844 )

Steps to test:

• Edit datasource on disk, making it invalid (e.g. introduce duplicate layer names or mags)

Issues:

• fixes https://scm.slack.com/archives/CMBMU5684/p1747643139269819

---

• Added changelog entry (create a $PR_NUMBER.md file in unreleased_changes or use ./tools/create-changelog-entry.py)
• Considered common edge cases
• Needs datastore update after deployment

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch datasource-validation

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[MichaelBuessemeyer]

Code looks good & testing went good as well.

I left 1 question :)

[MichaelBuessemeyer]

Why did you add this change? Is there a possible scenario where the datasource cannot be retrieved from the cache but exists on disk / in dir?

[fm3]

Yes, the cache only handles usable datasources. If the existing datasource is unusable due to an error, the cache will just return a Failure. However, in this context, we want to re-read the one from disk in this case, re-validate it, and sent it to webknossos if it is now usable. Only after that, it will also be available through the cache. So this part is actually the bugfix for the reload button in the dashboard (for non-virtual datasets). Does that make sense?