-
Notifications
You must be signed in to change notification settings - Fork 14
refactor dataset, remote_dataset, and layer #1371
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: s3_remote_datasets
Are you sure you want to change the base?
Conversation
|
||
def write_ome_metadata(dataset: "Dataset", layer: "Layer") -> None: | ||
if layer.path is None or not is_writable_path(layer.path): | ||
assert layer.path is not None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no assert needed anymore
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great! I think we should track how backwards-compatible this change is. The tests will probably tell us.
return dataset_bbox | ||
|
||
@staticmethod | ||
def get_remote_datasets( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This means it is both on Dataset
and RemoteDataset
?
def __init__(self, layer: "SegmentationLayer", properties: "AttachmentsProperties"): | ||
super().__init__(layer, properties) | ||
|
||
def _get_optional_dataset_path(self) -> UPath | None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this actually optional?
return dataset | ||
|
||
@classmethod | ||
def open_remote( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this and download
also be moved to AbstractDataset
as get_remote_datasets
?
|
||
@abstractmethod | ||
@property | ||
def attachments_type(self) -> type[AttachmentsTypeT]: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
protected?
def attachments_type(self) -> type[AttachmentsTypeT]: | |
def _attachments_type(self) -> type[AttachmentsTypeT]: |
def dataset(self) -> RemoteDataset: | ||
return self._dataset | ||
|
||
def to_layer_to_link(self) -> LayerToLink: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This feels unnessecary. Couldn't the function that takes LayerToLink
take a RemoteLayer
instead?
""" | ||
|
||
_layer: "Layer" | ||
_layer: "AbstractLayer" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it help to make MagView generic over the Layer class?
|
||
|
||
class RemoteDataset(AbstractDataset[RemoteLayer, RemoteSegmentationLayer]): | ||
"""A representation of a dataset on a webknossos server. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"""A representation of a dataset on a webknossos server. | |
"""A representation of a dataset managed by a WEBKNOSSOS server. |
"""Move the dataset to a folder. Specify the folder like this `RemoteFolder.get_by_path("Datasets/Folder_A")`.""" | ||
self._update_dataset_info(folder_id=folder.id) | ||
|
||
def download_mesh( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Out of scope for this PR, but this need to move. Probably to the RemoteSegmentationLayer
.
Please also make sure that the docs work for the new classes. |
Description:
First draft for the refactoring of the dataset, remote_dataset, layer, and attachments.
builds upon Read directly from remote paths, upload Dataset with reserve_manual_upload #1359
Fix module structure, and imports
Todos:
Make sure to delete unnecessary points or to check all before merging: