Skip to content

Commit 7d4ec3b

Browse files
authored
Update torchvision/prototype/datasets/_builtin/README.md
1 parent f6301f3 commit 7d4ec3b

File tree

1 file changed

+1
-1
lines changed
  • torchvision/prototype/datasets/_builtin

1 file changed

+1
-1
lines changed

torchvision/prototype/datasets/_builtin/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,7 @@ Of course, there are some common building blocks that should suffice in 95% of t
8383

8484
All of them can be imported `from torchdata.datapipes.iter`. In addition, use `functools.partial` in case a callable needs extra arguments. If the provided `IterDataPipe`'s are not sufficient for the use case, it is also not complicated to add one. See the MNIST or CelebA datasets for example.
8585

86-
`make_datapipe()` gets passes `resource_dps`, which is a list of datapipes that has a 1-to-1 correspondence with the return value of `resources()`. In case of archives with regular extension, the datapipe will contain tuples comprised of the path and the handle for every file in the archive. Otherwise the datapipe will only contain one of such tuples for the file specified by the resource.
86+
`make_datapipe()` receives `resource_dps`, which is a list of datapipes that has a 1-to-1 correspondence with the return value of `resources()`. In case of archives with regular suffixes (`.tar`, `.zip`, ...), the datapipe will contain tuples comprised of the path and the handle for every file in the archive. Otherwise the datapipe will only contain one of such tuples for the file specified by the resource.
8787

8888
Since the datapipes are iterable in nature, some datapipes feature an in-memory buffer, e.g. `IterKeyZipper` and `Grouper`. There are two issues with that:
8989
1. If not used carefully, this can easily overflow the host memory, since most datasets will not fit in completely.

0 commit comments

Comments
 (0)