Skip to content

Conversation

acardona
Copy link
Contributor

New utility method ImgUtil.copy(RandomAccessibleInterval, IterableInterval, int) for multi-threaded copying with int threads.

Addresses the issue of copying "images" that are actually transformed views using linear interpolators and others, which can be costly and, from a scripting framework, it is not entirely trivial to partition a RandomAccessibleInterval for multi-threaded processing.
While it could be done via Views.interval, it is simpler to use a utility method to do it all.

@acardona acardona force-pushed the ImgUtil-copy-RAI-to-II-multithreaded branch from 6d94147 to a071832 Compare January 23, 2019 22:24
@maarzt
Copy link
Contributor

maarzt commented Apr 9, 2019

The copy method in this PR is rather complex, because it needs to deal with all the details of multi-threading. This motivated me to add multi-threading to LoopBuilder #250. The multi-threaded LoopBuilder allows to dramatically simplify the copy method:

public static < T extends Type< T >> void copy(
	final RandomAccessibleInterval< T > src,
	final RandomAccessibleInterval< T > dest )
{
	LoopBuilder.setImages( src, dest ).multiThreaded().forEachPixel(
		(s, d) -> d.set( s )
	);
}

BUT: The destination's type is now an RandomAccessibleInterval rather than an IterableInterval.

@acardona I would like to know your opinion: Is there a benefit of choosing IterableInterval as destination type? What's the motivation for having n_threads as a parameter?

@acardona
Copy link
Contributor Author

Hi @maarzt,

  1. IterableInterval is preferable if the destination type was IterableInterval. Otherwise wrapping is needed, which is very undesirable.
  2. n_threads is needed when running e.g. 4 only threads doing an expensive operation (due to memory constraints) but then the copying can fully use e.g. 32/4 = 8 threads for the copying of each result image into e.g. an ArrayImg. (This is pulled from a real world situation).

The ImgUtil.copy serves a real need in scripting languages that can't optimize loops like for example jython.

@ctrueden
Copy link
Member

ctrueden commented Jan 15, 2020

The pyimagej project would greatly benefit from a performant single-line RAI-to-RAI (and/or RAI-to-II) copy method, which does not require writing any lambdas.

@maarzt Does a way currently exist to achieve such a performant single-line copy operation currently? If not, can we get something added to the library for this soon? We will need it for the pyimagej 1.0.0 release.

See also imagej/pyimagej#46.

ctrueden added a commit to imagej/imagej-common that referenced this pull request Jan 23, 2020
@maarzt maarzt force-pushed the ImgUtil-copy-RAI-to-II-multithreaded branch 2 times, most recently from 6445516 to 437b65d Compare January 28, 2020 13:24
@maarzt maarzt force-pushed the ImgUtil-copy-RAI-to-II-multithreaded branch from 437b65d to 9ff573d Compare January 28, 2020 13:27
@maarzt maarzt merged commit 7e37c9b into master Jan 28, 2020
@ctrueden
Copy link
Member

ctrueden commented Feb 4, 2020

Thanks, @maarzt!

@ctrueden ctrueden deleted the ImgUtil-copy-RAI-to-II-multithreaded branch February 4, 2020 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants