update Unet3d #4
Conversation
add_brats fix eval OOM fix mis issues
* fix tb gsc issue * fix tb gsc issue #2 * fix flexibale out_dim of result for both dataset
fix quality_threshold typo
* add simple loader * add simple loader * add simple loader * add simple loader * update run sripts * update list hard-coded * using gfile * using gfile * try with no loading * try with no loading * remove glob * fix type * fix type * fix type * try reduce list * remove dummy loader * fix val loader * comment out transpose * rm hard coded code * clean up * clean up
|
I don't have enough context to review the full pr, @miladm Do you want to give it a try? |
There was a problem hiding this comment.
Thanks for the PR @suexu1025
Left a few questions and comments here.
We need to outline the perf gains w/ and w/o Brats in this PR. Feel free to make a separate PR for Brats if more convenient - your call.
address PR comment and summary of the PR benchmark result.UNET3D @ BRATS with BS 64 convergent at mean epoch of 63.7 with std 9.2 @0.845 DICE general performance from this PR: |
UNET3D @ BRATS with BS 64 convergent at mean epoch of 63.7 with std 9.2 @0.845 DICE
UNET3D @ KITS with BS 56 convergent at mean epoch of 2213.4 with std 499.9. @0.908 DICE
e2e training time of BRATS vs KITS ~ 1:4
general performance from this PR:
on v4-8 expected throughputs of 17.8 samples/s on GCS.
Improvement over original impl: 28% using disk, 48% using GCS
on v4-128 expected throughputs of 263.2 samples/s on GCS
Improvement over original impl: 2.7x using disk 6.4x using GCS.