Skip to content

Improve channel joining in FlashLoader #128

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
zain-sohail opened this issue Jun 16, 2023 · 4 comments
Closed

Improve channel joining in FlashLoader #128

zain-sohail opened this issue Jun 16, 2023 · 4 comments
Assignees
Labels
wontfix This will not be worked on

Comments

@zain-sohail
Copy link
Member

Is this sparse and dense channels likewise? Is it mist efficient doing it this way (creating lots of dataframe and joining?)

Originally posted by @rettigl in #118 (comment)

Improvement in concatenate channels can be made to the join operation.

@zain-sohail
Copy link
Member Author

          I had a brief look into this. Most of the time is spend merging the dataframes. I think this very inefficient. But I don't know if it can be done better with your multi-index stuff. Creating the per-electron multi-index also takes some time. How long does this example file take on maxwell, and what is the hardware there?

Originally posted by @rettigl in #118 (comment)

@zain-sohail
Copy link
Member Author

I think this is a nice improvement but currently not really slowing us majorly. Shall I look into this or put is as wontfix @rettigl @steinnymir

@steinnymir
Copy link
Member

I agree, the timing for generating the dataframe currently is really fast, so this is not really an issue worth investing on.

@rettigl rettigl added the wontfix This will not be worked on label Oct 26, 2023
@rettigl
Copy link
Member

rettigl commented Oct 26, 2023

This essentially is parallelized very efficiently, so also from my side no real need to further improve. The major game changer was the improvements done to the fill_na. Closing this.

@rettigl rettigl closed this as completed Oct 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wontfix This will not be worked on
Projects
None yet
Development

No branches or pull requests

3 participants