Running 2D Class Averaging with Relion Particles

[juliacpowell1999]

Hello,

I would like to run 2D Class averaging within the Aspire package with particles extracted from relion 4. I have installed Aspire and can successfully run the tests. However, I am a bit confused about how to run class averaging in Aspire. It does not appear that any of the Aspire commands allows for class averaging; I saw some python scripts in the tutorial section of the readthedocs page, but I am a bit confused as to how to run these scripts. Is it assumed that I download these scripts and run them with python in the conda aspire enviroment? And do I need to some how import the relion particles to Aspire?

I really like some of the unique features for class averaging that aspire offers, but I need more detailed instructions regarding how to run these packages on my own.

Thank you!

[garrettwrong]

Hi Julia, thank you for your interest in ASPIRE. I would be happy to help with this. I moved your post to a discussion area, hope you don't mind.

You are correct that there currently is not a command line interface for the class averaging component.

The class averaging tutorial and the experiments section of our gallery demonstrate using the RIRClass2D component, but they cover quite a few different setups. I understand how it might be confusing. The most similar to your question is probably
https://computationalcryoem.github.io/ASPIRE-Python/auto_experiments/experimental_abinitio_pipeline.html#sphx-glr-auto-experiments-experimental-abinitio-pipeline-py . This is a more involved example though.

What I am going to do is trim that down to a more straightforward example along with some notes and how to save out the resulting classes. I'd like to run the paired down script before I post it for you. I'll follow up after it completes.

[juliacpowell1999]

@garrettwrong

Hi Garrett,

I am now running into a memory issue. The script worked well when I tested it on 1000 images; however, it appears that aspire is only running on cpu (when I type in nvidia-smi on the computer, it shows 0 processes running). This was fine for 1000 particles; however, the program understandably freezes when running all 1.3M particles. How do I get aspire to communicate with the gpus?

For reference, I am running aspire on a computer with 4 NVIDIA gpus (191GiB total memory). Aspire reported that I needed 156GiB to process the 1.3M particles, so there should be enough memory.

[garrettwrong]

Hi Julia.

This component in the configuration you are running will not use the GPU. If you are running develop the class averaging component should attempt to use multiple cpus during that stage.

I'll need some more information to help you. How many cores and how much RAM does the machine have? How large is /tmp? Also, ASPIRE should write out a log file in a logs folder. Please try to post the log corresponding to the run you mentioned. With the log I will be able to answer many question about what you are trying to do, and where you were in the process. If you are unable to post the log, maybe you can upload it somewhere I can access it. Thanks

[juliacpowell1999]

The machine has 64 cores and 502GB of RAM. The size of the tmp directory is 252GB.

[juliacpowell1999]

And here is the log file
log.txt

[garrettwrong]

Got it. Ah, busted. Yes, it looks like you haven't done anything wrong, and could just be running out of memory. To be fair, that problem is about 10x what we've run before in testing (or with the parallel code) experimental data.

Coincidentally, batching the class averaging coefficient calculation is on my list of development tasks. I am working on it between other things. It would be great to be able to test it with your problem once it is ready, but I don't have a firm date yet.

Until then you might need to scale back the number of images you are trying to process. Maybe 100k would be adequate.

[garrettwrong]

Hi @juliacpowell1999 , sorry it is taken a bit to get back to you, but we have just recently pushed a new release. I'm hoping the changes might help you run your large (million images) problem. In particular the latest code should compute the area that was consuming the most memory in small batches now.

There were a couple API changes this release. Let me know if I can help. Thanks!