Running NEMO on CPU cluster

[philm001]

Hello all,

I know that Nemo is meant to run on Nvidia. Before I drop the 2k to purchase the latest RTX, I would like to evaluate Nemo and learn how it works with what I have. I know that I may not get all of the features. Right now, I don't have a gpu that can run NEMO. But I would like to run it with my CPU. I wouldn't be working with the latest and greatest models but at least something to start learning.

Is it possible for NEMO to run on the CPU? Or is it only GPU?

I found this post here, it is 2 years old. Has there been any changes since then?
#3553

[philm001]

I have an older cluster computer that doesn't have the GPU power but has a lot of CPU power. I am thinking to set it up as a SLUM cluster. I would like to play around with Nvidia NEMO and show some simple results with a simple model (something less then 1B parameters). Can I still use NEMO with SLUM cluster if it doesn't have GPU?

[ko3n1g]

Hey @philm001, thanks for reaching out to us - and apologies for the delay of getting back to you!

It is indeed the case that a few collections like LLM only work on GPU environments (due to their dependency to TransformerEngine). Other collections like the ones from Speech (ASR, TTS, SpeechLM) should be more suitable for exploration of NeMo. For those, we mostly aim at serving model predictions through the CPU-paths for now.

However, CPU-support is very young and not battle-tested. We're happy to robust it by feedback that you provide.

Which collections are you interested in exploring for your use case?