Couldn't detect gpu when generation using ray data_parallel_size > 1

[zhaocaibei123]

I'm trying to use data_parallel to generation, this is my command:

CUDA_VISIBLE_DEVICES="0,1"
lm_eval --model vllm \
    --model_args pretrained=./downloads/llama3.2-1b-instruct,trust_remote_code=true,tensor_parallel_size=1,dtype=auto,gpu_memory_utilization=0.3,data_parallel_size=2 \
    --tasks truthfulqa_gen \
    --batch_size auto

I got an error:

And I try to print cuda devices like this:

ray: 2.39.0
vllm: 0.6.4.post1
torch: 2.5.1+cu124
I wonder is there something wrong with my environment and how to solve this problem.

[zhaocaibei123]

I reset my environment and it's ok now.

[KSUN63]

Hi there, I ran into a similar problem of getting a ValueError saying no GPU available when it literally prints out the two GPUs that I requested. Can I know how you reset your environment to make everything work? I tried to do the same thing but this error still shows up.

ValueError: Current node has no GPU available. current_node_resource={'bundle_group_e8bc9d1a4e9c3b75ca8af7086f2b01000000': 2000.0, 'GPU_group_e8bc9d1a4e9c3b75ca8af7086f2b01000000': 2.0, 'node:10.0.8.9': 0.999, 'bundle_group_1_e8bc9d1a4e9c3b75ca8af7086f2b01000000': 1000.0, 'GPU_group_0_e8bc9d1a4e9c3b75ca8af7086f2b01000000': 1.0, 'node:__internal_head__': 1.0, 'node:10.0.8.9_group_0_e8bc9d1a4e9c3b75ca8af7086f2b01000000': 0.001, 'bundle_group_0_e8bc9d1a4e9c3b75ca8af7086f2b01000000': 1000.0, 'CPU': 12.0, 'object_store_memory': 76673319321.0, 'accelerator_type:A40': 1.0, 'memory': 168904411751.0, 'node:10.0.8.9_group_e8bc9d1a4e9c3b75ca8af7086f2b01000000': 0.001, 'GPU_group_1_e8bc9d1a4e9c3b75ca8af7086f2b01000000': 1.0}. vLLM engine cannot start without GPU. Make sure you have at least 1 GPU available in a node current_node_id='61912222bec3e4a6e27043a94fb7162fc186a6830292f0f4e67ae25f' current_ip='10.0.8.9'.