File tree 2 files changed +18
-4
lines changed
cloud-infrastructure/ai-infra-gpu/ai-infrastructure/llm-benchmark-docker
2 files changed +18
-4
lines changed Original file line number Diff line number Diff line change @@ -125,6 +125,24 @@ are gated and require an access token.
125
125
results in the directory ` ./results ` , containing information about the vLLM
126
126
parameters and the shape used.
127
127
128
+ To run only certain scenarios and concurrent request settings, modify
129
+ [ ` compose.yaml ` ] ( files/compose.yaml ) and have the ` command ` for the ` perf `
130
+ container read, i.e:
131
+ ``` yaml
132
+ command :
133
+ - " wait-for-it.sh"
134
+ - " --timeout=300"
135
+ - " llm:8000"
136
+ - " --"
137
+ - " /appli/scripts/benchmark.py"
138
+ - " --scenario"
139
+ - " chatbot"
140
+ - " --concurrency"
141
+ - " 1"
142
+ - " 4"
143
+ - " 16"
144
+ ` ` `
145
+
128
146
5. Run the plotting:
129
147
` ` ` sh
130
148
docker-compose run plot
Original file line number Diff line number Diff line change @@ -32,10 +32,6 @@ services:
32
32
- " llm:8000"
33
33
- " --"
34
34
- " /appli/scripts/benchmark.py"
35
- - " --concurrency"
36
- - " 1"
37
- - " 4"
38
- - " 16"
39
35
plot :
40
36
build : plot
41
37
container_name : plot
You can’t perform that action at this time.
0 commit comments