How do I deploy ACP Server in a High Availability configuration? #130
-
|
How do I deploy ACP Server given in acp-sdk in a High Availability configuration? |
Beta Was this translation helpful? Give feedback.
Replies: 5 comments 1 reply
-
|
I'll be assuming deployment with multiple replicas of the server and a load balancer setup to achieve HA. The server isn't currently able to be deployed in multi-replica configuration while maintaining full consistency as it holds all the structures in memory, including execution environment for runs. We plan to resolve this in the future. However, you can still achieve a good approximation with sticky sessions on your load balancer. Additionally, we are thinking about adding a router component into the ecosystem that would be capable of application-level load balancing even across heterogenous servers. |
Beta Was this translation helpful? Give feedback.
-
|
@bikram990 ACP currently doesn't synchronize sessions or runs across replicas. Right now, ACP supports HA (agents running multiple replicas with load balancing) only under limited scenarios: specifically, stateless agents consumed via sync mode without awaits. For cases involving stateful agents, async mode, or awaits, session management across multiple instances isn't supported yet. To handle these scenarios, you'd currently need session affinity (sticky sessions) or implement application-level session synchronization. We plan improvements in this area. Please consider opening a feature request describing your use case so we can prioritize appropriately. |
Beta Was this translation helpful? Give feedback.
-
|
@matoushavlena and @pilartomas Just to clarify, ACP as a protocol supports HA mode but |
Beta Was this translation helpful? Give feedback.
-
|
Correct 👍 |
Beta Was this translation helpful? Give feedback.
-
|
@bikram990 thank you for the question. This issue (already work in progress) should introduce the needed HA support to ACP SDK: #143. |
Beta Was this translation helpful? Give feedback.
-
|
@matoushavlena we have been working on deploying ACP servers/agents on Kubernetes using an operator - this should allow to scale agents up and down - as deployments or statefulsets - pls. see discussion here i-am-bee/beeai-platform#503 (comment) |
Beta Was this translation helpful? Give feedback.
@bikram990 thank you for the question. This issue (already work in progress) should introduce the needed HA support to ACP SDK: #143.