Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix] Redis cluster to exceed the maximum number of connections #484

Merged
merged 1 commit into from
Jan 16, 2025

Conversation

MoFHeka
Copy link
Collaborator

@MoFHeka MoFHeka commented Jan 13, 2025

Description

When there are a large number of client NODES, the initialization commands CLUSTER SLOTS 0 and CLUSTER NODES 0 cause shard 0 in the Redis cluster to exceed the maximum number of connections.

Type of change

  • Bug fix
  • New Tutorial
  • Updated or additional documentation
  • Additional Testing
  • New Feature

Checklist:

  • I've properly formatted my code according to the guidelines
    • By running yapf
    • By running clang-format
  • This PR addresses an already submitted issue for TensorFlow Recommenders-Addons
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works

How Has This Been Tested?

Create a model running with hundreds of Redis tables and distributed in hundreds of nodes sharing same Redis cluster.

…on commands CLUSTER SLOTS 0 and CLUSTER NODES 0 cause shard 0 in the Redis cluster to exceed the maximum number of connections.
@MoFHeka MoFHeka requested a review from rhdong as a code owner January 13, 2025 07:52
Copy link
Member

@Lifann Lifann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -213,6 +214,12 @@ class RedisWrapper<RedisInstance, K, V,
return nullptr;
}

int GenerateRedisRandomTag() {
static thread_local std::mt19937 generator(std::random_device{}());
std::uniform_int_distribution<int> distribution(0, 16383);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make 16383 a const?
btw, any reason to pick 16383?

@MoFHeka MoFHeka merged commit b3bc3d4 into tensorflow:master Jan 16, 2025
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants