Issue connecting to RedisCluster that has more than one network interface

[derekste]

Version: What redis-py and what redis version is the issue happening on?
redis-py 4.3.4, 4.3.5
redis 7.0.4

Platform: What platform / version? (For example Python 3.5.1 on Windows 7 / Ubuntu 15.10 / Azure)
Python 3.6.8, 3.8.2 Linux
Python 3.10.7 MacOS

Description: Description of your issue, stack traces from errors and code that reproduces the issue

I have a redis cluster running on a server that has two network interfaces.

Locally on the server, the following code works for connecting to the cluster:

>>> import redis
>>> rc = redis.RedisCluster(host="127.0.0.1", port=30001)
>>> rc = redis.RedisCluster(host="my.public.ip.address", port=30001)
>>> rc = redis.RedisCluster(host="my.private.ip.address", port=30001)

On a separate host that can only reach the cluster via its public IP address, the following succeeds:

>>> import redis
>>> rc = redis.RedisCluster(host="my.public.ip.address", port=30001)

On a third host that can only reach the cluster via its private IP address, the following FAILS:

>>> import redis
>>> rc = redis.RedisCluster(host="my.private.ip.address", port=30001)

And produces the following errors:

<class 'redis.exceptions.ConnectionError'>
Traceback (most recent call last):
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 611, in connect
    sock = self.retry.call_with_retry(
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 612, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 677, in _connect
    raise err
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 665, in _connect
    sock.connect(socket_address)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1067, in _execute_command
    connection = get_connection(redis_node, *args, **kwargs)
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 50, in get_connection
    return redis_node.connection or redis_node.connection_pool.get_connection(
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 1387, in get_connection
    connection.connect()
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 617, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:30001. Connection refused.
<class 'redis.exceptions.ConnectionError'>
Traceback (most recent call last):
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 611, in connect
    sock = self.retry.call_with_retry(
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 612, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 677, in _connect
    raise err
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 665, in _connect
    sock.connect(socket_address)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1067, in _execute_command
    connection = get_connection(redis_node, *args, **kwargs)
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 50, in get_connection
    return redis_node.connection or redis_node.connection_pool.get_connection(
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 1387, in get_connection
    connection.connect()
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 617, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:30001. Connection refused.
<class 'redis.exceptions.ConnectionError'>
Traceback (most recent call last):
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 611, in connect
    sock = self.retry.call_with_retry(
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 612, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 677, in _connect
    raise err
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 665, in _connect
    sock.connect(socket_address)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1067, in _execute_command
    connection = get_connection(redis_node, *args, **kwargs)
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 50, in get_connection
    return redis_node.connection or redis_node.connection_pool.get_connection(
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 1387, in get_connection
    connection.connect()
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 617, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:30001. Connection refused.
<class 'redis.exceptions.ConnectionError'>
Traceback (most recent call last):
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 611, in connect
    sock = self.retry.call_with_retry(
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 612, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 677, in _connect
    raise err
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 665, in _connect
    sock.connect(socket_address)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1067, in _execute_command
    connection = get_connection(redis_node, *args, **kwargs)
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 50, in get_connection
    return redis_node.connection or redis_node.connection_pool.get_connection(
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 1387, in get_connection
    connection.connect()
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 617, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:30001. Connection refused.
<class 'redis.exceptions.ConnectionError'>
Traceback (most recent call last):
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 611, in connect
    sock = self.retry.call_with_retry(
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 612, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 677, in _connect
    raise err
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 665, in _connect
    sock.connect(socket_address)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1067, in _execute_command
    connection = get_connection(redis_node, *args, **kwargs)
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 50, in get_connection
    return redis_node.connection or redis_node.connection_pool.get_connection(
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 1387, in get_connection
    connection.connect()
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 617, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:30001. Connection refused.
Traceback (most recent call last):
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 611, in connect
    sock = self.retry.call_with_retry(
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 612, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 677, in _connect
    raise err
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 665, in _connect
    sock.connect(socket_address)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1067, in _execute_command
    connection = get_connection(redis_node, *args, **kwargs)
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 50, in get_connection
    return redis_node.connection or redis_node.connection_pool.get_connection(
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 1387, in get_connection
    connection.connect()
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 617, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:30001. Connection refused.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1450, in initialize
    copy_kwargs = copy.deepcopy(kwargs)
  File "/usr/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.8/copy.py", line 237, in _deepcopy_method
    return type(x)(x.__func__, deepcopy(x.__self__, memo))
  File "/usr/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib/python3.8/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.8/copy.py", line 161, in deepcopy
    rv = reductor(4)
TypeError: cannot pickle '_thread.lock' object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 572, in __init__
    self.commands_parser = CommandsParser(self)
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/commands/parser.py", line 16, in __init__
    self.initialize(redis_connection)
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/commands/parser.py", line 19, in initialize
    commands = r.execute_command("COMMAND")
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1029, in execute_command
    raise e
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1019, in execute_command
    res[node.name] = self._execute_command(node, *args, **kwargs)
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1103, in _execute_command
    self.nodes_manager.initialize()
  File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1486, in initialize
    raise RedisClusterException(
redis.exceptions.RedisClusterException: ERROR sending "cluster slots" command to redis server 127.0.0.1:30001. error: cannot pickle '_thread.lock' object
>>> 

From this same third host that produces the above error, I can successfully connect with redis-cli via the following command:
redis-cli -c -h my.private.ip.address -p 30001

[uglide]

Hello @derekste

To be able to connect to your Redis Cluster on a private network, you need to update your redis.conf:
bind {{ private_ip }} 127.0.0.1

and add an entry like myrediscluster.internal 127.0.0.1 in hosts on the server running the Redis Cluster
and an entry like myrediscluster.internal {{ private_ip }} on remote hosts.

Finally, update the cluster creation and use myrediscluster.internal to join Redis instances to the cluster.

Make sure that you have a firewall enabled on the Redis Cluster host to ensure that only whitelisted IP addresses can access your Redis Cluster.