Nicktorwald/gh 52 sql tests

.travis.yml

@@ -4,24 +4,34 @@ sudo: required
 dist: trusty
 
 jdk:
-  - oraclejdk11
-  - openjdk10
+  - openjdk8
   - openjdk11
+  - openjdk12
+
+env:
+  - TNT_VERSION=1.9
+  - TNT_VERSION=1.10
+  - TNT_VERSION=2x
+  - TNT_VERSION=2.2
+
+stages:
+  - checkstyle
+  - test
+
+jobs:
+  include:
+    - stage: checkstyle
+      env: []
+      jdk: openjdk11
+      before_script: skip
+      script: mvn checkstyle:check
+      after_success: skip
 
 before_script:
-  - src/test/travis.pre.sh
+  - .travis/travis.pre.sh
 
 script:
-  - |
-    if [ "${TRAVIS_JDK_VERSION}" = openjdk11 ]; then
-        mvn verify jacoco:report
-    else
-        mvn verify
-    fi
-  - head -n -0 testroot/*.log
+  - .travis/travis.build.sh
 
 after_success:
-  - |
-    if [ "${TRAVIS_JDK_VERSION}" = openjdk11 ]; then
-        mvn coveralls:report -DrepoToken=${COVERALLS_TOKEN}
-    fi
+  - .travis/travis.post.sh

.travis/travis.build.sh

@@ -0,0 +1,11 @@
+#!/bin/sh
+
+set -exu # Strict shell (w/o -o pipefail)
+
+if [ "${TRAVIS_JDK_VERSION}" = "openjdk11" ] && [ "${TNT_VERSION}" = "2x" ]; then
+    mvn verify jacoco:report
+else
+    mvn verify
+fi
+
+head -n -0 testroot/*.log

.travis/travis.post.sh

@@ -0,0 +1,7 @@
+#!/bin/sh
+
+set -exu # Strict shell (w/o -o pipefail)
+
+if [ "${TRAVIS_JDK_VERSION}" = "openjdk11" ] && [ "${TNT_VERSION}" = "2x" ]; then
+    mvn coveralls:report -DrepoToken=${COVERALLS_TOKEN}
+fi

.travis/travis.pre.sh

@@ -0,0 +1,17 @@
+#!/bin/bash
+
+set -exuo pipefail # Strict shell
+
+curl http://download.tarantool.org/tarantool/${TNT_VERSION}/gpgkey | sudo apt-key add -
+RELEASE=`lsb_release -c -s`
+
+sudo rm -f /etc/apt/sources.list.d/*tarantool*.list
+sudo tee /etc/apt/sources.list.d/tarantool_${TNT_VERSION/./_}.list <<- EOF
+deb http://download.tarantool.org/tarantool/${TNT_VERSION}/ubuntu/ ${RELEASE} main
+deb-src http://download.tarantool.org/tarantool/${TNT_VERSION}/ubuntu/ ${RELEASE} main
+EOF
+
+sudo apt-get update
+sudo apt-get -y install tarantool tarantool-common
+
+sudo tarantoolctl stop example

README.md

@@ -5,101 +5,106 @@ align="right">
 
 # Java connector for Tarantool 1.7.4+
 
-[![Coverage Status][coveralls-badge]][coveralls-page]
-
-[coveralls-badge]: https://coveralls.io/repos/github/tarantool/tarantool-java/badge.svg?branch=master
-[coveralls-page]: https://coveralls.io/github/tarantool/tarantool-java?branch=master
-
 [![Join the chat at https://gitter.im/tarantool/tarantool-java](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/tarantool/tarantool-java?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
+[![Maven Central](https://maven-badges.herokuapp.com/maven-central/org.tarantool/connector/badge.svg)](https://maven-badges.herokuapp.com/maven-central/org.tarantool/connector)
+[![Build Status](https://travis-ci.org/tarantool/tarantool-java.svg?branch=master)](https://travis-ci.org/tarantool/tarantool-java)
+[![Coverage Status](https://coveralls.io/repos/github/tarantool/tarantool-java/badge.svg?branch=master)](https://coveralls.io/github/tarantool/tarantool-java?branch=master)
 
 To get the Java connector for Tarantool 1.6.9, visit
 [this GitHub page](https://github.com/tarantool/tarantool-java/tree/connector-1.6.9).
 
 ## Table of contents
 * [Getting started](#getting-started)
+* [Cluster support](#cluster-support)
 * [Where to get help](#where-to-get-help)
 
 ## Getting started
 
-1. Add a dependency to your `pom.xml` file.
+1. Add a dependency to your `pom.xml` file:
 
 ```xml
 <dependency>
   <groupId>org.tarantool</groupId>
   <artifactId>connector</artifactId>
-  <version>1.7.4</version>
+  <version>1.9.1</version>
 </dependency>
 ```
 
-2. Configure `TarantoolClientConfig`.
+2. Configure `TarantoolClientConfig`:
 
 ```java
 TarantoolClientConfig config = new TarantoolClientConfig();
 config.username = "test";
 config.password = "test";
 ```
 
-3. Implement your `SocketChannelProvider`.
-   It should return a connected `SocketChannel`.
+3. Create a client:
 
 ```java
-SocketChannelProvider socketChannelProvider = new SocketChannelProvider() {
-           @Override
-           public SocketChannel get(int retryNumber, Throwable lastError) {
-               if (lastError != null) {
-                   lastError.printStackTrace(System.out);
-               }
-               try {
-                   return SocketChannel.open(new InetSocketAddress("localhost", 3301));
-               } catch (IOException e) {
-                   throw new IllegalStateException(e);
-               }
-           }
-       };
+TarantoolClient client = new TarantoolClientImpl("host:3301", config);
+```
+
+using `TarantoolClientImpl(String, TarantoolClientConfig)` is equivalent to:
+
+```java
+SocketChannelProvider socketChannelProvider = new SingleSocketChannelProviderImpl("host:3301")
+TarantoolClient client = new TarantoolClientImpl(socketChannelProvider, config);
 ```
 
-Here you could also implement some reconnection or fallback policy.
-Remember that `TarantoolClient` adopts a
-[fail-fast](https://en.wikipedia.org/wiki/Fail-fast) policy
-when a client is not connected.
+You could implement your own `SocketChannelProvider`. It should return 
+a connected `SocketChannel`. Feel free to implement `get(int retryNumber, Throwable lastError)`
+using your appropriate strategy to obtain the channel. The strategy can take into
+account current attempt number (retryNumber) and the last transient error occurred on
+the previous attempt.
 
-The `TarantoolClient` will stop functioning if your implementation of a socket
-channel provider raises an exception or returns a null. You will need a new
-instance of client to recover. Hence, you should only throw in case you have
-met unrecoverable error.
+The `TarantoolClient` will be closed if your implementation of a socket
+channel provider raises exceptions. However, throwing a `SocketProviderTransientException`
+or returning `null` value are handled by the client as recoverable errors. In these cases,
+the client will make next attempt to obtain the socket channel. Otherwise, you will need
+a new instance of client to recover. Hence, you should only throw an error different
+to `SocketProviderTransientException` in case you have met unrecoverable error.
 
-Below is an example of `SocketChannelProvider` implementation that handles short
-tarantool restarts.
+Below is an example of `SocketChannelProvider` implementation that tries
+to connect no more than 3 times, two seconds for each attempt at max.
 
 ```java
 SocketChannelProvider socketChannelProvider = new SocketChannelProvider() {
     @Override
     public SocketChannel get(int retryNumber, Throwable lastError) {
-        long deadline = System.currentTimeMillis() + RESTART_TIMEOUT;
-        while (!Thread.currentThread().isInterrupted()) {
-            try {
-                return SocketChannel.open(new InetSocketAddress("localhost", 3301));
-            } catch (IOException e) {
-                if (deadline < System.currentTimeMillis())
-                    throw new RuntimeException(e);
-                try {
-                    Thread.sleep(100);
-                } catch (InterruptedException ignored) {
-                    Thread.currentThread().interrupt();
-                }
+        if (retryNumber > 3) {
+            throw new RuntimeException("Too many attempts");
+        }
+        SocketChannel channel = null;
+        try {
+            channel = SocketChannel.open();
+            channel.socket().connect(new InetSocketAddress("localhost", 3301), 2000);
+            return channel;
+        } catch (IOException e) {
+            if (channel != null) {
+                 try {
+                     channel.close();
+                 } catch (IOException ignored) { }
             }
+            throw new SocketProviderTransientException("Couldn't connect to server", e);
         }
-        throw new RuntimeException(new TimeoutException("Connect timed out."));
     }
 };
 ```
 
-4. Create a client.
+Same behaviour can be achieved using built-in `SingleSocketChannelProviderImpl`:
 
 ```java
+TarantoolClientConfig config = new TarantoolClientConfig();
+config.connectionTimeout = 2_000; // two seconds timeout per attempt
+config.retryCount = 3;            // three attempts at max
+
+SocketChannelProvider socketChannelProvider = new SingleSocketChannelProviderImpl("localhost:3301")
 TarantoolClient client = new TarantoolClientImpl(socketChannelProvider, config);
 ```
 
+`SingleSocketChannelProviderImpl` implements `ConfigurableSocketChannelProvider` that
+makes possible for the client to configure a socket provider.
+
 > **Notes:**
 > * `TarantoolClient` is thread-safe and asynchronous, so you should use one
 >   client inside the whole application.
@@ -156,6 +161,117 @@ System.out.println(template.query("select * from hello_world where hello=:id", C
 
 For more implementation details, see [API documentation](http://tarantool.github.io/tarantool-java/apidocs/index.html).
 
+## Cluster support
+
+To be more fault-tolerant the connector provides cluster extensions. In
+particular `TarantoolClusterClient` and built-in `RoundRobinSocketProviderImpl`
+used as a default `SocketProvider` implementation. When currently connected
+instance is down then the client will try to reconnect to the first available
+instance using strategy defined in a socket provider. You need to supply
+a list of nodes which will be used by the cluster client to provide such
+ability. Also you can prefer to use a [discovery mechanism](#auto-discovery)
+in order to dynamically fetch and apply the node list.
+
+### The RoundRobinSocketProviderImpl class
+
+This cluster-aware provider uses addresses pool to connect to DB server.
+The provider picks up next address in order the addresses were passed.
+
+Similar to `SingleSocketChannelProviderImpl` this RR provider also
+relies on two options from the config: `TarantoolClientConfig.connectionTimeout`
+and `TarantoolClientConfig.retryCount` but in a bit different way.
+The latter option says how many times the provider should try to establish a
+connection to _one instance_ before failing an attempt. The provider requires
+positive retry count to work properly. The socket timeout is used to limit
+an interval between connections attempts per instance. In other words, the provider
+follows a pattern _connection should succeed after N attempts with M interval between
+them at max_. 
+
+### Basic cluster client usage
+
+1. Configure `TarantoolClusterClientConfig`:
+
+```java
+TarantoolClusterClientConfig config = new TarantoolClusterClientConfig();
+// fill other settings
+config.operationExpiryTimeMillis = 2000;
+config.executor = Executors.newSingleThreadExecutor();
+```
+
+2. Create an instance of `TarantoolClusterClientImpl`. You need to provide
+an initial list of nodes:
+
+```java
+String[] nodes = new String[] { "myHost1:3301", "myHost2:3302", "myHost3:3301" };
+TarantoolClusterClient client = new TarantoolClusterClient(config, nodes);
+``` 
+
+3. Work with the client using same API as defined in `TarantoolClient`:
+
+```java
+client.syncOps().insert(23, Arrays.asList(1, 1));
+```
+
+### Auto-discovery
+
+Auto-discovery feature allows a cluster client to fetch addresses of 
+cluster nodes to reflect changes related to the cluster topology. To achieve
+this you have to create a Lua function on the server side which returns 
+a single array result. Client periodically polls the server to obtain a 
+fresh list and apply it if its content changes.
+
+1. On the server side create a function which returns nodes:
+
+```bash
+tarantool> function get_cluster_nodes() return { 'host1:3301', 'host2:3302', 'host3:3301' } end
+```
+
+You need to pay attention to a function contract we are currently supporting:
+* The client never passes any arguments to a discovery function.
+* A discovery function _should_ return a single result of strings (i.e. single 
+  string `return 'host:3301'` or array of strings `return {'host1:3301', 'host2:3301'}`).
+* A discovery function _may_ return multi-results but the client takes
+  into account only first of them (i.e. `return {'host:3301'}, discovery_delay`, where 
+  the second result is unused). Even more, any extra results __are reserved__ by the client
+  in order to extend its contract with a backward compatibility.
+* A discovery function _should NOT_ return no results, empty result, wrong type result,
+  and Lua errors. The client discards such kinds of results but it does not affect the discovery
+  process for next scheduled tasks. 
+
+2. On the client side configure discovery settings in `TarantoolClusterClientConfig`:
+
+```java
+TarantoolClusterClientConfig config = new TarantoolClusterClientConfig();
+// fill other settings
+config.clusterDiscoveryEntryFunction = "get_cluster_nodes"; // discovery function used to fetch nodes 
+config.clusterDiscoveryDelayMillis = 60_000;                // how often client polls the discovery server  
+```
+
+3. Create a client using the config made above.
+
+```java
+TarantoolClusterClient client = new TarantoolClusterClient(config);
+client.syncOps().insert(45, Arrays.asList(1, 1));
+```
+
+### Auto-discovery caveats
+
+* You need to set _not empty_ value to `clusterDiscoveryEntryFunction` to enable auto-discovery feature.
+* There are only two cases when a discovery task runs: just after initialization of the cluster
+  client and a periodical scheduler timeout defined in `TarantoolClusterClientConfig.clusterDiscoveryDelayMillis`. 
+* A discovery task always uses an active client connection to get the nodes list.
+  It's in your responsibility to provide a function availability as well as a consistent
+  nodes list on all instances you initially set or obtain from the task.
+* If some error occurs while a discovery task is running then this task
+  will be aborted without any after-effects for next task executions. These cases, for instance, are 
+  a wrong function result (see discovery function contract) or a broken connection. 
+  There is an exception if the client is closed then discovery process will stop permanently.
+* It's possible to obtain a list which does NOT contain the node we are currently
+  connected to. It leads the client to try to reconnect to another node from the 
+  new list. It may take some time to graceful disconnect from the current node.
+  The client does its best to catch the moment when there are no pending responses
+  and perform a reconnection.  
+
 ## Where to get help
 
 Got problems or questions? Post them on
@@ -164,7 +280,15 @@ Got problems or questions? Post them on
 base for possible answers and solutions.
 
 ## Building
-To run tests
-```
+
+To run unit tests use:
+
+```bash
 ./mvnw clean test
+``` 
+
+To run integration tests use:
+
+```bash
+./mvnw clean verify
 ```