Skip to content

Upgrade to Fabric8's 5.0 client and Quarkus 1.11.0.Final #288

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Jan 18, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
c55d335
feat: initial upgrade of client to 5.0.0 and quarkus to 1.11.0.CR1
metacosm Jan 7, 2021
6eb2467
feat: add logging for tests
metacosm Jan 8, 2021
990a1f2
fix: add Kind annotation to match CRD descriptors
metacosm Jan 8, 2021
80d65bf
fix: test CRs are also Namespaced so must be marked as such
metacosm Jan 8, 2021
8139479
fix: match kind with CRD manifest
metacosm Jan 8, 2021
de1c901
fix: add Plural annotation to match CRD manifest
metacosm Jan 8, 2021
2ed67c9
refactor!: remove unneeded registration of specific CR client
metacosm Jan 13, 2021
61b44d1
fix: format
metacosm Jan 13, 2021
e9a2a4a
fix: remove getDoneableClass on JSON wrapper
metacosm Jan 14, 2021
38d419a
feat: update to Quarkus 1.11.0.Final
metacosm Jan 14, 2021
ae27837
feat: skip registration of controllers whose configuration isn't found
metacosm Jan 15, 2021
7dc1c23
fix: unwrap CDI proxies to get the proper class name
metacosm Jan 15, 2021
bb0a6d8
feat: Application-scope controllers automatically if not already scoped
metacosm Jan 15, 2021
8657ee3
docs: update samples
metacosm Jan 15, 2021
e104e58
refactor(quarkus): remove extra feature production step
metacosm Jan 15, 2021
682ceb7
feat: check that CR class is properly annotated
metacosm Jan 15, 2021
5e0c0af
feat: log information about processed controller
metacosm Jan 15, 2021
6a34a87
docs: add upgrade information
metacosm Jan 15, 2021
100b149
feat: make CR Namespaced
metacosm Jan 15, 2021
b348ba6
fix(docs): WebServer implements Namespaced now
metacosm Jan 15, 2021
579662e
fix: remove isClusterScoped method from Controller
metacosm Jan 18, 2021
22fcef3
chore(build): upgrade to version 1.7.0-SNAPSHOT
metacosm Jan 18, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 38 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ Build Kubernetes Operators in Java without hassle. Inspired by [operator-sdk](ht

#### Features
* Framework for handling Kubernetes API events
* Registering Custom Resource watches
* Automatic registration of Custom Resource watches
* Retry action on failure
* Smart event scheduling (only handle latest event for the same resource)
* Smart event scheduling (only handle the latest event for the same resource)

Check out this [blog post](https://blog.container-solutions.com/a-deep-dive-into-the-java-operator-sdk)
about the non-trivial yet common problems needed to be solved for every operator.
Expand All @@ -44,6 +44,37 @@ about the non-trivial yet common problems needed to be solved for every operator
You can (will) find detailed documentation [here](docs/DOCS.md).
Note that these docs are currently in progress.

> :warning: 1.7.0 Upgrade
> The 1.7.0 upgrade comes with big changes due to the update to the 5.0.0 version of the fabric8
> Kubernetes client. While this should improve the user experience quite nicely, there are a couple
> of things to be aware of when upgrading from a previous version as detailed below.

##### Overview of the 1.7.0 changes

- `Doneable` classes have been removed along with all the involved complexity
- `Controller` annotation has been simplified: the `crdName` field has been removed as that value is
computed from the associated custom resource implementation
- Custom Resource implementation classes now need to be annotated with `Group` and `Version`
annotations so that they can be identified properly. Optionally, they can also be annotated with
`Kind` (if the name of the implementation class doesn't match the desired kind) and `Plural` if
the plural version cannot be automatically computed (or the default computed version doesn't match
your expectations).
- The `CustomResource` class that needs to be extended is now parameterized with spec and status
types, so you can have an empty default implementation that does what you'd expect. If you don't
need a status, using `Void` for the associated type should work.
- Custom Resources that are namespace-scoped need to implement the `Namespaced` interface so that
the client can generate the proper URLs. This means, in particular, that `CustomResource`
implementations that do **not** implement `Namespaced` are considered cluster-scoped. As a
consequence, the `isClusterScoped` method/field has been removed from the appropriate
classes (`Controller` annotation, in particular) as this is now inferred from the `CustomResource`
type associated with your `Controller`.

Many of these changes might not be immediately apparent but will result in `404` errors when
connecting to the cluster. Please check that the Custom Resource implementations are properly
annotated and that the value corresponds to your CRD manifest. If the namespace appear to be missing
in your request URL, don't forget that namespace-scoped Custom Resources need to implement
the `Namescaped` interface.

#### Usage

We have several sample Operators under the [samples](samples) directory:
Expand All @@ -52,6 +83,7 @@ Implemented with and without Spring Boot support. The two samples share the comm
* *webserver*: More realistic example creating an nginx webserver from a Custom Resource containing html code.
* *mysql-schema*: Operator managing schemas in a MySQL database
* *spring-boot-plain/auto-config*: Samples showing integration with Spring Boot.
* *quarkus*: Minimal application showing automatic configuration / injection of Operator / Controllers.

Add [dependency](https://search.maven.org/search?q=a:operator-framework) to your project with Maven:

Expand Down Expand Up @@ -89,7 +121,7 @@ public class Runner {
The Controller implements the business logic and describes all the classes needed to handle the CRD.

```java
@Controller(crdName = "webservers.sample.javaoperatorsdk")
@Controller
public class WebServerController implements ResourceController<WebServer> {

@Override
Expand All @@ -110,28 +142,9 @@ public class WebServerController implements ResourceController<WebServer> {
A sample custom resource POJO representation

```java
public class WebServer extends CustomResource {

private WebServerSpec spec;

private WebServerStatus status;

public WebServerSpec getSpec() {
return spec;
}

public void setSpec(WebServerSpec spec) {
this.spec = spec;
}

public WebServerStatus getStatus() {
return status;
}

public void setStatus(WebServerStatus status) {
this.status = status;
}
}
@Group("sample.javaoperatorsdk")
@Version("v1")
public class WebServer extends CustomResource<WebServerSpec, WebServerStatus> implements Namespaced {}

public class WebServerSpec {

Expand Down Expand Up @@ -183,7 +196,6 @@ public class QuarkusOperator implements QuarkusApplication {
public int run(String... args) throws Exception {
final var config = configuration.getConfigurationFor(new CustomServiceController(client));
System.out.println("CR class: " + config.getCustomResourceClass());
System.out.println("Doneable class = " + config.getDoneableClass());

Quarkus.waitForExit();
return 0;
Expand Down
2 changes: 1 addition & 1 deletion operator-framework-core/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<parent>
<groupId>io.javaoperatorsdk</groupId>
<artifactId>java-operator-sdk</artifactId>
<version>1.6.4-SNAPSHOT</version>
<version>1.7.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,8 @@
package io.javaoperatorsdk.operator;

import io.fabric8.kubernetes.api.model.apiextensions.v1beta1.CustomResourceDefinition;
import io.fabric8.kubernetes.client.CustomResource;
import io.fabric8.kubernetes.client.CustomResourceDoneable;
import io.fabric8.kubernetes.client.CustomResourceList;
import io.fabric8.kubernetes.client.KubernetesClient;
import io.fabric8.kubernetes.client.dsl.MixedOperation;
import io.fabric8.kubernetes.client.dsl.base.CustomResourceDefinitionContext;
import io.fabric8.kubernetes.client.dsl.internal.CustomResourceOperationsImpl;
import io.fabric8.kubernetes.internal.KubernetesDeserializer;
import io.javaoperatorsdk.operator.api.ResourceController;
import io.javaoperatorsdk.operator.api.config.ConfigurationService;
import io.javaoperatorsdk.operator.processing.CustomResourceCache;
Expand All @@ -19,8 +13,6 @@
import io.javaoperatorsdk.operator.processing.retry.GenericRetry;
import io.javaoperatorsdk.operator.processing.retry.Retry;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand All @@ -30,8 +22,6 @@ public class Operator {
private static final Logger log = LoggerFactory.getLogger(Operator.class);
private final KubernetesClient k8sClient;
private final ConfigurationService configurationService;
private Map<Class<? extends CustomResource>, CustomResourceOperationsImpl> customResourceClients =
new HashMap<>();

public Operator(KubernetesClient k8sClient, ConfigurationService configurationService) {
this.k8sClient = k8sClient;
Expand All @@ -41,9 +31,18 @@ public Operator(KubernetesClient k8sClient, ConfigurationService configurationSe
public <R extends CustomResource> void register(ResourceController<R> controller)
throws OperatorException {
final var configuration = configurationService.getConfigurationFor(controller);
final var retry = GenericRetry.fromConfiguration(configuration.getRetryConfiguration());
final var targetNamespaces = configuration.getNamespaces().toArray(new String[] {});
registerController(controller, configuration.watchAllNamespaces(), retry, targetNamespaces);
if (configuration == null) {
log.warn(
"Skipping registration of {} controller named {} because its configuration cannot be found.\n"
+ "Known controllers are: {}",
controller.getClass().getCanonicalName(),
ControllerUtils.getNameFor(controller),
configurationService.getKnownControllerNames());
} else {
final var retry = GenericRetry.fromConfiguration(configuration.getRetryConfiguration());
final var targetNamespaces = configuration.getNamespaces().toArray(new String[] {});
registerController(controller, configuration.watchAllNamespaces(), retry, targetNamespaces);
}
}

public <R extends CustomResource> void registerControllerForAllNamespaces(
Expand Down Expand Up @@ -76,12 +75,8 @@ private <R extends CustomResource> void registerController(
throws OperatorException {
final var configuration = configurationService.getConfigurationFor(controller);
Class<R> resClass = configuration.getCustomResourceClass();
CustomResourceDefinitionContext crd = getCustomResourceDefinitionForController(controller);
KubernetesDeserializer.registerCustomKind(crd.getVersion(), crd.getKind(), resClass);
String finalizer = configuration.getFinalizer();
MixedOperation client =
k8sClient.customResources(
crd, resClass, CustomResourceList.class, configuration.getDoneableClass());
MixedOperation client = k8sClient.customResources(resClass);
EventDispatcher eventDispatcher =
new EventDispatcher(
controller, finalizer, new EventDispatcher.CustomResourceFacade(client));
Expand All @@ -95,8 +90,6 @@ private <R extends CustomResource> void registerController(
defaultEventHandler.setEventSourceManager(eventSourceManager);
eventDispatcher.setEventSourceManager(eventSourceManager);

customResourceClients.put(resClass, (CustomResourceOperationsImpl) client);

controller.init(eventSourceManager);
CustomResourceEventSource customResourceEventSource =
createCustomResourceEventSource(
Expand Down Expand Up @@ -137,30 +130,4 @@ private CustomResourceEventSource createCustomResourceEventSource(

return customResourceEventSource;
}

private CustomResourceDefinitionContext getCustomResourceDefinitionForController(
ResourceController controller) {
final var crdName = configurationService.getConfigurationFor(controller).getCRDName();
CustomResourceDefinition customResourceDefinition =
k8sClient.customResourceDefinitions().withName(crdName).get();
if (customResourceDefinition == null) {
throw new OperatorException("Cannot find Custom Resource Definition with name: " + crdName);
}
CustomResourceDefinitionContext context =
CustomResourceDefinitionContext.fromCrd(customResourceDefinition);
return context;
}

public Map<Class<? extends CustomResource>, CustomResourceOperationsImpl>
getCustomResourceClients() {
return customResourceClients;
}

public <
T extends CustomResource,
L extends CustomResourceList<T>,
D extends CustomResourceDoneable<T>>
CustomResourceOperationsImpl<T, L, D> getCustomResourceClients(Class<T> customResourceClass) {
return customResourceClients.get(customResourceClass);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,6 @@

String NULL = "";

String crdName();

String name() default NULL;

/**
Expand All @@ -28,7 +26,15 @@
*/
boolean generationAwareEventProcessing() default true;

boolean isClusterScoped() default false;

/**
* Specified which namespaces this Controller monitors for custom resources events. If no
* namespace is specified then the controller will monitor the namespace it is deployed in (or the
* namespace to which the Kubernetes client is connected to). To specify that the controller needs
* to monitor all namespaces, add {@link
* io.javaoperatorsdk.operator.api.config.ControllerConfiguration#WATCH_ALL_NAMESPACES_MARKER} to
* this field.
*
* @return the list of namespaces this controller monitors
*/
String[] namespaces() default {};
}
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import io.javaoperatorsdk.operator.ControllerUtils;
import io.javaoperatorsdk.operator.api.ResourceController;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;

public abstract class AbstractConfigurationService implements ConfigurationService {
Expand Down Expand Up @@ -35,4 +36,9 @@ public <R extends CustomResource> ControllerConfiguration<R> getConfigurationFor
ResourceController<R> controller) {
return configurations.get(ControllerUtils.getNameFor(controller));
}

@Override
public Set<String> getKnownControllerNames() {
return configurations.keySet();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import io.fabric8.kubernetes.client.Config;
import io.fabric8.kubernetes.client.CustomResource;
import io.javaoperatorsdk.operator.api.ResourceController;
import java.util.Set;

public interface ConfigurationService {

Expand All @@ -12,4 +13,6 @@ <R extends CustomResource> ControllerConfiguration<R> getConfigurationFor(
default Config getClientConfiguration() {
return Config.autoConfigure(null);
}

Set<String> getKnownControllerNames();
}
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package io.javaoperatorsdk.operator.api.config;

import io.fabric8.kubernetes.client.CustomResource;
import io.fabric8.kubernetes.client.CustomResourceDoneable;
import java.util.Collections;
import java.util.Set;

Expand All @@ -19,14 +18,8 @@ public interface ControllerConfiguration<R extends CustomResource> {

Class<R> getCustomResourceClass();

Class<? extends CustomResourceDoneable<R>> getDoneableClass();

String getAssociatedControllerClassName();

default boolean isClusterScoped() {
return false;
}

default Set<String> getNamespaces() {
return Collections.emptySet();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -191,10 +191,9 @@ private void addFinalizerIfNotPresent(CustomResource resource) {
// created to support unit testing
public static class CustomResourceFacade {

private final MixedOperation<?, ?, ?, Resource<CustomResource, ?>> resourceOperation;
private final MixedOperation<?, ?, Resource<CustomResource>> resourceOperation;

public CustomResourceFacade(
MixedOperation<?, ?, ?, Resource<CustomResource, ?>> resourceOperation) {
public CustomResourceFacade(MixedOperation<?, ?, Resource<CustomResource>> resourceOperation) {
this.resourceOperation = resourceOperation;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,10 @@
import static io.javaoperatorsdk.operator.processing.KubernetesResourceUtils.getUID;
import static io.javaoperatorsdk.operator.processing.KubernetesResourceUtils.getVersion;
import static io.javaoperatorsdk.operator.processing.KubernetesResourceUtils.markedForDeletion;
import static java.net.HttpURLConnection.HTTP_GONE;

import io.fabric8.kubernetes.client.CustomResource;
import io.fabric8.kubernetes.client.KubernetesClientException;
import io.fabric8.kubernetes.client.Watcher;
import io.fabric8.kubernetes.client.WatcherException;
import io.fabric8.kubernetes.client.dsl.MixedOperation;
import io.fabric8.kubernetes.client.dsl.internal.CustomResourceOperationsImpl;
import io.javaoperatorsdk.operator.ControllerUtils;
Expand Down Expand Up @@ -151,11 +150,11 @@ public void eventSourceDeRegisteredForResource(String customResourceUid) {
}

@Override
public void onClose(KubernetesClientException e) {
public void onClose(WatcherException e) {
if (e == null) {
return;
}
if (e.getCode() == HTTP_GONE) {
if (e.isHttpGone()) {
log.warn("Received error for watch, will try to reconnect.", e);
registerWatch();
} else {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,38 +1,10 @@
package io.javaoperatorsdk.operator.sample.simple;

import io.fabric8.kubernetes.client.CustomResource;
import io.fabric8.kubernetes.model.annotation.Group;
import io.fabric8.kubernetes.model.annotation.Version;

public class TestCustomResource extends CustomResource {

private TestCustomResourceSpec spec;

private TestCustomResourceStatus status;

public TestCustomResourceSpec getSpec() {
return spec;
}

public void setSpec(TestCustomResourceSpec spec) {
this.spec = spec;
}

public TestCustomResourceStatus getStatus() {
return status;
}

public void setStatus(TestCustomResourceStatus status) {
this.status = status;
}

@Override
public String toString() {
return "TestCustomResource{"
+ "spec="
+ spec
+ ", status="
+ status
+ ", extendedFrom="
+ super.toString()
+ '}';
}
}
@Group("sample.javaoperatorsdk.io")
@Version("v1")
public class TestCustomResource
extends CustomResource<TestCustomResourceSpec, TestCustomResourceStatus> {}
Loading