On this page
Kubernetes
This project eases Kubernetes integration with Micronaut by providing the following two different implementation of kubernetes client:
-
micronaut-kubernetes-clientmodule provides an implementation which integrates with the official Kubernetes Java SDK client -
micronaut-kubernetes-client-openapimodule provides an in-house client implementation which uses Micronaut Netty HTTP Client and generated apis and modules from the Kubernetes Java SDK OpenApi Spec
The project adds support for the following features:
-
Service Discovery
-
Configuration client for config maps and secrets
-
Kubernetes blocking and non-blocking clients
To use the BUILD-SNAPSHOT version of this library, check the
documentation to use snapshots.
Namespace configuration
When a Micronaut application with this module is running within a Pod in a Kubernetes cluster, it will
infer automatically the namespace it’s running from by reading it from the service account secret (which will be
provisioned at /var/run/secrets/kubernetes.io/serviceaccount/namespace).
However, the namespace can still be overridden via configuration in bootstrap configuration files:
kubernetes.client.namespace=other-namespaceMicronaut Kubernetes 8.0.0
-
The Kubernetes Configuration Client support has been deprecated and replaced by Configuration Import
-
RxJava2 support has been removed
Micronaut Kubernetes 7.1.0
-
Kubernetes Operator support for Micronaut Kubernetes Client OpenApi
Micronaut Kubernetes 7.0.0
-
Kubernetes Watcher support for Micronaut Kubernetes Client OpenApi
-
Kubernetes Informer support for Micronaut Kubernetes Client OpenApi
-
Kubernetes Service Discovery and Configuration Client support for Micronaut Kubernetes Client OpenApi
Micronaut Kubernetes 6.2.1
-
Reactive support for Micronaut Kubernetes Client OpenApi
Micronaut Kubernetes 6.2.0
-
Implementation of in-house kubernetes client: Micronaut Kubernetes Client OpenApi
Micronaut Kubernetes 3.0.0
-
Official K8s JAVA SDK client with reactive support
For this project, you can find a list of releases (with release notes) here:
The micronaut-kubernetes-client module gives you the ability to use official Kubernetes Java SDK apis objects as a regular Micronaut beans.
The complete list of available beans is declared in the Apis#values annotation value.
First you need add a dependency on the micronaut-kubernetes-client module:
implementation("io.micronaut.kubernetes:micronaut-kubernetes-client")Then you can simply use Micronaut injection to get configured apis object from package io.kubernetes.client.openapi.apis :
import io.kubernetes.client.openapi.ApiException;
import io.kubernetes.client.openapi.apis.CoreV1Api;
import io.kubernetes.client.openapi.models.V1PodList;
import jakarta.inject.Singleton;
@Singleton
public class MyService {
private final CoreV1Api coreV1Api;
public MyService(CoreV1Api coreV1Api) {
this.coreV1Api = coreV1Api;
}
public void myMethod(String namespace) throws ApiException {
V1PodList v1PodList = coreV1Api.listNamespacedPod(namespace).execute();
}
}Authentication
The Kubernetes client source of authentication options is automatically detected by the ClientBuilder#standard(), specifically:
Creates a builder which is pre-configured in the following way
-
If
$KUBECONFIGis defined, use that config file. -
If
$HOME/.kube/configcan be found, use that. -
If the in-cluster service account can be found, assume in cluster config.
-
Default to
localhost:8080as a last resort.
Also for specific cases you can update the authentication configuration options by using kubernetes.client properties listed below:
Name |
Description |
basePath |
Kubernetes API base path. Example: |
caPath |
CA file path. |
tokenPath |
Token file path. |
kubeConfigPath |
Kube config file path. |
verifySsl |
Boolean if the api should verify ssl. Default: |
Reactive Support
In addition to the official Kubernetes Java SDK Async clients, this module provides clients that use RxJava or Reactor to allow reactive programming with Micronaut for each Api.
RxJava 3 Reactive Support
For RxJava 3 add the following dependency:
implementation("io.micronaut.kubernetes:micronaut-kubernetes-client-rxjava3")The module contains all official Kubernetes API beans in format <ApiClassName>RxClient,
for example the CoreV1Api class is injected as CoreV1ApiRxClient.
import io.kubernetes.client.openapi.ApiException;
import io.kubernetes.client.openapi.models.V1PodList;
import io.micronaut.kubernetes.client.rxjava3.CoreV1ApiRxClient;
import io.reactivex.Single;
import jakarta.inject.Singleton;
@Singleton
public class MyService {
private final CoreV1ApiRxClient coreV1ApiRxClient;
public MyService(CoreV1ApiRxClient coreV1ApiRxClient) {
this.coreV1ApiRxClient = coreV1ApiRxClient;
}
public void myMethod(String namespace) {
Single<V1PodList> v1PodList = coreV1ApiRxClient.listNamespacedPod(namespace).execute();
}
}Reactor Reactive Support
For Reactor add the following dependency:
implementation("io.micronaut.kubernetes:micronaut-kubernetes-client-reactor")The module contains all official Kubernetes API beans in format <ApiClassName>ReactiveClient,
for example the CoreV1Api class is injected as CoreV1ApiReactiveClient.
import io.kubernetes.client.openapi.ApiException;
import io.kubernetes.client.openapi.models.V1PodList;
import io.micronaut.kubernetes.client.reactive.CoreV1ApiReactiveClient;
import reactor.core.publisher.Mono;
import jakarta.inject.Singleton;
@Singleton
public class MyService {
private final CoreV1ApiReactiveClient coreV1ApiReactiveClient;
public MyService(CoreV1ApiReactiveClient coreV1ApiReactiveClient) {
this.coreV1ApiReactiveClient = coreV1ApiReactiveClient;
}
public void myMethod(String namespace) {
Mono<V1PodList> v1PodList = coreV1ApiReactiveClient.listNamespacedPod(namespace).execute();
}
}Advanced Configuration
For advanced configuration options of ApiClient that are not suitable to provide via application.yml, you can declare a BeanCreatedEventListener bean that listens for ApiClient bean creation, and apply any further customisation to OkHttpClient there:
@Singleton
public class ApiClientListener implements BeanCreatedEventListener<ApiClient> {
@Override
public ApiClient onCreated(BeanCreatedEvent<ApiClient> event) {
ApiClient apiClient = event.getBean();
OkHttpClient okHttpClient = apiClient.getHttpClient().newBuilder()
.readTimeout(5345, TimeUnit.MILLISECONDS)
.build();
apiClient.setHttpClient(okHttpClient);
return apiClient;
}
}The Service Discovery module allows Micronaut HTTP clients to discover Kubernetes services.
To get started, you need to declare the following dependency:
implementation("io.micronaut.kubernetes:micronaut-kubernetes-discovery-client")By default in any client you can use as Service ID the Kubernetes Endpoints name generated by a Kubernetes Service for the configured namespace.
Consider the following Kubernetes service definition:
my-service.ymlkind: Service
apiVersion: v1
metadata:
name: my-service
spec:
selector:
app: MyApp
ports:
- protocol: TCP
port: 80
targetPort: 9376This specification will create a new Service object named my-service, as well as an Endpoints object also named my-service.
In your HTTP client, you can use my-service as Service ID: @Client("my-service").
Note that service discovery is enabled by default in Micronaut. To disable it, set kubernetes.client.discovery.enabled
to false.
Service specific client configuration
Kubernetes Service is a complex resource that can handle various use cases by providing specific configuration. For this Micronaut Kubernetes supports a manual service discovery configuration per Service http client that allows you to configure custom:
| Key | Description |
|---|---|
|
name of the resource in Kubernetes in case it is different than the Service ID |
|
namespace of the resource in case it’s different than the configured namespace |
|
port name in case the target resource is a Multi-Port Service |
|
service specific discovery mode in case it’s different than the globally configured discovery mode |
Examples of service configurations
Multi-port service
For the following Multi-Port Service:
my-service.ymlapiVersion: v1
kind: Service
metadata:
name: my-service
spec:
selector:
app: MyApp
ports:
- name: http
protocol: TCP
port: 80
targetPort: 9376
- name: https
protocol: TCP
port: 443
targetPort: 9377the manual service configuration for http port will be:
kubernetes.client.discovery.services.my-service.port=httpHeadless service with selector
For the following Headless service with selector:
my-service.ymlapiVersion: v1
kind: Service
metadata:
name: my-service
spec:
clusterIP: None
selector:
app: MyApp
ports:
- name: http
protocol: TCP
port: 80
targetPort: 9376the manual service configuration will be:
kubernetes.client.discovery.services.my-service.mode=endpointExternalName service type
For the following ExternalName service:
my-service.ymlapiVersion: v1
kind: Service
metadata:
name: my-service
namespace: prod
spec:
type: ExternalName
externalName: launch.micronaut.iothe manual service configuration will be:
kubernetes.client.discovery.services.my-service.mode=serviceService discovery modes
Service discovery mode is a mechanism that allows to support different strategies for the actual service discovery in Kubernetes by implementing KubernetesServiceInstanceProvider interface.
Currently Micronaut Kubernetes implements two discovery modes:
-
endpointmode uses the KubernetesEndpointsAPI for the service discovery. Note that the service load balancing is handled by Micronaut application. -
servicemode uses the KubernetesServiceAPI for the service discovery. Theservicemode extracts the serviceClusterIPaddress from the Service status.
Both discovery modes are using the metadata.name for the Service ID identificator.
The discovery mode can be configured globally for all Service IDs or per service.
Note that endpoint is the default global discovery mode. That can be overridden via configuration in bootstrap.yml:
kubernetes.client.discovery.mode=serviceWatching for changes
Both discovery modes support watching for changes of their respective resources. To enable it, set kubernetes.client.discovery.mode-configuration.endpoint.watch.enabled to true for the endpoint mode. For the service mode set kubernetes.client.discovery.mode-configuration.service.watch.enabled to true.
Kubernetes API authentication
Micronaut authenticates to the Kubernetes API using the token mounted at /var/run/secrets/kubernetes.io/serviceaccount/token.
Note that by default, the service account used may only have permissions over the kube-system namespace. The service discovery
functionality requires some additional read permissions. Refer to the Kubernetes documentation for more information
about Role-based access control (RBAC).
One of the options is to create the following Role and RoleBinding (make sure to apply them to the service account used, if not default):
auth.ymlkind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: service-discoverer
namespace: micronaut-kubernetes
rules:
- apiGroups: [""]
resources: ["services", "endpoints", "configmaps", "secrets", "pods"]
verbs: ["get", "watch", "list"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: default-service-discoverer
namespace: micronaut-kubernetes
subjects:
- kind: ServiceAccount
name: default
namespace: micronaut-kubernetes
roleRef:
kind: Role
name: service-discoverer
apiGroup: rbac.authorization.k8s.io|
Note
|
In Google Cloud’s Kubernetes Engine, in order to create the above, you must grant your user the ability to create roles in Kubernetes by running the following command: |
kubectl create clusterrolebinding cluster-admin-binding --clusterrole cluster-admin --user yourGoogleAccount@gmail.com
Connecting to services using HTTPS
There are three ways for this library to determine whether a service should be connected to using SSL (the following
examples assume there is a Deployment named secure-deployment).
Using https as port name
my-service.ymlapiVersion: v1
kind: Service
metadata:
name: secure-service-port-name
spec:
selector:
app: secure-deployment
type: NodePort
ports:
- port: 1234
protocol: TCP
name: httpsUsing a port ending in 443
Port numbers like 443, 8443, etc. will match.
my-service.ymlapiVersion: v1
kind: Service
metadata:
name: secure-service-port-number
spec:
selector:
app: secure-deployment
type: NodePort
ports:
- port: 443
protocol: TCPUsing labels
Set a label named secure with value true to have the client use HTTPS.
my-service.ymlapiVersion: v1
kind: Service
metadata:
name: secure-service-labels
labels:
secure: "true"
spec:
selector:
app: secure-deployment
type: NodePort
ports:
- port: 1234
protocol: TCPService filtering
You can filter the services discovered by using kubernetes.client.discovery.includes or
kubernetes.client.discovery.excludes:
kubernetes.client.discovery.includes[0]=my-service
kubernetes.client.discovery.includes[1]=other-serviceOr:
kubernetes.client.discovery.excludes=not-this-serviceIn addition to that, Kubernetes labels can be used to better match the services that should be available for service discovery:
kubernetes.client.discovery.labels[0].app=my-app
kubernetes.client.discovery.labels[1].env=prodNote that the filtering is not applied on manually configured service configurations.
Service discovery guides
See the following guides to learn more about Kubernetes service discovery using the Micronaut Framework:
|
Important
|
The Kubernetes configuration client support is deprecated and has been replaced by Configuration Import. If you are adding new configuration loading from Kubernetes The deprecated configuration client documentation remains available for existing applications, but both the documentation and implementation will be removed in a future release, and new applications should use configuration import. Configuration import uses |
The Configuration client will read Kubernetes' ConfigMaps and Secrets instances and make them available as PropertySources
instances in your application.
To get started, you need to declare the following dependency:
implementation("io.micronaut.kubernetes:micronaut-kubernetes-discovery-client")Then, in any bean you can read the configuration values from the ConfigMap or Secret using @Value or
any other way to read configuration values.
Configuration parsing happens in the bootstrap phase. Therefore, to enable distributed configuration clients, define the
following in bootstrap.yml (or .properties, .json, etc):
micronaut.config-client.enabled=trueConfigMaps
Supported formats for ConfigMaps are:
-
Java
.properties. -
YAML.
-
JSON.
-
Literal values.
The configuration client by default will read all the ConfigMaps for the configured namespace. You can further filter
which config map names are processed by defining kubernetes.client.config-maps.includes or
kubernetes.client.config-maps.excludes:
kubernetes.client.config-maps.includes[0]=my-config-map
kubernetes.client.config-maps.includes[1]=other-config-mapOr:
kubernetes.client.config-maps.excludes=not-this-config-mapIn addition to that, Kubernetes labels can be used to better match the config maps that should be available as property sources. This can be done by defining the label and value directly:
kubernetes.client.config-maps.labels[0].app=my-app
kubernetes.client.config-maps.labels[1].env=prodOr by including every config map that has the same Kubernetes label as the pod the Micronaut application is running in.
This is handy if you use a package manager like Helm.
A good example would be app.kubernetes.io/instance which is a unique label identifying the instance of an application:
kubernetes.client.config-maps.pod-labels[0]=app.kubernetes.io/instanceAdditionally, you can also enable exception-on-pod-labels-missing property in case you want an exception to be thrown if at least
one of the labels, specified in pod-labels section is not found among the application’s pod labels. This can be useful if you want
to prevent loading all config maps available in namespace in case of mistyping or misconfiguration:
kubernetes.client.config-maps.exception-on-pod-labels-missing=true
kubernetes.client.config-maps.pod-labels[0]=app.kubernetes.io/instanceNote that on the resulting config maps, you can still further filter them with includes/excludes properties.
Reading ConfigMaps from mounted volumes
In the case of ConfigMapss, reading them from the Kubernetes API requires additional permissions, as stated above.
Therefore, you may want to read them from mounted volumes in the pod.
Given the following ConfigMap:
apiVersion: v1
kind: ConfigMap
metadata:
name: mounted-config
data:
mounted.yml: |-
foo: barIt can be mounted as a volume in a pod or deployment definition:
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: mypod
image: micronautapp
volumeMounts:
- name: configuration
mountPath: /etc/configuration
readOnly: true
volumes:
- name: configuration
configMap:
name: mounted-configThis will make Kubernetes to create file per data entry:
-
/etc/configuration/mounted.yml
While you could potentially use the java.io or java.nio APIs to read the contents yourself, this configuration module
can convert them into a PropertySource so that you can consume the values much more easily. In order to do so, define
the following configuration:
kubernetes.client.config-maps.paths[0]=/etc/configurationEach file in the directory will become a property source file. The file format will be automatically deduced based on the file suffix. Supported formats for mounted ConfigMap files are:
-
Java
.properties -
YAML
-
JSON
|
Note
|
When In this scenario, if there are property keys defined in both type of config maps, the ones coming from mounted volumes will take precedence over the ones coming from the API. |
Watching for changes in ConfigMaps
By default, this configuration module will watch for ConfigMaps added/modified/deleted, and provided that the changes
match with the above filters, they will be propagated to the Environment and refresh it.
This means that those changes will be immediately available in your application without a restart.
If you want to disable watching for ConfigMap changes, set kubernetes.client.config-maps.watch to false.
This should be done in the bootstrap.yml configuration file because the configuration client is initialized during the bootstrap phase, which happens before evaluating the application.yml configuration file.
|
Note
|
When |
Examples
You can create a Kubernetes ConfigMap off an existing file with the following command:
kubectl create configmap my-config --from-file=my-config.properties
Or:
kubectl create configmap my-config --from-file=my-config.yml
Or:
kubectl create configmap my-config --from-file=my-config.json
You can also create a ConfigMap from literal values:
kubectl create configmap my-config --from-literal=special.how=very --from-literal=special.type=charm
Secrets
Secrets read from the Kubernetes API will be base64-decoded and made available as PropertySource s, so that they can be
also read with @Value, @ConfigurationProperties, etc.
|
Note
|
Only Opaque secrets will be considered.
|
By default, secrets access is disabled. To enable them, set in bootstrap.yml:
kubernetes.client.secrets.enabled=trueThe configuration client, by default, will read all the Secrets for the configured namespace. You can further filter
which secret names are processed by defining kubernetes.client.secrets.includes or kubernetes.client.secrets.excludes:
kubernetes.client.secrets.enabled=true
kubernetes.client.secrets.includes=this-secretOr:
kubernetes.client.secrets.enabled=true
kubernetes.client.secrets.excludes=not-this-secretSimilarly to ConfigMaps, labels can also be used to match the desired secrets:
kubernetes.client.secrets.enabled=true
kubernetes.client.secrets.labels[0].app=my-app
kubernetes.client.secrets.labels[1].env=prodThis also works for pod labels:
kubernetes.client.secrets.enabled=true
kubernetes.client.secrets.pod-labels[0]=app.kubernetes.io/instanceAs well as exception-on-pod-labels-missing property:
kubernetes.client.secrets.enabled=true
kubernetes.client.secrets.exception-on-pod-labels-missing=true
kubernetes.client.secrets.pod-labels[0]=app.kubernetes.io/instanceReading Secrets from mounted volumes
In the case of Secrets, reading them from the Kubernetes API requires additional permissions, as stated above.
Therefore, you may want to read them from mounted volumes in the pod.
Given the following secret:
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
data:
username: YWRtaW4=
password: MWYyZDFlMmU2N2RmIt can be mounted as a volume in a pod or deployment definition:
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: mypod
image: redis
volumeMounts:
- name: foo
mountPath: "/etc/foo"
readOnly: true
volumes:
- name: foo
secret:
secretName: mysecretThis will make Kubernetes to create 2 files:
-
/etc/foo/username. -
/etc/foo/password.
Their content will be the decoded strings from the original base-64 encoded values.
While you could potentially use the java.io or java.nio APIs to read the contents yourself, this configuration module
can convert them into a PropertySource so that you can consume the values much more easily. In order to do so, define
the following configuration:
kubernetes.client.secrets.enabled=true
kubernetes.client.secrets.paths[0]=/etc/fooEach file in the directory will become the property key, and the file contents, the property value.
|
Note
|
When In this scenario, if there are property keys defined in both type of secrets, the ones coming from mounted volumes will take precedence over the ones coming from the API. |
Watching for changes in Secrets
If watch is enabled, this configuration module will watch for Secrets added/modified/deleted, and provided that the
changes match with the above filters, they will be propagated to the Environment and refresh it.
This means that those changes will be immediately available in your application without a restart.
If you want to enable watching for Secret changes, set kubernetes.client.secrets.watch to true.
This should be done in the bootstrap.yml configuration file because the configuration client is initialized during the
bootstrap phase, which happens before evaluating the application.yml configuration file.
|
Note
|
When |
Health Indicators
This configuration module provides a KubernetesHealthIndicator that probes communication with the Kubernetes API, and provides some information about the pod where the application is running from.
The service discovery client will also display all the services that were resolved from Kubernetes.
An example output of a /health request would be:
{
"name": "micronaut-service",
"status": "UP",
"details": {
"kubernetes": {
"name": "micronaut-service",
"status": "UP",
"details": {
"namespace": "default",
"podName": "example-service-786cd45b78-bzfw5",
"podPhase": "Running",
"podIP": "10.1.3.124",
"hostIP": "192.168.65.3",
"containerStatuses": [
{
"name": "example-service",
"image": "registry.hub.docker.com/alvarosanchez/example-service:latest",
"ready": true
}
]
}
},
"compositeDiscoveryClient(kubernetes)": {
"name": "micronaut-service",
"status": "UP",
"details": {
"services": {
"example-service": [
"http://10.1.3.124:8081",
"http://10.1.3.126:8081"
],
"non-secure-service": [
"http://10.1.3.127:1234"
],
"kubernetes": [
"https://kubernetes:443"
],
"secure-service-port-name": [
"https://10.1.3.127:1234"
],
"example-client": [
"http://10.1.3.125:8082"
],
"secure-service-port-number": [
"https://10.1.3.127:443"
],
"secure-service-labels": [
"https://10.1.3.127:1234"
]
}
}
},
"diskSpace": {
"name": "micronaut-service",
"status": "UP",
"details": {
"total": 109702647808,
"free": 69758287872,
"threshold": 10485760
}
}
}
}|
Important
|
Health checks require the following dependency: Also note that in order to see the full details of the health checks you may need additional configuration. Check the documentation of the Health Endpoint for more information about how to configure it. |
By default the KubernetesHealthIndicator is enabled. To disable it use the following configuration:
endpoints.health.kubernetes.enabled=falseThe micronaut-kubernetes-informer module integrates the SharedIndexInformer that is part of official Kubernetes SDK and simplifies its creation. The Informer is similar to a Watch but an Informer tries to re-list and re-watch to hide failures from the caller and provides a store of all the current resources. Note that the default official implementation SharedInformerFactory creates shared informers per the Kubernetes resource type. However the module micronaut-kubernetes-informer creates namespace scoped informers of the Kubernetes resource type, meaning that the informer is shared per specified namespace and kind.
First you need add a dependency on the micronaut-kubernetes-informer module:
implementation("io.micronaut.kubernetes:micronaut-kubernetes-informer")Then create a bean that implements ResourceEventHandler with the Kubernetes resource type of your choice and add the @Informer annotation trough which you provide the configuration for the SharedIndexInformer.
The example below illustrates the declaration of the @Informer with the ResourceEventHandler for handling the changes of the Kubernetes V1ConfigMap resource.
To create an Informer for non-namespaced resource like V1ClusterRole, configure the @Informer the same way like it is done for namespaced resource.
The ResourceEventHandlerConstructorInterceptor logic takes care of automated evaluation of the resource apiGroup and resourcePlural by fetching the API resource details from the Kubernetes API by using Discovery. The API resource discovery can be disabled by: kubernetes.client.api-discovery.enabled: false. In the case the discovery is disabled the resourcePlural and apiGroup needs to be provided manually in the @Informer annotation.
The @Informer annotation provides several configuration options:
Element |
Description |
|
The resource api type that must extend from the |
|
The resource api list type. For example |
|
The resource api group. For example some of the Kubernetes core resources has no group like |
|
The resource plural that identifies the Api. For example for the resource |
|
Namespace of the watched resource. If left empty then namespace is resolved by NamespaceResolver. To watch resources from all namespaces configure this parameter to |
|
List of the namespaces of the watcher resource. The SharedIndexInformer will be created for every namespace and all events will be delivered to the specified |
|
|
|
Informer label selector, see Label selectors for more information. By default there is no label selector. |
|
|
|
How often to check the need for resync of resources. If left empty the default resync check period is used. |
|
Important
|
The concept of shared informer means that the SharedIndexInformer for the respective Kubernetes resource type is registered just once for the given namespace. The next request to register another informer of the same Kubernetes resource type within the same namespace will result in returning of the previously created informer. In practice if you create two ResourceEventHandler<V1ConfigMap> but the @Informer annotation will have different optional configuration for labelSelector then the SharedInformerFactory creates just one SharedInformer, meaning the other @Informer configuration will be ignored. If the labelSelector resp. labelSelectorSupplier differs then create one labelSelector that matches both cases.
|
Programmatic creation of SharedIndexInformer
Use the bean SharedIndexInformerFactory to create the SharedIndexInformer programmatically:
By using the SharedIndexInformerFactory bean you can also get existing informer:
SharedIndexInformer local cache
The SharedIndexInformer has internal cache that is eventually consistent with the authoritative state. The local cache starts out empty, and gets populated and updated. For the detailed description of SharedIndexInformer internals visit the reference implementation pkg.go.dev/k8s.io/client-go#SharedIndexInformer in Go language.
The cache is exposed by SharedIndexInformer#getIndexer() method:
@Singleton
public class SharedInformerCache {
private final SharedIndexInformerFactory sharedIndexInformerFactory;
public SharedInformerCache(SharedIndexInformerFactory sharedIndexInformerFactory) {
this.sharedIndexInformerFactory = sharedIndexInformerFactory;
}
/**
* Get all config maps from informer from namespace.
*/
List<V1ConfigMap> getConfigMaps(String namespace) {
SharedIndexInformer<V1ConfigMap> sharedIndexInformer = sharedIndexInformerFactory.getExistingSharedIndexInformer(namespace, V1ConfigMap.class);
if (sharedIndexInformer != null) {
Indexer<V1ConfigMap> indexer = sharedIndexInformer.getIndexer();
return indexer.list();
} else {
return null;
}
}
}The micronaut-kubernetes-operator module integrates the official Kubernetes Java SDK controller support that is part of the extended client module. The micronaut-kubernetes-operator module is build on top of the micronaut-kubernetes-informer module and allows you to easily create the reconciler for both native and custom resources.
First you need add a dependency on the micronaut-kubernetes-operator module:
implementation("io.micronaut.kubernetes:micronaut-kubernetes-operator")Then create a bean that implements ResourceReconciler with the Kubernetes resource type of your choice. Then add the @Operator annotation trough the which you provide the configuration for the controllers that will be created by Micronaut for your reconciler.
The example below illustrates the use of the @Operator with the ResourceReconciler that reconciles the Kubernetes V1ConfigMap resource.
The @Operator annotation provides several configuration options:
Element |
Description |
|
The name of the operator controller thread. Defaults to the |
|
The @Informer annotation used for the configuration of the Operator’s informer. This value is required. |
|
The |
|
The |
|
The |
Leader election
The LeaderElectingController is responsible for the leader election of the application replica that will reconcile the resources. Generally if the lock is not renewed within the specified amount of time, other replicas may try to acquire the lock and become the leader.
You can adjust the leader elector configuration by using Micronaut configuration properties kubernetes.client.operator.leader-election.lock:
Element |
Description |
|
The lock lease duration. Defaults to |
|
The lock renew deadline. If the LeaderElector fails to renew the lock within the deadline then the controller looses the lock. Defaults to |
|
The lock acquire retry period. Defaults to |
For example:
kubernetes.client.operator.leader-election.lock.lease-duration=60s
kubernetes.client.operator.leader-election.lock.renew-deadline=50s
kubernetes.client.operator.leader-election.lock.retry-period=20sAdditionally, when the lock is acquired the LeaseAcquiredEvent is emitted. Similarly on a lost lease the LeaseLostEvent event is emitted.
Lock identity
The lock identity is used to uniquely identify the application that holds the lock and thus is responsible for reconciling the resources. By default, the POD name the application runs within is the source for the lock identity. This means the application must run in the Kubernetes cluster.
To create custom lock identity, create a bean that implements the LockIdentityProvider interface:
import io.micronaut.context.annotation.Replaces;
import io.micronaut.kubernetes.client.operator.leaderelection.LockIdentityProvider;
import jakarta.inject.Singleton;
@Singleton
@Replaces(LockIdentityProvider.class)
public class CustomLockIdentityProvider implements LockIdentityProvider {
@Override
public String getIdentity() {
// ...
}
}Lock resource types
The LeaderElectingController uses the native Kubernetes resource to store the lock information. Currently supported resources are V1ConfigMap, V1Endpoints and V1Lease.
By default, the micronaut-kubernetes-operator module uses the V1Lease. This can be changed to V1Endpoints by configuring the property kubernetes.client.operator.leader-election.lock.resource-kind to endpoints , resp. to configmaps in case the V1ConfigMap resource is requested.
Note that the resource for the lock is created in the application namespace. Then the application name is used as the lock resource name. This can be changed by using Micronaut configuration properties:
kubernetes.client.operator.leader-election.lock.resource-name=custom-name
kubernetes.client.operator.leader-election.lock.resource-namespace=custom-namespaceNote that in case the RBAC authorization is enabled in your Kubernetes cluster, your application needs to have properly configured role with respect to the lock resource type.
V1Lease lock resource type:---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: operator-lease-role
rules:
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- get
- create
- update
---Resource filtering
The @Operator annotation allows you to configure the resource filters that are executed before the resource is added into the reconciler work queue. You can configure three types of filters that are distinguished by the resource lifecycle: onAddFilter, onUpdateFilter and onDeleteFilter.
The onAddFilter predicate processes the newly created resources. Create a bean that implements java.util.function.Predicate with the same Kubernetes resource type like the operator is. Example below illustrates such filter for the V1ConfigMap resource:
import io.kubernetes.client.openapi.models.V1ConfigMap;
import jakarta.inject.Singleton;
import java.util.function.Predicate;
@Singleton
public class OnAddFilter implements Predicate<V1ConfigMap> {
@Override
public boolean test(V1ConfigMap v1ConfigMap) {
if (v1ConfigMap.getMetadata().getAnnotations() != null) {
return v1ConfigMap.getMetadata().getAnnotations().containsKey("io.micronaut.operator");
}
return false;
}
}The onUpdateFilter bi-predicate processes modified resources. Create a bean that implements java.util.function.BiPredicate with the same Kubernetes resource type like the operator is. Example below illustrates such filter for the V1ConfigMap resource:
import io.kubernetes.client.openapi.models.V1ConfigMap;
import jakarta.inject.Singleton;
import java.util.function.BiPredicate;
@Singleton
public class OnUpdateFilter implements BiPredicate<V1ConfigMap, V1ConfigMap> {
@Override
public boolean test(V1ConfigMap oldObj, V1ConfigMap newObj) {
if (newObj.getMetadata().getAnnotations() != null) {
return newObj.getMetadata().getAnnotations().containsKey("io.micronaut.operator");
}
return false;
}
}The onDeleteFilter bi-predicate processes deleted resources. Create a bean that implements java.util.function.BiPredicate with the same Kubernetes resource type like the operator is and the Boolean as second type. Example below illustrates such filter for the V1ConfigMap resource:
import io.kubernetes.client.openapi.models.V1ConfigMap;
import jakarta.inject.Singleton;
import java.util.function.BiPredicate;
@Singleton
public class OnDeleteFilter implements BiPredicate<V1ConfigMap, Boolean> {
@Override
public boolean test(V1ConfigMap v1ConfigMap, Boolean deletedFinalStateUnknown) {
if (v1ConfigMap.getMetadata().getAnnotations() != null) {
return v1ConfigMap.getMetadata().getAnnotations().containsKey("io.micronaut.operator");
}
return false;
}
}|
Important
|
Note that in case of onDeleteFilter the predicate receives the resource for the test method, but when the ResouceReconciler’s reconcile method is executed the lister will return Optional.empty since the resource was already deleted. To properly reconcile the resource on it’s removal, use finalizers.
|
Example below illustrates the configuration of the filters:
The Micronaut Kubernetes Client OpenApi is a kubernetes client which uses Micronaut Netty HTTP Client and generated apis and modules from the OpenApi Spec of the official Java client library for Kubernetes.
Advantages of this client over the official Java client library for Kubernetes:
-
No extra dependencies needed (OkHttp, Bouncy Castle, Kotlin etc.)
-
Unified configuration with Micronaut HTTP client
-
Support for plugging in filters
-
Native Image compatibility
The client comes in two flavors:
-
micronaut-kubernetes-client-openapimodule implements a blocking client -
micronaut-kubernetes-client-openapi-reactormodule implements a reactive client using Project Reactor
The micronaut-kubernetes-client-openapi-common module contains code that is shared between both above mentioned clients.
Blocking Client
First you need to add a dependency on the micronaut-kubernetes-client-openapi module:
implementation("io.micronaut.kubernetes:micronaut-kubernetes-client-openapi")Then you can simply use Micronaut injection to get configured apis object from package io.micronaut.kubernetes.client.openapi.api:
package micronaut.client;
import io.micronaut.http.annotation.Controller;
import io.micronaut.http.annotation.Get;
import io.micronaut.kubernetes.client.openapi.api.CoreV1Api;
import io.micronaut.kubernetes.client.openapi.model.V1Pod;
import io.micronaut.kubernetes.client.openapi.model.V1PodList;
import io.micronaut.scheduling.TaskExecutors;
import io.micronaut.scheduling.annotation.ExecuteOn;
import jakarta.inject.Inject;
import jakarta.validation.constraints.NotNull;
import java.util.Map;
import java.util.stream.Collectors;
@Controller("/pods")
@ExecuteOn(TaskExecutors.BLOCKING)
public class PodController {
@Inject
CoreV1Api coreV1Api;
@Get("/{namespace}/{name}")
public String getPod(final @NotNull String namespace, final @NotNull String name) {
V1Pod v1Pod = coreV1Api.readNamespacedPod(name, namespace, null);
return v1Pod.getStatus().getPhase();
}
@Get("/{namespace}")
public Map<String, String> getPods(final @NotNull String namespace) {
V1PodList v1PodList = coreV1Api.listNamespacedPod(namespace, null, null, null, null, null, null, null, null, null, null, null, null);
return v1PodList.getItems().stream()
.filter(p -> p.getStatus() != null)
.collect(Collectors.toMap(
p -> p.getMetadata().getName(),
p -> p.getStatus().getPhase()));
}
}Reactive Client
First you need to add a dependency on the micronaut-kubernetes-client-openapi-reactor module:
implementation("io.micronaut.kubernetes:micronaut-kubernetes-client-openapi-reactor")Then you can simply use Micronaut injection to get configured apis object from package io.micronaut.kubernetes.client.openapi.api.reactor:
package micronaut.client;
import io.micronaut.http.annotation.Controller;
import io.micronaut.http.annotation.Get;
import io.micronaut.kubernetes.client.openapi.reactor.api.CoreV1ApiReactor;
import jakarta.inject.Inject;
import jakarta.validation.constraints.NotNull;
import reactor.core.publisher.Mono;
import java.util.Map;
import java.util.stream.Collectors;
@Controller("/pods")
public class PodController {
@Inject
CoreV1ApiReactor coreV1ApiReactor;
@Get("/{namespace}/{name}")
public Mono<String> getPod(final @NotNull String namespace, final @NotNull String name) {
return coreV1ApiReactor.readNamespacedPod(name, namespace, null)
.map(it -> it.getStatus().getPhase());
}
@Get("/{namespace}")
public Mono<Map<String, String>> getPods(final @NotNull String namespace) {
return coreV1ApiReactor.listNamespacedPod(namespace, null, null, null, null, null, null, null, null, null, null, null, null)
.map(it -> it.getItems().stream()
.filter(p -> p.getStatus() != null)
.collect(Collectors.toMap(
p -> p.getMetadata().getName(),
p -> p.getStatus().getPhase())));
}
}Configuration
kubernetes.client.enabled=true
kubernetes.client.kube-config-path=file:/path/to/kubeconfig
kubernetes.client.namespace=test-namespace
kubernetes.client.service-account.enabled=true
kubernetes.client.service-account.certificate-authority-path=file:/path/to/certificate/authority/file
kubernetes.client.service-account.token-path=file:/path/to/token/file
kubernetes.client.service-account.namespace-path=file:/path/to/namespace/file
kubernetes.client.service-account.token-reload-interval=2mName |
Description |
enabled |
Whether to enable the client. Default: |
kube-config-path |
Absolute path to the kube config file. Default: |
namespace |
Namespace the app runs in. Not required. |
service-account.enabled |
Whether to enable the service account authentication. Default: |
service-account.certificate-authority-path |
Absolute path to the certificate authority file. Default: |
service-account.token-path |
Absolute path to the token file. Default: |
service-account.namespace-path |
Absolute path to the namespace file. Default: |
service-account.token-reload-interval |
Token reload interval. Default: |
Authentication
The client supports the following authentication strategies:
-
client certificate authentication (certificate and private key provided in the kube config file)
-
basic authentication (username and password provided in the kube config file)
-
token authentication (token provided in the kube config file or by executing the command from the kube config file)
-
service account authentication (used only if the kube config file not provided and running inside the kubernetes cluster)
Customization
There are several interfaces that can be implemented to change default implementations:
-
KubeConfigLoader - a custom implementation can be used when a kube config file needs to be loaded from a cloud service. There is also an option of extending AbstractKubeConfigLoader which caches the loaded kube config data and provides a few helper methods.
-
KubernetesTokenLoader - a custom blocking implementation can be used for loading a bearer token. KubernetesHttpClientFilter iterates through a list of blocking and reactive implementations and creates the Authorization header using the token from the first implementation which returns it
-
ReactiveKubernetesTokenLoader - a custom reactive implementation can be used for loading a bearer token. The client is using the following implementations by default:
-
ExecCommandCredentialLoader - implementation which executes the command from the kube config file to get the token.
-
KubeConfigTokenLoader - implementation which uses the token from the kube config file.
-
ServiceAccountTokenLoader - implementation which uses the service account token.
-
-
KubernetesPrivateKeyLoader - a custom implementation can be used if there is a need for usage of third party libraries (for example, Bouncy Castle) to load the private key when the client certificate authentication is used.
The Service Discovery module allows Micronaut HTTP clients to discover Kubernetes services.
To get started, you need to declare the following dependency:
implementation("io.micronaut.kubernetes:micronaut-kubernetes-client-openapi-discovery")By default in any client you can use as Service ID the Kubernetes Endpoints name generated by a Kubernetes Service for the configured namespace.
Consider the following Kubernetes service definition:
my-service.ymlkind: Service
apiVersion: v1
metadata:
name: my-service
spec:
selector:
app: MyApp
ports:
- protocol: TCP
port: 80
targetPort: 9376This specification will create a new Service object named my-service, as well as an Endpoints object also named my-service.
In your HTTP client, you can use my-service as Service ID: @Client("my-service").
Note that service discovery is enabled by default in Micronaut. To disable it, set kubernetes.client.discovery.enabled
to false.
Service specific client configuration
Kubernetes Service is a complex resource that can handle various use cases by providing specific configuration. For this Micronaut Kubernetes supports a manual service discovery configuration per Service http client that allows you to configure custom:
| Key | Description |
|---|---|
|
name of the resource in Kubernetes in case it is different than the Service ID |
|
namespace of the resource in case it’s different than the configured namespace |
|
port name in case the target resource is a Multi-Port Service |
|
service specific discovery mode in case it’s different than the globally configured discovery mode |
Examples of service configurations
Multi-port service
For the following Multi-Port Service:
my-service.ymlapiVersion: v1
kind: Service
metadata:
name: my-service
spec:
selector:
app: MyApp
ports:
- name: http
protocol: TCP
port: 80
targetPort: 9376
- name: https
protocol: TCP
port: 443
targetPort: 9377the manual service configuration for http port will be:
kubernetes.client.discovery.services.my-service.port=httpHeadless service with selector
For the following Headless service with selector:
my-service.ymlapiVersion: v1
kind: Service
metadata:
name: my-service
spec:
clusterIP: None
selector:
app: MyApp
ports:
- name: http
protocol: TCP
port: 80
targetPort: 9376the manual service configuration will be:
kubernetes.client.discovery.services.my-service.mode=endpointExternalName service type
For the following ExternalName service:
my-service.ymlapiVersion: v1
kind: Service
metadata:
name: my-service
namespace: prod
spec:
type: ExternalName
externalName: launch.micronaut.iothe manual service configuration will be:
kubernetes.client.discovery.services.my-service.mode=serviceService discovery modes
Service discovery mode is a mechanism that allows to support different strategies for the actual service discovery in Kubernetes by implementing KubernetesServiceInstanceProvider interface.
Currently Micronaut Kubernetes implements two discovery modes:
-
endpointmode uses the KubernetesEndpointsAPI for the service discovery. Note that the service load balancing is handled by Micronaut application. -
servicemode uses the KubernetesServiceAPI for the service discovery. Theservicemode extracts the serviceClusterIPaddress from the Service status.
Both discovery modes are using the metadata.name for the Service ID identificator.
The discovery mode can be configured globally for all Service IDs or per service.
Note that endpoint is the default global discovery mode. That can be overridden via configuration in bootstrap.yml:
kubernetes.client.discovery.mode=serviceWatching for changes
Both discovery modes support watching for changes of their respective resources. To enable it, set kubernetes.client.discovery.mode-configuration.endpoint.watch.enabled to true for the endpoint mode. For the service mode set kubernetes.client.discovery.mode-configuration.service.watch.enabled to true.
Kubernetes API authentication
Micronaut authenticates to the Kubernetes API using the token mounted at /var/run/secrets/kubernetes.io/serviceaccount/token.
Note that by default, the service account used may only have permissions over the kube-system namespace. The service discovery
functionality requires some additional read permissions. Refer to the Kubernetes documentation for more information
about Role-based access control (RBAC).
One of the options is to create the following Role and RoleBinding (make sure to apply them to the service account used, if not default):
auth.ymlkind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: service-discoverer
namespace: micronaut-kubernetes
rules:
- apiGroups: [""]
resources: ["services", "endpoints", "configmaps", "secrets", "pods"]
verbs: ["get", "watch", "list"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: default-service-discoverer
namespace: micronaut-kubernetes
subjects:
- kind: ServiceAccount
name: default
namespace: micronaut-kubernetes
roleRef:
kind: Role
name: service-discoverer
apiGroup: rbac.authorization.k8s.io|
Note
|
In Google Cloud’s Kubernetes Engine, in order to create the above, you must grant your user the ability to create roles in Kubernetes by running the following command: |
kubectl create clusterrolebinding cluster-admin-binding --clusterrole cluster-admin --user yourGoogleAccount@gmail.com
Connecting to services using HTTPS
There are three ways for this library to determine whether a service should be connected to using SSL (the following
examples assume there is a Deployment named secure-deployment).
Using https as port name
my-service.ymlapiVersion: v1
kind: Service
metadata:
name: secure-service-port-name
spec:
selector:
app: secure-deployment
type: NodePort
ports:
- port: 1234
protocol: TCP
name: httpsUsing a port ending in 443
Port numbers like 443, 8443, etc. will match.
my-service.ymlapiVersion: v1
kind: Service
metadata:
name: secure-service-port-number
spec:
selector:
app: secure-deployment
type: NodePort
ports:
- port: 443
protocol: TCPUsing labels
Set a label named secure with value true to have the client use HTTPS.
my-service.ymlapiVersion: v1
kind: Service
metadata:
name: secure-service-labels
labels:
secure: "true"
spec:
selector:
app: secure-deployment
type: NodePort
ports:
- port: 1234
protocol: TCPService filtering
You can filter the services discovered by using kubernetes.client.discovery.includes or
kubernetes.client.discovery.excludes:
kubernetes.client.discovery.includes[0]=my-service
kubernetes.client.discovery.includes[1]=other-serviceOr:
kubernetes.client.discovery.excludes=not-this-serviceIn addition to that, Kubernetes labels can be used to better match the services that should be available for service discovery:
kubernetes.client.discovery.labels[0].app=my-app
kubernetes.client.discovery.labels[1].env=prodNote that the filtering is not applied on manually configured service configurations.
|
Important
|
The Kubernetes configuration client support is deprecated and has been replaced by Configuration Import. If you are adding new configuration loading from Kubernetes The deprecated configuration client documentation remains available for existing applications, but both the documentation and implementation will be removed in a future release, and new applications should use configuration import. Configuration import uses |
The Configuration client will read Kubernetes' ConfigMaps and Secrets instances and make them available as PropertySources
instances in your application.
To get started, you need to declare the following dependency:
implementation("io.micronaut.kubernetes:micronaut-kubernetes-client-openapi-discovery")Then, in any bean you can read the configuration values from the ConfigMap or Secret using @Value or
any other way to read configuration values.
Configuration parsing happens in the bootstrap phase. Therefore, to enable distributed configuration clients, define the
following in bootstrap.yml (or .properties, .json, etc):
micronaut.config-client.enabled=trueConfigMaps
Supported formats for ConfigMaps are:
-
Java
.properties. -
YAML.
-
JSON.
-
Literal values.
The configuration client by default will read all the ConfigMaps for the configured namespace. You can further filter
which config map names are processed by defining kubernetes.client.config-maps.includes or
kubernetes.client.config-maps.excludes:
kubernetes.client.config-maps.includes[0]=my-config-map
kubernetes.client.config-maps.includes[1]=other-config-mapOr:
kubernetes.client.config-maps.excludes=not-this-config-mapIn addition to that, Kubernetes labels can be used to better match the config maps that should be available as property sources. This can be done by defining the label and value directly:
kubernetes.client.config-maps.labels[0].app=my-app
kubernetes.client.config-maps.labels[1].env=prodOr by including every config map that has the same Kubernetes label as the pod the Micronaut application is running in.
This is handy if you use a package manager like Helm.
A good example would be app.kubernetes.io/instance which is a unique label identifying the instance of an application:
kubernetes.client.config-maps.pod-labels[0]=app.kubernetes.io/instanceAdditionally, you can also enable exception-on-pod-labels-missing property in case you want an exception to be thrown if at least
one of the labels, specified in pod-labels section is not found among the application’s pod labels. This can be useful if you want
to prevent loading all config maps available in namespace in case of mistyping or misconfiguration:
kubernetes.client.config-maps.exception-on-pod-labels-missing=true
kubernetes.client.config-maps.pod-labels[0]=app.kubernetes.io/instanceNote that on the resulting config maps, you can still further filter them with includes/excludes properties.
By default if an exception occurs reading config maps on startup, the exception is logged and the service is started.
This can be overridden to terminate the service startup when the service is not able to read config maps because of any
exception by setting the terminate-startup-on-exception property to true in your application configuration:
kubernetes.client.config-maps.terminate-startup-on-exception=trueReading ConfigMaps from mounted volumes
In the case of ConfigMapss, reading them from the Kubernetes API requires additional permissions, as stated above.
Therefore, you may want to read them from mounted volumes in the pod.
Given the following ConfigMap:
apiVersion: v1
kind: ConfigMap
metadata:
name: mounted-config
data:
mounted.yml: |-
foo: barIt can be mounted as a volume in a pod or deployment definition:
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: mypod
image: micronautapp
volumeMounts:
- name: configuration
mountPath: /etc/configuration
readOnly: true
volumes:
- name: configuration
configMap:
name: mounted-configThis will make Kubernetes to create file per data entry:
-
/etc/configuration/mounted.yml
While you could potentially use the java.io or java.nio APIs to read the contents yourself, this configuration module
can convert them into a PropertySource so that you can consume the values much more easily. In order to do so, define
the following configuration:
kubernetes.client.config-maps.paths[0]=/etc/configurationEach file in the directory will become a property source file. The file format will be automatically deduced based on the file suffix. Supported formats for mounted ConfigMap files are:
-
Java
.properties -
YAML
-
JSON
|
Note
|
When In this scenario, if there are property keys defined in both type of config maps, the ones coming from mounted volumes will take precedence over the ones coming from the API. |
Watching for changes in ConfigMaps
By default, this configuration module will watch for ConfigMaps added/modified/deleted, and provided that the changes
match with the above filters, they will be propagated to the Environment and refresh it.
This means that those changes will be immediately available in your application without a restart.
If you want to disable watching for ConfigMap changes, set kubernetes.client.config-maps.watch to false.
This should be done in the bootstrap.yml configuration file because the configuration client is initialized during the bootstrap phase, which happens before evaluating the application.yml configuration file.
|
Note
|
When |
Examples
You can create a Kubernetes ConfigMap off an existing file with the following command:
kubectl create configmap my-config --from-file=my-config.properties
Or:
kubectl create configmap my-config --from-file=my-config.yml
Or:
kubectl create configmap my-config --from-file=my-config.json
You can also create a ConfigMap from literal values:
kubectl create configmap my-config --from-literal=special.how=very --from-literal=special.type=charm
Secrets
Secrets read from the Kubernetes API will be base64-decoded and made available as PropertySource s, so that they can be
also read with @Value, @ConfigurationProperties, etc.
|
Note
|
Only Opaque secrets will be considered.
|
By default, secrets access is disabled. To enable them, set in bootstrap.yml:
kubernetes.client.secrets.enabled=trueThe configuration client, by default, will read all the Secrets for the configured namespace. You can further filter
which secret names are processed by defining kubernetes.client.secrets.includes or kubernetes.client.secrets.excludes:
kubernetes.client.secrets.enabled=true
kubernetes.client.secrets.includes=this-secretOr:
kubernetes.client.secrets.enabled=true
kubernetes.client.secrets.excludes=not-this-secretSimilarly to ConfigMaps, labels can also be used to match the desired secrets:
kubernetes.client.secrets.enabled=true
kubernetes.client.secrets.labels[0].app=my-app
kubernetes.client.secrets.labels[1].env=prodThis also works for pod labels:
kubernetes.client.secrets.enabled=true
kubernetes.client.secrets.pod-labels[0]=app.kubernetes.io/instanceAs well as exception-on-pod-labels-missing property:
kubernetes.client.secrets.enabled=true
kubernetes.client.secrets.exception-on-pod-labels-missing=true
kubernetes.client.secrets.pod-labels[0]=app.kubernetes.io/instanceSimilar to ConfigMaps, if you want to terminate the service startup when the service is not able to read secrets
because of any unexpected exception, you can set terminate-startup-on-exception property to true in application configuration:
kubernetes.client.secrets.terminate-startup-on-exception=trueReading Secrets from mounted volumes
In the case of Secrets, reading them from the Kubernetes API requires additional permissions, as stated above.
Therefore, you may want to read them from mounted volumes in the pod.
Given the following secret:
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
data:
username: YWRtaW4=
password: MWYyZDFlMmU2N2RmIt can be mounted as a volume in a pod or deployment definition:
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: mypod
image: redis
volumeMounts:
- name: foo
mountPath: "/etc/foo"
readOnly: true
volumes:
- name: foo
secret:
secretName: mysecretThis will make Kubernetes to create 2 files:
-
/etc/foo/username. -
/etc/foo/password.
Their content will be the decoded strings from the original base-64 encoded values.
While you could potentially use the java.io or java.nio APIs to read the contents yourself, this configuration module
can convert them into a PropertySource so that you can consume the values much more easily. In order to do so, define
the following configuration:
kubernetes.client.secrets.enabled=true
kubernetes.client.secrets.paths[0]=/etc/fooEach file in the directory will become the property key, and the file contents, the property value.
|
Note
|
When In this scenario, if there are property keys defined in both type of secrets, the ones coming from mounted volumes will take precedence over the ones coming from the API. |
Watching for changes in Secrets
If watch is enabled, this configuration module will watch for Secrets added/modified/deleted, and provided that the
changes match with the above filters, they will be propagated to the Environment and refresh it.
This means that those changes will be immediately available in your application without a restart.
If you want to enable watching for Secret changes, set kubernetes.client.secrets.watch to true.
This should be done in the bootstrap.yml configuration file because the configuration client is initialized during the
bootstrap phase, which happens before evaluating the application.yml configuration file.
|
Note
|
When |
Health Indicators
This configuration module provides a KubernetesHealthIndicator that probes communication with the Kubernetes API, and provides some information about the pod where the application is running from.
The service discovery client will also display all the services that were resolved from Kubernetes.
An example output of a /health request would be:
{
"name": "micronaut-service",
"status": "UP",
"details": {
"kubernetes": {
"name": "micronaut-service",
"status": "UP",
"details": {
"namespace": "default",
"podName": "example-service-786cd45b78-bzfw5",
"podPhase": "Running",
"podIP": "10.1.3.124",
"hostIP": "192.168.65.3",
"containerStatuses": [
{
"name": "example-service",
"image": "registry.hub.docker.com/alvarosanchez/example-service:latest",
"ready": true
}
]
}
},
"compositeDiscoveryClient(kubernetes)": {
"name": "micronaut-service",
"status": "UP",
"details": {
"services": {
"example-service": [
"http://10.1.3.124:8081",
"http://10.1.3.126:8081"
],
"non-secure-service": [
"http://10.1.3.127:1234"
],
"kubernetes": [
"https://kubernetes:443"
],
"secure-service-port-name": [
"https://10.1.3.127:1234"
],
"example-client": [
"http://10.1.3.125:8082"
],
"secure-service-port-number": [
"https://10.1.3.127:443"
],
"secure-service-labels": [
"https://10.1.3.127:1234"
]
}
}
},
"diskSpace": {
"name": "micronaut-service",
"status": "UP",
"details": {
"total": 109702647808,
"free": 69758287872,
"threshold": 10485760
}
}
}
}|
Important
|
Health checks require the following dependency: Also note that in order to see the full details of the health checks you may need additional configuration. Check the documentation of the Health Endpoint for more information about how to configure it. |
By default the KubernetesHealthIndicator is enabled. To disable it use the following configuration:
endpoints.health.kubernetes.enabled=falseThe Micronaut Kubernetes Client OpenApi Watcher implements streaming of Kubernetes events using the Micronaut Kubernetes Client OpenApi client.
First you need to add a dependency on the micronaut-kubernetes-client-openapi-watcher module:
implementation("io.micronaut.kubernetes:micronaut-kubernetes-client-openapi-watcher")Then you can simply use Micronaut injection to get configured apis object from package io.micronaut.kubernetes.client.openapi.watcher.api:
package micronaut.client;
import io.micronaut.http.annotation.Controller;
import io.micronaut.http.annotation.Get;
import io.micronaut.kubernetes.client.openapi.watcher.api.CoreV1ApiWatcher;
import jakarta.inject.Inject;
import jakarta.validation.constraints.NotNull;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@Controller("/secrets")
public class SecretController {
private static final Logger LOG = LoggerFactory.getLogger(SecretController.class);
@Inject
CoreV1ApiWatcher coreV1ApiWatcher;
@Get("/{namespace}")
public void startWatchingSecrets(final @NotNull String namespace) {
coreV1ApiWatcher.listNamespacedSecret(namespace, null, null, null, null, null, null, null, null, null, null, null, true)
.subscribe(event -> LOG.info(event.toString()));
}
}|
Important
|
The streaming will work only if the watch parameter is set to true.
|
The Kubernetes streamed events are deserialized into the WatchEvent class:
@Serdeable
public record WatchEvent<T>(
String type,
@Nullable T object,
@Nullable V1Status status
) {
}Name |
Description |
type |
The type of the streamed event. At the moment the type can be one of the following values: ADDED, MODIFIED, DELETED, ERROR, BOOKMARK |
object |
The data for the watched object (V1Namespace, V1Pod etc.). The value will be empty only if the event type is |
status |
The instance of V1Status which won’t be empty only if the event type is |
The Micronaut Kubernetes Client OpenApi Informer implements the Kubernetes Informer feature. It is built on top of Micronaut Kubernetes Client OpenApi Watcher and provides an internal cache of watched kubernetes objects.
First you need to add a dependency on the micronaut-kubernetes-client-openapi-informer module:
implementation("io.micronaut.kubernetes:micronaut-kubernetes-client-openapi-informer")Then create a bean that implements ResourceEventHandler with the Kubernetes resource type of your choice and add the @Informer annotation through which you provide the configuration for the SharedIndexInformer.
The example below illustrates the declaration of the @Informer with the
ResourceEventHandler for handling the changes of the Kubernetes V1Secret resource.
To create an Informer for non-namespaced resource like V1ClusterRole, configure the @Informer with namespace set to Informer.ALL_NAMESPACES.
The @Informer annotation provides several configuration options:
Element |
Description |
|
The resource api type that must extend from the KubernetesObject.
For Kubernetes core resources the types can be found in |
|
Namespace of the watched resource. If left empty then namespace is resolved by NamespaceResolver.
To watch resources from all namespaces configure this parameter to |
|
List of the namespaces of the watcher resource. The SharedIndexInformer
will be created for every namespace and all events will be delivered to the specified |
|
|
|
Informer label selector, see Label selectors for more information. By default there is no label selector. |
|
|
|
How often to check the need for resync of resources. If left empty the default resync check period is used. |
|
If set to |
|
Important
|
The concept of shared informer means that the SharedIndexInformer
for the respective Kubernetes resource type is registered just once for the given namespace. The next request to register another SharedIndexInformer of the
same Kubernetes resource type within the same namespace will result in throwing the Informer has been already created exception.
|
Programmatic creation of SharedIndexInformer
Use the bean SharedIndexInformerFactory to create the SharedIndexInformer programmatically:
package micronaut.client.configmap;
import io.micronaut.context.annotation.Context;
import io.micronaut.kubernetes.client.openapi.informer.SharedIndexInformer;
import io.micronaut.kubernetes.client.openapi.informer.SharedIndexInformerFactory;
import io.micronaut.kubernetes.client.openapi.informer.handler.ResourceEventHandler;
import io.micronaut.kubernetes.client.openapi.model.V1ConfigMap;
import io.micronaut.kubernetes.client.openapi.resolver.NamespaceResolver;
import jakarta.annotation.PostConstruct;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@Context
public class ConfigMapInformer {
private static final Logger LOG = LoggerFactory.getLogger(ConfigMapInformer.class);
private final SharedIndexInformerFactory sharedIndexInformerFactory;
private final NamespaceResolver namespaceResolver;
ConfigMapInformer(SharedIndexInformerFactory sharedIndexInformerFactory, NamespaceResolver namespaceResolver) {
this.sharedIndexInformerFactory = sharedIndexInformerFactory;
this.namespaceResolver = namespaceResolver;
}
@PostConstruct
void initialize() {
String namespace = namespaceResolver.resolveNamespace();
SharedIndexInformer<V1ConfigMap> sharedIndexInformer = sharedIndexInformerFactory.sharedIndexInformerFor(
V1ConfigMap.class, namespace);
sharedIndexInformer.addEventHandler(
new ResourceEventHandler<>() {
@Override
public void onAdd(V1ConfigMap obj) {
LOG.info("{} config map added!", obj.getMetadata().getName());
}
@Override
public void onUpdate(V1ConfigMap oldObj, V1ConfigMap newObj) {
LOG.info("{} config map updated!", oldObj.getMetadata().getName());
}
@Override
public void onDelete(V1ConfigMap obj, boolean deletedFinalStateUnknown) {
LOG.info("{} config map deleted!", obj.getMetadata().getName());
}
});
}
}By using the SharedIndexInformerFactory bean you can also get an existing informer:
SharedIndexInformer<V1ConfigMap> informer = sharedIndexInformerFactory.getExistingSharedIndexInformer(
V1ConfigMap.class,
namespace);SharedIndexInformer local cache
The SharedIndexInformer has an internal cache that is eventually consistent with the authoritative state. The local cache starts out empty, and gets populated and updated.
The cache is exposed by SharedIndexInformer getIndexer() method:
List<V1ConfigMap> all() {
String namespace = namespaceResolver.resolveNamespace();
SharedIndexInformer<V1ConfigMap> informer = sharedIndexInformerFactory.getExistingSharedIndexInformer(
V1ConfigMap.class,
namespace);
Indexer<V1ConfigMap> indexer = informer.getIndexer();
return indexer.list();
}Informer for Kubernetes Custom Resource
To create an Informer for a kubernetes custom resource you need:
-
class which implements KubernetesObject interface:
@Serdeable
public class CustomObject implements KubernetesObject {
private String apiVersion;
private String kind;
private V1ObjectMeta metadata;
private String value;
// getters/setters omitted
}-
class which implements KubernetesListObject interface:
@Serdeable
public class CustomObjectList implements KubernetesListObject {
private String apiVersion;
private String kind;
private V1ListMeta metadata;
private List<CustomObject> items;
// getters/setters omitted
}-
non-blocking operations interface for the custom resource
@KubernetesClientApiReactor
@BootstrapContextCompatible
@Client("kubernetes")
public interface CustomObjectApiReactor {
@Post("/apis/custom.test.io/v1/namespaces/{namespace}/customobjects")
@Consumes({"application/json", "application/yaml", "application/vnd.kubernetes.protobuf", "application/cbor"})
Mono<CustomObject> createNamespacedCustomObject(
@PathVariable("namespace") String namespace,
@Body CustomObject body,
@QueryValue("pretty") @Nullable String pretty,
@QueryValue("dryRun") @Nullable String dryRun,
@QueryValue("fieldManager") @Nullable String fieldManager,
@QueryValue("fieldValidation") @Nullable String fieldValidation
);
@Delete("/apis/custom.test.io/v1/namespaces/{namespace}/customobjects/{name}")
@Consumes({"application/json", "application/yaml", "application/vnd.kubernetes.protobuf", "application/cbor"})
Mono<DeleteResponse<CustomObject>> deleteNamespacedCustomObject(
@PathVariable("name") String name,
@PathVariable("namespace") String namespace,
@QueryValue("pretty") @Nullable String pretty,
@QueryValue("dryRun") @Nullable String dryRun,
@QueryValue("gracePeriodSeconds") @Nullable Integer gracePeriodSeconds,
@QueryValue("ignoreStoreReadErrorWithClusterBreakingPotential") @Nullable Boolean ignoreStoreReadErrorWithClusterBreakingPotential,
@QueryValue("orphanDependents") @Nullable Boolean orphanDependents,
@QueryValue("propagationPolicy") @Nullable String propagationPolicy,
@Body @Nullable V1DeleteOptions body
);
@Get("/apis/custom.test.io/v1/namespaces/{namespace}/customobjects")
@Consumes({"application/json", "application/yaml", "application/vnd.kubernetes.protobuf", "application/cbor", "application/json;stream=watch", "application/vnd.kubernetes.protobuf;stream=watch", "application/cbor-seq"})
Mono<CustomObjectList> listNamespacedCustomObject(
@PathVariable("namespace") String namespace,
@QueryValue("pretty") @Nullable String pretty,
@QueryValue("allowWatchBookmarks") @Nullable Boolean allowWatchBookmarks,
@QueryValue("continue") @Nullable String _continue,
@QueryValue("fieldSelector") @Nullable String fieldSelector,
@QueryValue("labelSelector") @Nullable String labelSelector,
@QueryValue("limit") @Nullable Integer limit,
@QueryValue("resourceVersion") @Nullable String resourceVersion,
@QueryValue("resourceVersionMatch") @Nullable String resourceVersionMatch,
@QueryValue("sendInitialEvents") @Nullable Boolean sendInitialEvents,
@QueryValue("timeoutSeconds") @Nullable Integer timeoutSeconds,
@QueryValue("watch") @Nullable Boolean watch
);
@Get("/apis/custom.test.io/v1/namespaces/{namespace}/customobjects/{name}")
@Consumes({"application/json", "application/yaml", "application/vnd.kubernetes.protobuf", "application/cbor"})
Mono<CustomObject> readNamespacedCustomObject(
@PathVariable("name") String name,
@PathVariable("namespace") String namespace,
@QueryValue("pretty") @Nullable String pretty
);
@Put("/apis/custom.test.io/v1/namespaces/{namespace}/customobjects/{name}")
@Consumes({"application/json", "application/yaml", "application/vnd.kubernetes.protobuf", "application/cbor"})
Mono<CustomObject> replaceNamespacedCustomObject(
@PathVariable("name") String name,
@PathVariable("namespace") String namespace,
@Body CustomObject body,
@QueryValue("pretty") @Nullable String pretty,
@QueryValue("dryRun") @Nullable String dryRun,
@QueryValue("fieldManager") @Nullable String fieldManager,
@QueryValue("fieldValidation") @Nullable String fieldValidation
);
}-
watcher interface for the custom resource
@KubernetesClientApiWatcher
@Client("kubernetes")
public interface CustomObjectApiWatcher {
@Get("/apis/custom.test.io/v1/namespaces/{namespace}/customobjects")
@Consumes({"application/json", "application/yaml", "application/vnd.kubernetes.protobuf", "application/cbor", "application/json;stream=watch", "application/vnd.kubernetes.protobuf;stream=watch", "application/cbor-seq"})
Flux<WatchEvent<CustomObject>> listCustomObject(
@PathVariable("namespace") String namespace,
@QueryValue("pretty") @Nullable String pretty,
@QueryValue("allowWatchBookmarks") @Nullable Boolean allowWatchBookmarks,
@QueryValue("continue") @Nullable String _continue,
@QueryValue("fieldSelector") @Nullable String fieldSelector,
@QueryValue("labelSelector") @Nullable String labelSelector,
@QueryValue("limit") @Nullable Integer limit,
@QueryValue("resourceVersion") @Nullable String resourceVersion,
@QueryValue("resourceVersionMatch") @Nullable String resourceVersionMatch,
@QueryValue("sendInitialEvents") @Nullable Boolean sendInitialEvents,
@QueryValue("timeoutSeconds") @Nullable Integer timeoutSeconds,
@QueryValue("watch") @Nullable Boolean watch
);
}In the OpenApi Spec of the official Java client library, names of list types are usually equal to names of list item types plus List suffix.
For example: V1SecretList = V1Secret + List, V1ConfigMapList = V1ConfigMap + List, etc. If that is not case for your classes which
implement KubernetesObject and KubernetesListObject interfaces,
you need to provide a mapping. Lets say that you have CustomObject and CustomObjectCollection classes, then you need a class
which maps the list type name to list item type name, like in this example:
@Singleton
public class CustomTypeNameMapper implements TypeNameMapper {
@Override
public Map<String, String> getMappings() {
Map<String, String> mappings = new HashMap<>();
mappings.put("io.micronaut.kubernetes.client.openapi.informer.example2.CustomObjectCollection", "io.micronaut.kubernetes.client.openapi.informer.example2.CustomObject");
return mappings;
}
}The Micronaut Kubernetes Client OpenApi Operator implements the Kubernetes Controller feature. It is built on top of Micronaut Kubernetes Client OpenApi Informer and allows you to easily create a reconciler for a Kubernetes resource.
First you need to add a dependency on the micronaut-kubernetes-client-openapi-operator module:
implementation("io.micronaut.kubernetes:micronaut-kubernetes-client-openapi-operator")Then create a bean that implements ResourceReconciler with the Kubernetes resource type of your choice and add the @Operator annotation through which you provide the configuration for the controllers that will be created by Micronaut for your reconciler.
The example below illustrates the use of the @Operator with the
ResourceReconciler that reconciles the Kubernetes V1ConfigMap resource.
The @Operator annotation provides several configuration options:
Element |
Description |
|
The name of the operator controller thread. Defaults to the |
|
The @Informer annotation used for the configuration of the Operator’s informer. This value is required. |
|
The |
|
The |
|
The |
The filter examples for the V1ConfigMap resource:
onAddFilter
package micronaut.operator;
import io.micronaut.kubernetes.client.openapi.model.V1ConfigMap;
import jakarta.inject.Singleton;
import java.util.function.Predicate;
@Singleton
public class OnAddFilter implements Predicate<V1ConfigMap> {
@Override
public boolean test(V1ConfigMap configMap) {
return configMap.getMetadata().getAnnotations() != null
&& configMap.getMetadata().getAnnotations().containsKey("io.micronaut.operator");
}
}onUpdateFilter
package micronaut.operator;
import io.micronaut.kubernetes.client.openapi.model.V1ConfigMap;
import jakarta.inject.Singleton;
import java.util.function.BiPredicate;
@Singleton
public class OnUpdateFilter implements BiPredicate<V1ConfigMap, V1ConfigMap> {
@Override
public boolean test(V1ConfigMap oldConfigMap, V1ConfigMap newConfigMap) {
return newConfigMap.getMetadata().getAnnotations() != null
&& newConfigMap.getMetadata().getAnnotations().containsKey("io.micronaut.operator");
}
}onDeleteFilter
NOTE: Missing snippet source `micronaut.operator.onDeleteFilter`.|
Important
|
Note that in case of onDeleteFilter the predicate receives the resource for the test method, but when the ResouceReconciler’s reconcile method is executed, the lister will return Optional.empty since the resource was already deleted. To properly reconcile the resource on it’s removal, use finalizers.
|
and to set the filters in @Operator annotation:
@Operator(informer = @Informer(apiType = V1ConfigMap.class),
onAddFilter = OnAddFilter.class,
onUpdateFilter = OnUpdateFilter.class,
onDeleteFilter = OnDeleteFilter.class)Programmatic creation of Controller
Use the bean ControllerFactory to create the Controller programmatically.
The ControllerFactory's createController methods require already created
SharedIndexInformer instances for the same kubernetes resources in the same namespaces as provided to createController methods.
package micronaut.operator;
import io.micronaut.context.annotation.Context;
import io.micronaut.kubernetes.client.openapi.informer.SharedIndexInformerFactory;
import io.micronaut.kubernetes.client.openapi.model.V1Secret;
import io.micronaut.kubernetes.client.openapi.operator.OperatorResourceLister;
import io.micronaut.kubernetes.client.openapi.operator.controller.ControllerFactory;
import io.micronaut.kubernetes.client.openapi.operator.controller.reconciler.Request;
import io.micronaut.kubernetes.client.openapi.operator.controller.reconciler.ResourceReconciler;
import io.micronaut.kubernetes.client.openapi.operator.controller.reconciler.Result;
import io.micronaut.kubernetes.client.openapi.resolver.NamespaceResolver;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Collections;
import java.util.Optional;
@Context
class SecretOperator {
private static final Logger LOG = LoggerFactory.getLogger(SecretOperator.class);
SecretOperator(SharedIndexInformerFactory sharedIndexInformerFactory, ControllerFactory controllerFactory, NamespaceResolver namespaceResolver) {
String namespace = namespaceResolver.resolveNamespace();
sharedIndexInformerFactory.sharedIndexInformerFor(V1Secret.class, namespace);
controllerFactory.createController(V1Secret.class, Collections.singleton(namespace), new SecretReconciler());
}
private class SecretReconciler implements ResourceReconciler<V1Secret> {
@Override
public Result reconcile(Request request, OperatorResourceLister<V1Secret> lister) {
Optional<V1Secret> secretOpt = lister.get(request);
LOG.info("Reconciling secret: {}", request);
// .. reconcile
return new Result(false);
}
}
}Note that informer and controller need to be created before the micronaut context has completed the startup process since those are started by InformerFactoryLifecycleListener and ControllersLifecycleListener.
Controllers created programmatically or via the @Operator annotation are running only on the leader instance. The leader instance is selected by the leader election background job.
Leader election
The LeaderElector is responsible for the leader election between application replicas. It runs as a background job on each application replica trying to acquire the lock. After the job has acquired the lock, it keeps renewing the lock. If the lock is not renewed within the specified amount of time, other replicas may try to acquire the lock and become the leader.
You can adjust the leader elector configuration by using Micronaut configuration properties kubernetes.client.operator.leader-election.lock:
Element |
Description |
|
The lock lease duration. Defaults to |
|
The lock renew deadline. If the LeaderElector fails to renew the lock within the deadline then the controller looses the lock. Defaults to |
|
The lock acquire retry period. Defaults to |
kubernetes.client.operator.leader-election.lock.lease-duration=60s
kubernetes.client.operator.leader-election.lock.renew-deadline=50s
kubernetes.client.operator.leader-election.lock.retry-period=20sEvents emitted by LeaderElector:
-
LeaseAcquiredEvent - emitted when this application replica has acquired the lock
-
LeaseLostEvent - emitted when this application replica has lost the lock
-
LeaderChangedEvent - emitted when the leader has changed
Lock identity
The lock identity is used to uniquely identify the application that holds the lock. By default, the Pod name the application runs within is the source for the lock identity. This means the application must run in the Kubernetes cluster.
To create a custom lock identity, create a bean that implements the LockIdentityProvider interface.
Lock resource types
The LeaderElector uses a native Kubernetes resource to store the lock information. Currently supported resources are:
-
V1Lease(default) -
V1ConfigMap -
V1Endpoints
The selected resource can be changed by configuring the property kubernetes.client.operator.leader-election.lock.resource-kind to configmaps or endpoints.
Note that the resource for the lock is created in the application namespace. Then the application name is used as the lock resource name. This can be changed by using Micronaut configuration properties:
kubernetes.client.operator.leader-election.lock.resource-name=custom-name
kubernetes.client.operator.leader-election.lock.resource-namespace=custom-namespaceNote that in case the RBAC authorization is enabled in your Kubernetes cluster, your application needs to have properly configured role with respect to the lock resource type.
V1Lease lock resource type:---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: operator-lease-role
rules:
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- get
- create
- update
---The Configuration import feature reads Kubernetes ConfigMap and Secret resources and makes them available as
PropertySource instances in your application.
To get started, declare one of the following dependencies:
-
If you want to use the official Kubernetes client:
implementation("io.micronaut.kubernetes:micronaut-kubernetes-discovery-client")-
If you want to use the OpenAPI Kubernetes client:
implementation("io.micronaut.kubernetes:micronaut-kubernetes-client-openapi-discovery")Then configure micronaut.config.import entries that point to the Kubernetes resources you want to load.
Once imported, you can read the configuration values using @Value or
any other way to read configuration values.
micronaut.config.import[0]=optional:kubernetes://config-map?name=my-config-map
micronaut.config.import[1]=optional:kubernetes://secret?name=my-secretUnlike the deprecated configuration client, configuration import is configured in application.yml
(or .properties, .json, etc) through micronaut.config.import.
When Kubernetes configuration import is used, Micronaut creates a separate ApplicationContext that is used only to
resolve properties from Kubernetes ConfigMap and Secret resources. This internal context is started specifically for
config import processing, uses the same environments that are active for the main application context but loads limited
set of beans that are required just for config import processing.
|
Warning
|
If you have a custom implementation of one of the following interfaces, those won’t be loaded when processing config imports:
|
ConfigMaps
Supported formats for ConfigMap resources are:
-
Java
.properties -
YAML
-
JSON
-
Literal values
To import a specific ConfigMap by name, add an entry such as:
micronaut.config.import[0]=optional:kubernetes://config-map?name=my-config-mapYou can import multiple config maps by repeating the entry:
micronaut.config.import[0]=optional:kubernetes://config-map?name=my-config-map
micronaut.config.import[1]=optional:kubernetes://config-map?name=other-config-mapIn addition to selecting a single config map by name, Kubernetes labels can be used to match multiple config maps:
micronaut.config.import[0]=optional:kubernetes://config-map?labels=app=my-app,env=prodConfig maps can also be selected by matching labels from the pod the Micronaut application is running in.
This is handy if you use a package manager like Helm.
A good example would be app.kubernetes.io/instance, which is a unique label identifying the instance of an application:
micronaut.config.import[0]=optional:kubernetes://config-map?podLabels=app.kubernetes.io/instanceAdditionally, you can enable exceptionOnPodLabelsMissing if you want an exception to be thrown when at least one of
the labels specified in podLabels is not found among the application’s pod labels. This can be useful if you want to
prevent a mistyped or misconfigured label selector from matching all config maps in the namespace:
micronaut.config.import[0]=optional:kubernetes://config-map?podLabels=app.kubernetes.io/instance&exceptionOnPodLabelsMissing=trueBy default if an exception occurs reading config maps on startup, the exception is logged and the service is started.
This can be overridden to terminate the service startup when the service is not able to read config maps because of any
exception by setting the terminateStartupOnException query parameter to true in the import URI:
micronaut.config.import[0]=optional:kubernetes://config-map?name=my-config-map&terminateStartupOnException=trueReading ConfigMap resources from mounted volumes
In the case of ConfigMap resources, reading them from the Kubernetes API requires additional permissions.
Therefore, you may want to read them from mounted volumes in the pod.
Given the following ConfigMap:
apiVersion: v1
kind: ConfigMap
metadata:
name: mounted-config
data:
mounted.yml: |-
foo: barIt can be mounted as a volume in a pod or deployment definition:
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: mypod
image: micronautapp
volumeMounts:
- name: configuration
mountPath: /etc/configuration
readOnly: true
volumes:
- name: configuration
configMap:
name: mounted-configThis makes Kubernetes create one file per data entry:
-
/etc/configuration/mounted.yml
To import that mounted file, use a standard Micronaut file import:
micronaut.config.import[0]=optional:file:///etc/configuration/mounted.ymlThe file format is deduced from the file suffix. Supported formats for mounted ConfigMap files are:
-
Java
.properties -
YAML
-
JSON
|
Note
|
Mounted volume imports and Kubernetes API imports are configured independently with configuration import.
If you want both, list both the |
Watching for changes in ConfigMaps
Watching for ConfigMap changes is enabled by default for Kubernetes configuration imports.
The following import explicitly enables it:
micronaut.config.import[0]=optional:kubernetes://config-map?name=my-config-map&watch=trueWhen watch is enabled, matching ConfigMap resources added, modified, or deleted in Kubernetes will be propagated to
the Micronaut Environment and refresh it.
This means those changes become available in your application without a restart.
To disable watching for ConfigMap changes, set watch=false in the import URI.
micronaut.config.import[0]=optional:kubernetes://config-map?name=my-config-map&watch=falseExamples
You can create a Kubernetes ConfigMap from an existing file with the following command:
kubectl create configmap my-config --from-file=my-config.properties
Or:
kubectl create configmap my-config --from-file=my-config.yml
Or:
kubectl create configmap my-config --from-file=my-config.json
You can also create a ConfigMap from literal values:
kubectl create configmap my-config --from-literal=special.how=very --from-literal=special.type=charm
Secrets
Secrets imported from the Kubernetes API are base64-decoded and made available as PropertySource instances, so that
they can also be read with @Value, @ConfigurationProperties, etc.
To import a specific secret by name, add an entry such as:
micronaut.config.import[0]=optional:kubernetes://secret?name=my-secretThe same way as for ConfigMap resources, you can import multiple secrets by repeating the entry:
micronaut.config.import[0]=optional:kubernetes://secret?name=this-secret
micronaut.config.import[1]=optional:kubernetes://secret?name=other-secretSimilarly to ConfigMap resources, labels can also be used to match the desired secrets:
micronaut.config.import[0]=optional:kubernetes://secret?labels=app=my-app,env=prodThis also works for pod labels:
micronaut.config.import[0]=optional:kubernetes://secret?podLabels=app.kubernetes.io/instanceAs well as exceptionOnPodLabelsMissing:
micronaut.config.import[0]=optional:kubernetes://secret?podLabels=app.kubernetes.io/instance&exceptionOnPodLabelsMissing=trueSimilar to ConfigMap resources, if you want to terminate the service startup when the service is not able to read
secrets because of any unexpected exception, you can set terminateStartupOnException to true in the import URI:
micronaut.config.import[0]=optional:kubernetes://secret?name=my-secret&terminateStartupOnException=trueReading Secret resources from mounted volumes
In the case of Secret resources, reading them from the Kubernetes API requires additional permissions.
Therefore, you may want to read them from mounted volumes in the pod.
Given the following secret:
apiVersion: v1
kind: Secret
metadata:
name: mounted-secret
type: Opaque
data:
application.properties: Zm9vPWJhcg==where decoded secret content is foo=bar
It can be mounted as a volume in a pod or deployment definition:
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: mypod
image: redis
volumeMounts:
- name: foo
mountPath: "/etc/foo"
readOnly: true
volumes:
- name: foo
secret:
secretName: mounted-secretThis makes Kubernetes create /etc/foo/application.properties file.
The file content is the decoded configuration text from the original base64-encoded values.
When secrets are imported through file: config import entries, the mounted file name must use one of the supported
configuration file extensions, such as .properties, .json, .yml, or .yaml. The secret value must then contain
content that matches that file format so Micronaut can parse it correctly.
For example, a mounted secret file named /etc/foo/application.yml should contain YAML content, while
/etc/foo/application.properties should contain Java properties content.
To import those mounted files, use standard Micronaut file imports:
micronaut.config.import[0]=optional:file:///etc/foo/application.properties|
Note
|
Mounted secret files and Kubernetes API imports can be combined by listing both |
Watching for changes in Secrets
Watching for Secret changes is enabled by default for Kubernetes configuration imports.
If watch is enabled, this configuration module will watch for matching Secret resources added, modified, or deleted,
and the changes will be propagated to the Environment and refresh it.
This means those changes become available in your application without a restart.
The following import explicitly enables it:
micronaut.config.import[0]=optional:kubernetes://secret?name=my-secret&watch=trueTo disable watching for Secret changes, set watch=false in the import URI.
micronaut.config.import[0]=optional:kubernetes://secret?name=my-secret&watch=falseIf you need to debug the Micronaut Kubernetes module, you need to set the io.micronaut.kubernetes logger level
to DEBUG:
<logger name="io.micronaut.kubernetes" level="DEBUG"/>By configuring the logger level to TRACE, the module will produce detailed responses from the Kubernetes API.
<logger name="io.micronaut.kubernetes" level="TRACE"/>Other package that might produce relevant logging is io.micronaut.discovery, which belongs to Micronaut Core.
In addition to that, another source of information is
the Environment Endpoint, which outputs all
the resolved PropertySources from ConfigMaps, and their corresponding properties.
Micronaut Kubernetes 8.x
The official Kubernetes Client upgrade
Micronaut Kubernetes 8.0.0 updates the official Kubernetes Client from v22 to v26.
RxJava2 support removal
Micronaut 5 doesn’t support RxJava2 so micronaut-kubernetes-client-rxjava2 module has been removed.
Delete object methods of Kubernetes Client OpenAPI client
In Kubernetes Client OpenAPI client, delete methods have been modified to return DeleteResponse instead of V1Status or type of object that is being deleted.
For example, deleteNamespace method of CoreV1Api class returned V1Status
@Delete("/api/v1/namespaces/{name}")
@Consumes({"application/json", "application/yaml", "application/vnd.kubernetes.protobuf", "application/cbor"})
V1Status deleteNamespace(
@PathVariable("name") @NotNull String name,
@QueryValue("pretty") @Nullable String pretty,
@QueryValue("dryRun") @Nullable String dryRun,
@QueryValue("gracePeriodSeconds") @Nullable Integer gracePeriodSeconds,
@QueryValue("ignoreStoreReadErrorWithClusterBreakingPotential") @Nullable Boolean ignoreStoreReadErrorWithClusterBreakingPotential,
@QueryValue("orphanDependents") @Nullable Boolean orphanDependents,
@QueryValue("propagationPolicy") @Nullable String propagationPolicy,
@Body @Nullable @Valid V1DeleteOptions body
);but now returns DeleteResponse<V1Namespace>
@Delete("/api/v1/namespaces/{name}")
@Consumes({"application/json", "application/yaml", "application/vnd.kubernetes.protobuf", "application/cbor"})
DeleteResponse<V1Namespace> deleteNamespace(
@PathVariable("name") @NotNull String name,
@QueryValue("pretty") @Nullable String pretty,
@QueryValue("dryRun") @Nullable String dryRun,
@QueryValue("gracePeriodSeconds") @Nullable Integer gracePeriodSeconds,
@QueryValue("ignoreStoreReadErrorWithClusterBreakingPotential") @Nullable Boolean ignoreStoreReadErrorWithClusterBreakingPotential,
@QueryValue("orphanDependents") @Nullable Boolean orphanDependents,
@QueryValue("propagationPolicy") @Nullable String propagationPolicy,
@Body @Nullable @Valid V1DeleteOptions body
);The same change has been applied to delete methods from Kubernetes Client OpenAPI Reactive client.
Utility methods
The configMapAsPropertySource methods of KubernetesUtils class require additional argument Collection<PropertySourceLoader> propertySourceLoaders which can be provided using getPropertySourceLoaders method from Environment.
Micronaut Kubernetes 7.x
Micronaut Kubernetes 7.0.0 updates the official Kubernetes Client from v19 to v22.
Starting from Kubernetes Client v20 the client api was refactored to use the builder pattern when creating client requests which caused breaking changes in all micronaut modules which are using the official kubernetes client. To support the builder pattern, the reactive client micronaut modules (Reactor, RxJava2 and RxJava3) are recreated and will also cause breaking changes.
Micronaut Kubernetes 4.x
Micronaut Kubernetes 4.0.0 updates to Kubernetes Client v18 (major). This major upgrade of the Kubernetes Client addresses several security CVEs.
Micronaut Kubernetes 3.x
This section documents breaking changes between Micronaut Kubernetes 2.x and Micronaut Kubernetes 3.x:
In-house Kubernetes client removed
The in-house Kubernetes client io.micronaut.kubernetes.client.v1.* was deprecated and removed. Instead use the new module micronaut-kubernetes-client or the reactive alternatives micronaut-client-reactor or micronaut-client-rxjava2 that extends the client API classes by the reactive support of respective reactive framework.
See the following list of guides to learn more about working with Kubernetes in the Micronaut Framework:
You can find the source code of this project in this repository: