thirumurthi

Istio - Kubernetes Gateway API in Kind Cluster

Thirumurthi S — Sun, 01 Sep 2024 20:29:28 GMT

Istio with Kubernetes Gateway API

In this blog have detailed deploying Istio in Kind cluster and configure Kubernetes Gateway API.

Pre-requisites:

Docker desktop installed and running
Kind CLI
Helm CLI (v3.15.3+)
Understanding of Service Mesh (Istio basics and Gateway API)

Istio with Kubernetes Gateway API

Istio documentation recommends to use Kubernetes Gateway API since the release of version v1.1.
In the Istio Ingress Gateway VirtualService is created for routing, with Kubernetes Gateway API to route the HTTPRoute is created that configures the service of the app.
To configure Kubernetes Gateway API in Kind cluster the Gateway API CRD should be deployed first. In this example the standard install is used. Referred from https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/standard-install.yaml.

Summary of components deployed

Kind cluster creation with configuration. Few ports are exposed to access the apps.
Istio is deployed using helm charts. Istio - base and istiod charts are deployed with basic configuration. (Not production ready configuration).
Kiali Server (Istio Dashboard) is deployed using helm charts, configured to use the external services of Prometheus and Grafana deployed on monitor namespace.
Kubernetes Gateway API CRD deployed to cluster.
Gateway API custom resource deployed to cluster.
HTTPRoute configured and deployed to access the Kiali from browser.
Prometheus community chart deployed which installs Prometheus, Grafana and AlertManager. All the components are deployed in monitor namespace. (Not production ready configuration). Refer Chart documentation
Simple backend-app (NGINX) deployed to serve simple JSON response.
HTTPRoute configured and deployed to access the backend-app.

Note
The Istio helm charts are downloaded to local machine and then deployed.
It can also be deployed directly once the repo is added to helm CLI. Refer Istio documentation.

Representation of the app deployed in the Kind cluster

Kind Cluster creation

Create Kind cluster named istio-dev with below configuration
Few ports exposed in the configuration
- Port 9001 and 9002 used to access the Prometheus and Grafana
- 8180 exposed to access the gateway from the host machine. Note, once the Gateway service is created it has to be edited to configure the port.

Kind configuration

kind: ClusterapiVersion: kind.x-k8s.io/v1alpha4nodes:- role: control-plane- role: worker  extraPortMappings:  - containerPort: 31000    hostPort: 8180    listenAddress: "127.0.0.1"    protocol: TCP  - containerPort: 31043    hostPort: 9443    listenAddress: "127.0.0.1"    protocol: TCP  - containerPort: 31100    hostPort: 9001    listenAddress: "127.0.0.1"    protocol: TCP  - containerPort: 31110    hostPort: 9002    listenAddress: "127.0.0.1"    protocol: TCP

Command to create kind cluster

If the above kind cluster configuration is stored in a file cluster_config.yaml, use below command.

kind create cluster --config cluster_config.yaml --name istio-dev

Install Istio

Download the charts locally

To add the repo to the helm for the Istio charts

helm repo add istio https://istio-release.storage.googleapis.com/chartshelm repo update

The Istio charts are download to charts/ folder. The Istio base and istiod charts to local with below commands

-- create and navigate charts foldermkdir chartscd charts/-- download install helm basehelm pull istio/base --untar-- download install helm istiodhelm pull istio/istiod --untar

Create namespace to deploy the Istio charts

To create istio-system namespace with below command, in which the Istio base and istiod charts will be deployed

kubectl create ns istio-system

Deploy the Istio base charts

Deploy Istio base charts to kind cluster.
The default revision is deployed in this case by passing default in value defaultRevision.

-- navigate to charts/basecd charts/basehelm upgrade -i istio-base . -n istio-system --set defaultRevision=default

Deploy the Istio istiod charts

Deploy Istio istiod, before deploying Istio CRD make sure the Istio base is deployed

-- deploy the Istio/istiod-- navigate to charts/istiodcd charts/istiodhelm upgrade -i istiod . -n istio-system --wait

Chart status check

Once Charts are installed the status can be checked using below helm command. Chart status should be deployed.

$ helm ls -n istio-systemNAME         NAMESPACE       REVISION   UPDATED                       STATUS          CHART                   APP VERSIONistio-base   istio-system    1          2024-09-01 08:10:11 -0700 PDT deployed        base-1.23.0             1.23.0istiod       istio-system    1          2024-09-01 08:10:28 -0700 PDT deployed        istiod-1.23.0           1.23.0

Deploy Kuberented Gateway API

To deploy the Gateway API CRD, use below command.

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/standard-install.yaml

Create and deploy Gateway API custom resource

The Kubenetes Gateway API custom resource YAML content looks like below (refer kuberenetes-gateway-api.yaml).
Note, in case of Istio Ingress Gateway the apiVersion would be networking.istio.io/v1alpha3.

# fileName: kuberenetes-gateway-api.yamlapiVersion: gateway.networking.k8s.io/v1kind: Gatewaymetadata:  name: gateway  namespace: istio-ingressspec:  gatewayClassName: istio  listeners:  - name: default    #hostname: "127.0.0.1"  # commented here but we can use "*.example.com" refer docs    port: 80    protocol: HTTP    allowedRoutes:      namespaces:        from: All

Deploy the Gateway API resource

Create namespace istio-ingress and deploy the Gateway resource, use below command.

-- create namespace commandkubectl create ns istio-ingress-- deploy the gateway api resourcekubectl apply -f kuberenetes-gateway-api.yaml

Once the Gateway is deployed, to validate and check the status of the service use below command.

kubectl -n istio-ingress get svc

Output might look like below with random nodeports assigned

NAME            TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                        AGEgateway-istio   LoadBalancer   10.96.201.160   <pending>     15021:32753/TCP,80:30526/TCP   8m20s

Edit the Gateway API service to update the nodePort to match Kind cluster configuration

Edit the Gateway service using kubectl -n istio-ingress edit svc/gateway-istio.
The nodePort(s) has to be updated like in below snippet.
The incoming traffic from port 8180 will be routed to 31000 nodePort, already updated in Kind cluster configuration.

    ipFamilies:    - IPv4    ipFamilyPolicy: SingleStack    ports:    - appProtocol: tcp      name: status-port      nodePort: 31021  #<--- update to 31021 port not used for this example      port: 15021      protocol: TCP      targetPort: 15021    - appProtocol: http      name: default      nodePort: 31000  # <--- update this port from any to 31000 configured in kind config      port: 80      protocol: TCP      targetPort: 80

Deploy Kiali Server

To deploy the Kiali server using helm chart, add the repo to helm CLI and download to local charts folder.

-- add repo to the helm clihelm repo add kiali https://kiali.org/helm-chartshelm repo update

Navigate to charts folder and download the charts to local.

cd charts/helm pull kiali/kiali-server --untar

Execute below command from the downloaded charts folder kaili-server.

Note :-
The external_service urls are configured to access the Prometheus and Grafana. These components will be deployed by single chart later.
The components are deployed to monitor namespace, hence the url pattern is <service-name>.<namespace>:port

helm upgrade -i kiali-server . \--set auth.strategy="anonymous" \--set external_services.prometheus.url="http://prometheus-operated.monitor:9090" \--set external_services.grafana.url="http://prometheus-grafana.monitor:80" \--set external_services.grafana.in_cluster_url="http://prometheus-grafana.monitor:80" \-n istio-system

Verify status of the chart deployment and the status might look like below

$ helm ls -n istio-systemNAME         NAMESPACE      REVISION   UPDATED                       STATUS          CHART                   APP VERSIONistio-base   istio-system    1         2024-09-01 08:10:11 -0700 PDT deployed        base-1.23.0             1.23.0istiod       istio-system    1         2024-09-01 08:10:28 -0700 PDT deployed        istiod-1.23.0           1.23.0kiali-server istio-system    1         2024-09-01 08:34:58 -0700 PDT deployed        kiali-server-1.89.0     v1.89.0

Create HTTPRoute resource to access Kiali server

Create HTTPRoute resource for kiali as shown in the below yaml content, save the config in a file named kiali-http-route.yaml

# filename: kiali-http-route.yamlapiVersion: gateway.networking.k8s.io/v1kind: HTTPRoutemetadata:  name: kiali-http  namespace: istio-systemspec:  parentRefs:  - name: gateway    namespace: istio-ingress  hostnames: ["127.0.0.1"]  # without hostname the service would not be accessible  rules:  - matches:    - path:        type: PathPrefix        value: /kiali    backendRefs:    - name: kiali      port: 20001

Deploy Kiali HTTPRoute

To deploy the kiali route use below command.

-- the resource will be deployed to istio-system namespace and uses the Gateway API previously deployedkubectl apply -f kiali-http-route.yaml

Accessing Kiali from browser

The address to access the Kiali UI is - http://127.0.0.1:8180/kiali

The Kiali UI looks like in below snapshot. The Nginx backend app, Prometheus and Grafana where deployed here in the snapshot.

Note :-Kiali UI might throw warning messages if the Prometheus chart is not deployed.

Deploy Prometheus

Add repo to helm local and download the chart to local charts folder
Note, below is not a production ready configuration, requires further hardening through configuration. Refer documentation.

helm repo add prometheus-community https://prometheus-community.github.io/helm-chartshelm repo updatecd charts/helm pull prometheus-community/kube-prometheus-stack --untar

Create namespace and deploy the charts to Kind cluster

To create namespace monitor and deploy Prometheus, use below set of commands

-- create namespacekubectl create ns monitor-- navigate to downloaded chartscd charts/kube-prometheus-stack-- deploy prometheushelm upgrade -i prometheus . \--namespace monitor \--set prometheus.service.nodePort=31100 \--set prometheus.service.type=NodePort \--set grafana.service.nodePort=31110 \--set grafana.service.type=NodePort \--set alertmanager.service.nodePort=31120 \--set alertmanager.service.type=NodePort \--set prometheus-node-exporter.service.nodePort=31130 \--set prometheus-node-exporter.service.type=NodePort

Status check of Prometheus chart deployment

Verify the status of chart deployment use the helm command mentioned below and the output of the status will be similar to below snippet.

$ helm ls -n monitorNAME        NAMESPACE   REVISION  UPDATED                       STATUS    CHART                           APP VERSIONprometheus  monitor     1         2024-09-01 08:43:38 -0700 PDT deployed  kube-prometheus-stack-62.3.1    v0.76.0

Access Prometheus and Grafana

Accessing Prometheus
- To access the Prometheus use http://127.0.0.1:9001/ or http://localhost:9001

Accessing Grafana
- To access the Grafana use http://127.0.0.1:9002/ or http://localhost:9002, when prompted use username: admin and password: prom-operator.

Creating the NGNIX backend app

Deploy the backend app to kind cluster

Below YAML content is resource defintion for the backend apps including namespace, service and deployment.
The namespace label istio-injection: enabled will automatically create the Istio Proxy when the backend pod is created.
Save the YAML content to a file named backend_app_deployment.yaml.

# filename: backend_app_deployment.yamlapiVersion: v1kind: Namespacemetadata:  name: backend-app  labels:    istio-injection: enabled---apiVersion: v1kind: ConfigMapmetadata:  name: backend-nginx-config  namespace: backend-appdata:  nginx.conf: |    worker_processes auto;    error_log stderr notice;    events {      worker_connections 1024;    }    http {      variables_hash_max_size 1024;      log_format main '$remote_addr - $remote_user [%time_local] "$request" '                      '$status $body_bytes_sent "$http_referer" '                      '"$http_user_agent" "$http_x_forwarded_for"';      access_log off;      real_ip_header X-Real-IP;      charset utf-8;      server {        listen 80;        location /greet {          default_type application/json;          return 200 '{"status":"OK","message":"Greetings!! from server","current_time":"$time_iso8601"}';        }      }    }---apiVersion: apps/v1kind: Deploymentmetadata:  name: backend-server  namespace: backend-app  labels:    app: backend-serverspec:  replicas: 1  selector:    matchLabels:      app: backend-server  template:    metadata:      labels:        app: backend-server    spec:      volumes:       - name: nginx-config         configMap:           name: backend-nginx-config           items:           - key: nginx.conf             path: nginx.conf      containers:      - name: backend-server        image: nginx        ports:        - containerPort: 80        volumeMounts:         - name: nginx-config           mountPath: /etc/nginx        resources:          requests:             memory: "128Mi"            cpu: "250m"          limits:            memory: "256Mi"            cpu: "500m"---apiVersion: v1kind: Servicemetadata:  name: backend-svc  namespace: backend-appspec:  selector:    app: backend-server  ports:    - name: tcp-port      protocol: TCP      port: 8081      targetPort: 80---

To deploy the backend app issue below command

-- the namespace will be created note, the label istio-injection is enabled which will create a envoy proxy sidecar automatically kubectl apply -f backend_app_deployment.yaml

Define HTTPRoute for the backend app

To access the backend we need to define a HTTPRoute like below and save it to a file named app-httproute.yaml

# app-httproute.yamlapiVersion: gateway.networking.k8s.io/v1kind: HTTPRoutemetadata:  name: http  namespace: backend-appspec:  parentRefs:  - name: gateway    namespace: istio-ingress  hostnames:    - "127.0.0.1"  rules:  - matches:    - path:        type: PathPrefix        value: /greet    backendRefs:    - name: backend-svc      port: 8081

To deploy the HTTPRoute use below command.

kubectl apply -f app-httproute.yaml

Access the Backend app API

From browser use http://127.0.0.1:8180/greet should see the response like below

]]>

Observability - Jaeger Distributed Tracings

Thirumurthi S — Mon, 26 Aug 2024 00:06:10 GMT

Create trace and spans manually in spring boot applications

Pre-requisites:

Jaeger installed and running (in this example deployed in kind cluster)
Understanding on traces and spans
Understanding on micrometer Spring configuration

Requirement

This blog details on how to create spans manually in Spring Boot application. This is based on a requirement in a project, to track the flow between application using Jaeger. To create the traces and spans manually so it can visualized in Jaeger UI and helps the business process on how much time it takes.

In order to demonstrate we have two Spring Boot application, use the tracer object configured in Spring boot to create the spans. The spans set with name with additional tags which is key value pair which can be viewed in Jaeger UI. In the example, the application named invoker-app exposes a REST API (/api/v2/execute), when invoked calls the REST API (/app/run?traceId=xxx&spanId=yyy) of second application named app-1. The app-1 application uses the traceId and spanId to create a trace context and adds to the current tracer.

The code snippet below is in app-1 application which creates traceContext with the traceId and spanId. The created traceContext is set to the configured tracer objects current Tracer Context scope.

var contextWithCustomTraceId = tracer.traceContextBuilder()                    .traceId(traceId)                    .spanId(spanId)                    .sampled(true)                    .build();// use traceContext as a newScopetry (var sc = tracer.currentTraceContext().newScope(contextWithCustomTraceId)) { // create spans process any operation}

Representation of the REST API invocation between the apps, in this case invoker-app and app-1.

The expected flow from Jager UI looks like below, where the invoker-app span and app-1 span can be seen as child spans.

Span with the tag added during creation.

Code

Invoker app

Below is the pom.xml with dependencies. Note the dependencies are same for both invoker-app and app-1 only change is the name of the apps.When using Spring starter create two projects, just copy paste the dependencies section.

<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">    <modelVersion>4.0.0</modelVersion>    <parent>        <groupId>org.springframework.boot</groupId>        <artifactId>spring-boot-starter-parent</artifactId>        <version>3.3.2</version>        <relativePath/>    </parent>    <groupId>com.trace</groupId>    <artifactId>invoker</artifactId>    <version>0.0.1-SNAPSHOT</version>    <name>invoker</name>    <description>Demo project for Spring Boot</description>    <url/>    <licenses>        <license/>    </licenses>    <developers>        <developer/>    </developers>    <scm>        <connection/>        <developerConnection/>        <tag/>        <url/>    </scm>    <properties>        <java.version>22</java.version>    </properties>    <dependencies>        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-starter</artifactId>        </dependency>        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-starter-actuator</artifactId>        </dependency>        <dependency>            <groupId>org.projectlombok</groupId>            <artifactId>lombok</artifactId>            <optional>true</optional>        </dependency>        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-starter-web</artifactId>        </dependency>        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-starter-test</artifactId>            <scope>test</scope>        </dependency>        <dependency>            <groupId>io.micrometer</groupId>            <artifactId>micrometer-core</artifactId>        </dependency>        <dependency>            <groupId>io.micrometer</groupId>            <artifactId>micrometer-tracing-bridge-brave</artifactId>        </dependency>        <dependency>            <groupId>io.micrometer</groupId>            <artifactId>micrometer-tracing</artifactId>        </dependency>        <dependency>            <groupId>io.zipkin.brave</groupId>            <artifactId>brave-instrumentation-okhttp3</artifactId>        </dependency>        <dependency>            <groupId>io.zipkin.reporter2</groupId>            <artifactId>zipkin-sender-urlconnection</artifactId>        </dependency>        <dependency>            <groupId>io.zipkin.reporter2</groupId>            <artifactId>zipkin-reporter-brave</artifactId>        </dependency>    </dependencies>    <dependencyManagement>        <dependencies>            <dependency>                <groupId>io.micrometer</groupId>                <artifactId>micrometer-tracing-bom</artifactId>                <version>${micrometer-tracing.version}</version>                <type>pom</type>                <scope>import</scope>            </dependency>            <dependency>                <groupId>io.zipkin.brave</groupId>                <artifactId>brave-bom</artifactId>                <version>${brave.version}</version>                <type>pom</type>                <scope>import</scope>            </dependency>        </dependencies>    </dependencyManagement>    <build>        <plugins>            <plugin>                <groupId>org.springframework.boot</groupId>                <artifactId>spring-boot-maven-plugin</artifactId>                <configuration>                    <excludes>                        <exclude>                            <groupId>org.projectlombok</groupId>                            <artifactId>lombok</artifactId>                        </exclude>                    </excludes>                </configuration>            </plugin>        </plugins>    </build></project>

The application.yaml for the spring boot application, which includes the tracing endpoint url

spring.application.name: invokerserver.port: 8080management:  server.port: 9145  tracing:    sampling:      probability: 1.0  zipkin:    tracing:      endpoint: 'http://localhost:9411/api/v2/spans'

Invoker app controller

The invoker app uses the RestTemplate to create the REST API GET request to access the app-1 app.
The traceId and spandId is obtained from the tracer object configured and created by the Spring boot application during deployment.
The use of Random class to induce additional random delay after the span is created just for demonstration to view it in Jaeger UI.```javapackage com.trace.invoker;

import io.micrometer.tracing.Span;import io.micrometer.tracing.Tracer;import lombok.extern.slf4j.Slf4j;import org.springframework.beans.factory.annotation.Qualifier;import org.springframework.boot.web.client.RestTemplateBuilder;import org.springframework.http.*;import org.springframework.web.bind.annotation.GetMapping;import org.springframework.web.bind.annotation.RequestMapping;import org.springframework.web.bind.annotation.RestController;import org.springframework.web.client.RestTemplate;

import java.util.Random;import java.util.concurrent.Callable;import java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;import java.util.concurrent.TimeUnit;

@RestController@RequestMapping("/api/v2/")@Slf4jpublic class AppInvokerController {

private final Tracer tracer;

AppInvokerController(Tracer tracer){ this.tracer = tracer; }

@GetMapping("/execute") public String invokeTask() {

// new span Span span1 = tracer.nextSpan().name("invoker-api-parent").start(); // URL for second app app-1 application String APPURL = "http://localhost:8082/app/v1/run?traceId=%s&spanId=%s"; String traceId = tracer.currentSpan().context().traceId(); String spanId = span1.context().spanId(); String url = String.format(APPURL,traceId,spanId);

RestTemplate clientTemplate = new RestTemplate(); log.info("tracer invoked app-1 invoked");

HttpHeaders httpHeaders = new HttpHeaders(); httpHeaders.setContentType(MediaType.TEXT_PLAIN);

HttpEntity request = new HttpEntity<>("",httpHeaders);

try (Tracer.SpanInScope spanInScope = tracer.withSpan(span1.name("invoke-api").start())) { span1.tag("appName","invokeApi"); span1.event("invoked from invoker api"); ResponseEntity response = clientTemplate.postForEntity(url,null,String.class); log.info("response {}",response.getBody());

}finally { span1.end(); } // New span- just for example to demonstrate the last step in invoker-app Span span2 = tracer.nextSpan().name("last step").start(); span2.event("completed"); span2.tag("app","invoker"); try{ Thread.sleep(700); } catch (InterruptedException e) { log.warn("Interrupted exception"); } span2.end(); return "completed"; }

}

- RestTemplate bean is used by the tracer object to publish the trace and span to Jaeger server.```javapackage com.trace.invoker;import org.springframework.boot.web.client.RestTemplateBuilder;import org.springframework.context.annotation.Bean;import org.springframework.web.client.RestTemplate;public class AppConfig {    @Bean    RestTemplate restTemplate(RestTemplateBuilder builder){        return builder.build();    }}

Application entry point usually class created by the spring starter itself.

package com.trace.invoker;import org.springframework.boot.SpringApplication;import org.springframework.boot.autoconfigure.SpringBootApplication;import org.springframework.context.annotation.Bean;@SpringBootApplicationpublic class InvokerApplication {    public static void main(String[] args) {        SpringApplication.run(InvokerApplication.class, args);    }}

app-1 application

The pom.xml with dependencies is same as the invoker-app dependencies. The only change would be the name and artifactId.
The app-1 controller code,
- With the provided traceId and spandId a new tracerContext is build and added to the current tracer context scope.
- The sampleThreadInvocation() uses Task Callable class (note, we can use Runnable as well) to run set of thread in parallel and create child spans with provided parent span
- The additionalProcess() method demonstrates creating the spans outside the thread.

package com.trace.app.one;import io.micrometer.tracing.Span;import io.micrometer.tracing.Tracer;import lombok.extern.slf4j.Slf4j;import org.springframework.web.bind.annotation.*;import java.util.Objects;import java.util.Random;import java.util.concurrent.Callable;import java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;import java.util.concurrent.TimeUnit;@RestController@RequestMapping("/app/v1")@Slf4jpublic class AppController {    private final Tracer tracer;    Random random = new Random();    AppController(Tracer tracer){        this.tracer = tracer;    }    @PostMapping("/run")    public String process(@RequestParam String traceId,                          @RequestParam String spanId){        // traceId and spanId can't be null        if(!Objects.isNull(traceId) && !Objects.isNull(spanId) ) {            log.info("traceId included - {}", traceId);           // create trace context with teh traceId and spanId            var contextWithCustomTraceId = tracer.traceContextBuilder()                    .traceId(traceId)                    .spanId(spanId)                    .sampled(true)                    .build();            // set the tracercontext to current trace context scope            try (var sc = tracer.currentTraceContext().newScope(contextWithCustomTraceId)) {               var span =  tracer.spanBuilder().name("app-one-tracing").start();                try(Tracer.SpanInScope spanInScope= tracer.withSpan(span)) {                    span.tag("app", "run invoked");                    log.info("tracing logs...");                    sleep(random.nextInt(750));                    sampleThreadInvocation();                    additionalProcess("process-1");                    additionalProcess("process-2");                }finally {                    span.end();                }            }        }        return "completed invocation";    }    void additionalProcess(String name){        Span span = tracer.nextSpan(tracer.currentSpan());        try(Tracer.SpanInScope spanInScope= tracer.withSpan(span.name(name).start())){            SpanHelper spanHelper = new SpanHelper();            Span innerSpan = spanHelper.createSpan(tracer,span,"ap-"+name,"proceed");            sleep(random.nextInt(1100));            spanHelper.endSpan(innerSpan);            spanHelper.endSpan(span);        }    }    void sampleThreadInvocation(){        log.info("api invoked..");        try(ExecutorService executor = Executors.newFixedThreadPool(5)) {            Span span = tracer.currentSpan();//.name("api-parent").start();//currentSpan();            for (int i = 0; i < 10; i++) {                try (Tracer.SpanInScope spanInScope = tracer.withSpan(span)) {                    Callable<Void> caller = new Task(tracer, span, "task" + i, random.nextInt(750), true);                    executor.submit(caller);                }            }            executor.shutdown();            try {                if (!executor.awaitTermination(60, TimeUnit.SECONDS)) {                    executor.shutdownNow();                }            } catch (InterruptedException interruptedException) {                executor.shutdownNow();            }        }        log.info("completed");    }    void sleep(int duration){        try {            Thread.sleep(duration);        } catch (InterruptedException e) {            log.warn("sleep interrupted");        }    }}

Below is Task class just used to demonstrate the use of parallel threads and span creation.

package com.trace.app.one;import io.micrometer.tracing.Span;import io.micrometer.tracing.Tracer;import lombok.extern.slf4j.Slf4j;import java.util.concurrent.Callable;@Slf4jpublic class Task implements Callable<Void> {    private final Tracer tracer;    SpanHelper spanHelper = new SpanHelper();    private final Span span;    private final String name;    private final int timeOut;    private final boolean useSpan;    public Task(Tracer tracer, Span span, String name, int timeOut, boolean useSpan){        this.tracer =tracer;        this.span = span;        this.name = name;        this.timeOut = timeOut;        this.useSpan = useSpan;    }    @Override    public Void call() throws Exception{        Span childSpan;        if(useSpan){            childSpan = spanHelper.createSpan(tracer,span,Thread.currentThread().getName(),name);        }else{            childSpan = spanHelper.createSpan(tracer,Thread.currentThread().getName(), name);        }        log.info("Running : {}",Thread.currentThread().getName());        Thread.sleep(timeOut);        spanHelper.endSpan(childSpan);        return null;    }}

SpanHelper class is a helper class used for creating the spans either with provides parent span or new span.

package com.trace.app.one;import io.micrometer.tracing.Span;import io.micrometer.tracing.Tracer;import lombok.extern.slf4j.Slf4j;@Slf4jpublic class SpanHelper {    Span createSpan(Tracer tracer, String spanName, String tags){        log.info("create new span..");        Span span = tracer.nextSpan(tracer.currentSpan());        span.name(spanName+"-"+tags);        span.event("starting "+spanName);        span.start();        return span;    }    Span createSpan(Tracer tracer, Span parentSpan, String spanName, String tags){        log.info("create new child span..");        Span span = tracer.nextSpan(parentSpan);        span.name(spanName+"-"+tags);        span.event("starting "+spanName);        span.start();        return span;    }    void endSpan(Span span){        if(span != null){            log.info("closing the span {}",span);            span.end();        }    }}

The entry point of the app-1 application, usually generated by the spring boot starter

package com.trace.app.one;import org.springframework.boot.SpringApplication;import org.springframework.boot.autoconfigure.SpringBootApplication;@SpringBootApplicationpublic class AppOneApplication {    public static void main(String[] args) {        SpringApplication.run(AppOneApplication.class, args);    }}

application.yaml configuration for the app-1 application

spring.application.name: app-1server.port: 8082management:  zipkin:    tracing:      endpoint: 'http://localhost:9411/api/v2/spans'

Jaeger in Kind cluster

Use the Kind cluster yaml configuration to create the cluster
Deploy Jaeger to the cluster
Port forward the necessary ports so spring boot application can send traces and spans.
Save the below kind configuration yaml as jaeger-cluster.yaml.
Use the command kind create cluster --config jaeger-cluster.yaml to create the cluster, make sure the docker desktop is running.

kind: ClusterapiVersion: kind.x-k8s.io/v1alpha4name: jaeger-clusternodes:- role: control-plane  extraPortMappings:  - containerPort: 31443    hostPort: 7443    protocol: TCP

Use below set of command to deploy the Jaeger in Kind cluster, we use jaeger-all-in-one image which is not production ready.
Download the cert-manager from https://github.com/cert-manager/cert-manager/releases/download/v1.15.3/cert-manager.yaml to cert-manager-v1.15.3.yaml. Use kubectl apply command below to deploy to kind cluster

kubectl apply -f cert-manager_v1.15.3.yaml

Create the namespace with below command
```
kubectl create namespace observability
```
Download the Jaeger operator yaml from https://github.com/jaegertracing/jaeger-operator/releases/download/v1.60.0/jaeger-operator.yaml and save it as jaeger-operator.yaml. Use below command to deploy to kind cluster

kubectl apply -f jaeger-operator.yaml

With the below jaeger-all-in-one configuration create a yaml file named jaeger_allinone.yaml

apiVersion: jaegertracing.io/v1kind: Jaegermetadata:  name: my-jaegerspec:  strategy: allInOne # <1>  allInOne:    image: jaegertracing/all-in-one:latest # <2>    options: # <3>      log-level: debug # <4>  storage:    type: memory # <5>    options: # <6>      memory: # <7>        max-traces: 100000  ingress:    enabled: false # <8>  agent:    strategy: DaemonSet # <9>  annotations:    scheduler.alpha.kubernetes.io/critical-pod: "" # <10>

Use below command to deploy the Jaeger all in one server.
```
kubectl apply -f jaeger_allinone.yaml
```

Port forward using below commands

# below ports will be used by the spring boot application to send traces and spanskubectl port-forward svc/my-jaeger-collector 9411:9411 14250:14250 14267:14267 14269:14269 4317:4317 4318:4318# below is to access the jaeger UI from localkubectl port-forward svc/my-jaeger-query 16686:16686

Accessing the application

With the spring boot applications running and Jaeger deployed, use http://localhost:8080/api/v2/execute to create traces.
The Jaeger UI can be viewed using http://localhost:16686/

]]>

Access Kubernetes Secrets In SpringBoot Application

Thirumurthi S — Thu, 04 Jul 2024 18:35:37 GMT

Access Kubernetes secrets in Spring Boot application without loading to environment variables of the pod.

This blog will show how to access Kubernetes secrets in a Spring application without setting properties as environment variable in the container.

To better understand this blog, basic knowledge of Spring framework and Kubernetes (especially how to create secrets and mount secrets to access as environment variables in Pods) is recommended.

Pre-requisites:

Docker desktop
Kind CLI

Spring configuration management

Spring Boot framework already provides multiple way to externalize the configuration. Instead of specifying the properties in application.yaml we can created as environment variable. For example, say if we need to enable a specific Spring profile the environment variable SPRING_PROFILES_ACTIVE can be set with specific profile when running the application.

Access Kubernetes secret in Spring Boot application without configuring to environment variable

In this approach the secret will be read by the Spring Boot application without loading to the environment variables.
- The secret is mount in the pod manifest
- The mount path of the secret is configured in Spring boot application with spring.config.import.

Demonstration of configuration management

Create a simple secret resource named secret-msg in Kubernetes cluster. Use below command

kubectl create secret generic secret-msg --from-literal=MESSAGE=from-k8s-secret-store

The created secret is mounted in Pod manifest. Below is the snippet of how secret is mounted.

  volumeMounts:       - name: secret-volume          mountPath: /etc/secrets/          readOnly: true

The Spring Boot application.yaml file is configured with the mounted secret path.

spring.config.import: "optional:configtree:/etc/secrets/"

In Spring Boot controller, we can access the secret using the Value annotation.

@Value("${message:default-from-app}")private String message;

The Spring Boot application code and Kubernetes manifests

Use Spring Boot starter to create the Spring Boot application, include spring-boot-starter-web dependency. Extract the zip file and configure in preferred IDE. Following are additional files to set the Spring Boot application.
application.yaml content

spring:   application.name: configtree   config.import: "optional:configtree:/etc/secrets/"server.port: 8095

The controller class includes @value annotation to read value from the message property, it is set with default value. This helps identify which values is being fetched when the application is running.

import org.springframework.beans.factory.annotation.Value;import org.springframework.web.bind.annotation.GetMapping;import org.springframework.web.bind.annotation.RequestMapping;import org.springframework.web.bind.annotation.RestController;@RestController@RequestMapping("/v1")public class ConfigController {    private String RESPONSE= "{\"secret-msg\": \"%s\"}";    @Value("${message:default-from-app}")    private String message;    @GetMapping("message")    public String getMessage(){        return String.format(RESPONSE,message);    }}

To create docker image use the below content in the Dockerfile. Before using the Dockerfile to create the image the jar must be created. Note, this is not the ideal approach for production.
The configtree is the project name used for demonstration.

FROM openjdk:22-jdk-slimVOLUME /tmpCOPY target/configtree-1.0.0.jar app.jarEXPOSE 8091ENTRYPOINT ["java", "-jar", "/app.jar"]

To build the docker image use below command, place the Dockerfile in the root of the Spring boot project application.

docker build -t configtree-app .

The kind configuration used for this demonstration.

kind: ClusterapiVersion: kind.x-k8s.io/v1alpha4name: configtreenodes:  - role: control-plane    extraPortMappings:      - containerPort: 30095        hostPort: 8095

Command to create the kind cluster. Make sure the Docker desktop is running and issue below command

kind create cluster --config deploy/kind-config.yaml

The image is not published to Docker hub, in order to use the image in Kind cluster it needs to be loaded using below command. The cluster name in this case is configtree and the image name is configtree-app

kind load docker-image configtree-app --name configtree

Kubernetes deployment manifest for the spring boot application looks like below.
- Note, the mounted secret path is configured in the application yaml.

apiVersion: apps/v1kind: Deploymentmetadata:  name: configtree-app  labels:    app: configtree-appspec:  replicas: 1  selector:    matchLabels:      app: configtree-app  template:    metadata:      labels:        app: configtree-app    spec:      containers:        - name: configree          image: configtree-app:latest          imagePullPolicy: IfNotPresent           volumeMounts:            - name: secret-volume              mountPath: /etc/secrets/          # secret mount path              readOnly: true          ports:            - containerPort: 8095      volumes:        - name: secret-volume          secret:            secretName: secret-msg      # name of the created secret

NodePort service, this helps us to access the application running in the pod without port-forwarding.

apiVersion: v1kind: Servicemetadata:  name: configtree-svcspec:  type: NodePort  selector:    app: configtree-app  ports:    - name: configtree-port      protocol: TCP      port: 8095      targetPort: 8095      nodePort: 30095

Once deployed the application can be accessed which will render the message value from secret.

curl localhost:8095/v1/message{"secret-msg": "from-k8s-secret-store"}

Other approach to load secret to Pod environment variables

There are other options in Kubernetes to configure the secret as environment variable to the pod. Refer Kubernetes documentation.
Pod manifest to load all secret to Pod environment variable using secretRef.

apiVersion: v1kind: Podmetadata:  name: secret-refspec:  containers:  - name: envars-container    image: nginx    envFrom:    - secretRef:        name: secret-msg

Alternatively we can use secretKeyRef to set as environment variable once specific key in the secret.

apiVersion: v1kind: Podmetadata:  name: secrets-key-refspec:  containers:  - name: envars-container    image: nginx    env:    - name: MESSAGE      valueFrom:        secretKeyRef:          name: secret-msg          key: MESSAGE

]]>

Apache Apisix and Zitadel OIDC provider

Thirumurthi S — Sun, 12 May 2024 06:24:55 GMT

Deploying Apache Apisix and Zitadel in Kubernetes

In this blog will be demonstrating how to setup Apisix and Zitadel in Kind Kubernetes cluster for development.

Pre-requisites:

Software to be installed

Docker desktop
Kind
Helm
Kubectl

Basic understanding on Gateway and OpenId Connect.

What is Apache Apisix?

Apache Apisix is opensource API Gateway based on Nginx and etcd.
In this demonstration Apache Apisix is installed to cluster using helm chart, along with Apisix Dashboard and Apisix Ingress controller.
This is only for development and not production grade. For more info refer the Apache Apisix documentation.

What is Zitadel?

Zitadel provides identity management service along with authentication management. Zitadel can also be installed as standalone application, refer Zitadel documentation.
In this demonstration Zitadel application is installed with helm chart along with UI dashboard.
Prior to deploying Zitadel application the define a domain name, this value should be set in the configuration ExternalDomain either in override values or passed as helm command.
Zitadel can be installed with either Cockroach db or Postgres db. In this demonstartion Postgres db is used.
The Externaldomain value is also used as part of default username and for password check the documentation.

Installing Kind Cluster

Below yaml defines the Kind cluster configuration, with Docker desktop running we can create the the cluster with this configuration.

Note :-
The ingress option is enabled in the kind cluster.
Apisix exposes the Gateway service as NodePort or Loadbalancer. In here NodePort service is used and port 30080 is exposed.
By configuring the ports in extraPortMappings the Apisix Dashboard can be accessed doesn't require port-forwarding.

# filename: kind-cluster.yamlkind: ClusterapiVersion: kind.x-k8s.io/v1alpha4name: api6-zitadelnodes:- role: control-plane  kubeadmConfigPatches:  - |    kind: InitConfiguration    nodeRegistration:      kubeletExtraArgs:        node-labels: "ingress-ready=true"  extraPortMappings:  - containerPort: 30080    hostPort: 80    protocol: TCP  - containerPort: 30443    hostPort: 443    protocol: TCP

Save the content of kind configuration to a file named kind-cluster.yaml and use below command to create cluster.

kind create cluster --config kind-cluster.yaml

Installing Apisix

To install Apsix with helm charts, add the helm repo to local and issue update command like below.

helm repo add apisix https://charts.apiseven.comhelm repo update

In here we pull the chart to local and use it. To download the latest chart to local use the command below.

helm pull apisix/apisix --untar

Alternatively, you can download the helm charts from Apisix Helm Chart repo, requires understanding on helm charts.

In the downloaded helm chart update the chart version of etcd, apisix-dashboard and apisix-ingress-controller in chart.yaml file to latest version (below is the version number at the time if this writing)

After updating the version issue the command helm dependency update from the chart directory, helm will pull the specified version of dependent chart under the charts folder.

appVersion: 3.9.1sources:  - https://github.com/apache/apisix-helm-chartdependencies:  - name: etcd    version: 10.0.4    repository: https://charts.bitnami.com/bitnami    condition: etcd.enabled  - name: apisix-dashboard    version: 0.8.2    repository: https://charts.apiseven.com    condition: dashboard.enabled    alias: dashboard  - name: apisix-ingress-controller    version: 0.14.0    repository: https://charts.apiseven.com    condition: ingress-controller.enabled    alias: ingress-controller

Custom override values yaml for Apisix chart

The override values file for Apisix deploymentis used for this deployment. Note, only few configuration is updated like image tag name of initContainer, http service ports for Apisix service. Refer the documentation for mode customization, but for local development this should suffice.

The Apisix discovery registry is enabled with Kubernetes cluster. There are other registry option which can be configured, refer documentation.

# filename: apisix-values.yamlimage:  repository: apache/apisix  pullPolicy: IfNotPresent  tag: 3.8.0-debianreplicaCount: 1initContainer:  image: busybox  tag: latestserviceAccount:  create: truerbac:  create: trueservice:  type: NodePort  externalTrafficPolicy: Cluster  externalIPs: []  http:    enabled: true    servicePort: 80    containerPort: 9080    nodePort: 30080   # port used in kind cluster configuration  tls:    servicePort: 443    nodePort: 30443apisix:  enableIPv6: true  enableServerTokens: true  admin:    enabled: true    type: ClusterIP    externalIPs: []    ip: 0.0.0.0    port: 9180    servicePort: 9180    cors: true    credentials:      admin: edd1c9f034335f136f87ad84b625c8f1  # Default value as in values yaml      viewer: 4054f7cf07e344346cd3f287985e76a2  discovery:   enabled: true   registry:     kubernetes:        service:          schema: http           host: ${KUBERNETES_SERVICE_HOST}          port: ${KUBERNETES_SERVICE_PORT}externalEtcd:  host:    - http://etcd.host:2379  user: root  password: ""  existingSecret: ""  secretPasswordKey: "etcd-root-password"etcd:  enabled: true  prefix: "/apisix"  timeout: 30  auth:    rbac:      create: false      rootPassword: ""    tls:      enabled: false      existingSecret: ""      certFilename: ""      certKeyFilename: ""      verify: true      sni: ""  service:    port: 2379  replicaCount: 3# Apisix Dashboard configurationdashboard:  enabled: true  image:    pullPolicy: IfNotPresent  config:    conf:      etcd:        # -- Supports defining multiple etcd host addresses for an etcd cluster        endpoints:          - apisix-etcd:2379        prefix: "/apisix"        username: ~        password: ~# -- Ingress controller configurationingress-controller:  enabled: true  image:    pullPolicy: IfNotPresent  config:    apisix:      adminAPIVersion: "v3"

Save the above content in a file apisix-values.yaml as override values yaml within the apisix chart directory.
Below command will install Apisix, Apisix Dashboard and Apisix Ingress Controller.

helm upgrade --install apisix -f apisix-values.yaml . \    --set service.type=NodePort \    --set service.http.nodePort=30080 \    --set ingress-controller.enabled=true \    --set dashboard.enabled=true \    --create-namespace \    --namespace ingress-apisix \    --set ingress-controller.config.apisix.serviceNamespace=ingress-apisix

Note :-
Not all the configuration in override values yaml is customized, compare it with default values yaml file.
Also, the line --set service.type=NodePort --set service.http.nodePort=30080 in helm command can be removed since they are already added override yaml.

To check the status of Apisix deployments use below command.

kubectl -n ingress-apisix get pods,svc

Install Ingress route for Apisix dashboard

Once the Apisix ingress controller pod is in running state,Apisix dashboard ingress route can be configured to access from browser.

Info :-
Apisix Ingress controller uses CRD's to resolve Apisix routes and also resolves Ingress resource definition. It creates route and registers in apisix application as well.

The ingress configuration is similar to the Kubernetes Ingress resources, in below yaml file any traffic from http://apisix.localhost/ will be routed to apisix-dashboard service to port 80.

#filename:  apisix-dashboard-ingress.yamlapiVersion: networking.k8s.io/v1kind: Ingressmetadata:  name: api6-dashboard-ingress  namespace: ingress-apisix  # The namespace where the apisix is installedspec:  ingressClassName: apisix  rules:  - host: apisix.localhost    http:      paths:      - backend:          service:            name: apisix-dashboard            port:              number: 80        path: /        pathType: Prefix

Save the ingress configuration in apisix-dashboard-ingress.yaml and use below command to deploy.

kubect apply -f apisix-dashboard-ingress.yaml

Configure hosts file with domain name for loop back address

Open the hosts file and add below domains for loop back address (127.0.0.1)

127.0.0.1 localhost apisix.localhost zitadel.local backend.localhost

Now, open a browser and we should be able to access the Apisix dashboard.

To login the user name and password is admin

The configured routes will be displayed in the dashbord like

Info :-
Installing Apisix ingress controller is optional, if not installed the route configuration should be created and updated manually.
Below is optional only if Apisix ingress controller is NOT installed. To update the route configuration manually to Apisix.
Port forward the apisix-admin service use command - kubectl -n ingress-apisix port-forward svc apisix-admin 9180:9180
Obtain the admin key from the override values yaml under apisix.admin.credentials.admin. In this demonstration this default value as mentioned in Apisix documentation.
Use Curl command to configure route. Below is the curl command to configure Apisix dashboard route.

  curl http://127.0.0.1:9180/apisix/admin/routes/1 \      -H "X-API-KEY: $admin_key" -X PUT -i -d '      {        "name": "apisix-dashboard-route",        "desc": "Route for apisix dashboard",        "labels": {"created-by": "user"},        "uris": ["/","/*"],        "host": "apisix.localhost",        "upstream": {            "name": "apisix-dashboard-upstream",            "desc": "upstream for apisix dashboard route",            "labels": {"created-by": "user"},            "type": "roundrobin",            "hash_on": "vars",            "scheme": "http",            "pass_host": "pass",            "nodes": [{"host":"apisix-dashboard.ingress-apisix","port":80, "weight": 100 }],            "timeout": { "connect": 60,"send": 60,"read": 60 }          }      }'

Apisix route configuration representation

Sample Apisix Dashboard route view showing all three parts route, plugins and upstream in route configuration.

Installing Zitadel

Zitadel installed with secure Postgres, below are the steps involved

Installing certificates
Installing Postgres DB
Installing Zitadel

In order to configure the standalone Zitadel we need to define the ExternalDomain and ExternalPort in the Zitadel override yaml configuration.

Note :-
Latter in this blog, we configure a Nginx backend application to be accessed using Apisix route with oidc-connector plugin to connect to Zitadel.
The Apisix pod tries to discover the Zitadel OIDC url within the pod which does not resolve http://zitadel.localhost. As a workaround, have created the helm release with the name zitadel (this creates the service as zitadel)and the app installed in local namespace. With this configuration, the url looks like https://zitadel.local, Apisix pod is able to resolve this url since kubernetes DNS resolution <service-name>.<namespace> is also same as the ExternalDomain.

Installing certificates

The certificate configuration yaml file is provided in the Zitadel documentation so we use that as such without any customization. Below are the instruction to deploy.

# Create namespacekubectl create namespace local# Generate TLS certificateskubectl apply -f https://raw.githubusercontent.com/zitadel/zitadel-charts/main/examples/2-postgres-secure/certs-job.yaml -n local# wait till the job is completed kubectl wait --for=condition=complete job/create-certs -n local

Install Postgres

Below is the Postgres override values yaml file which will be used for Postgres deployment.

# filename: postgres-values.yamlvolumePermissions:  enabled: truetls:  enabled: true  certificatesSecret: postgres-cert  certFilename: "tls.crt"  certKeyFilename: "tls.key"auth:  postgresPassword: "abc"

Save the above content in a file postgres-values.yaml
Use below commands to install the 15.2.11 (latest version at the time of writting).

helm repo add bitnami https://charts.bitnami.com/bitnamihelm repo updatehelm install --wait db bitnami/postgresql --version 15.2.11 --values postgres-values.yaml

Install Zitadel app

To add the helm repo to local

helm repo add zitadel https://charts.zitadel.comhelm repo update

Zitadel override values yaml file

# zitadel-values.yamlzitadel:  masterkey: x123456789012345678901234567891y  configmapConfig:    ExternalSecure: false    # modify the external domain to localhost or any domain exists    # currently this value will be overrided from helm command with set    ExternalDomain: localhost    # add this configuration since we have ingress    ExternalPort: 80    TLS:      Enabled: false    Database:      Postgres:        Host: db-postgresql        Port: 5432        Database: zitadel        MaxOpenConns: 20        MaxIdleConns: 10        MaxConnLifetime: 30m        MaxConnIdleTime: 5m        User:          Username: zitadel          SSL:            Mode: verify-full        Admin:          Username: postgres          SSL:            Mode: verify-full  secretConfig:    Database:      Postgres:        User:          Password: xyz        Admin:          Password: abc  dbSslCaCrtSecret: postgres-cert  dbSslAdminCrtSecret: postgres-cert  dbSslUserCrtSecret: zitadel-certreplicaCount: 1

Save the content in file named zitadel-values.yaml, Note, the service port is set as 80, instead of default 8080 which will be configured in Apisix ingress configuration yaml as well.
Use below command to deploy the zitadel in local namespace.

helm upgrade --install zitadel -f ./zitadel-values.yaml . \    --set zitadel.configmapConfig.ExternalDomain=zitadel.local \    --set zitadel.configmapConfig.ExternalPort=80 \    --set service.port=80 \    --create-namespace \    --namespace local

Creating Zitadel Apisix route

The Apisix route configuration

# apisix-zitadel-ingress.yamlapiVersion: networking.k8s.io/v1kind: Ingressmetadata:  name: api6-zitadel-ingressspec:  ingressClassName: apisix  rules:  - host: zitadel.local    http:      paths:      - backend:          service:            name: zitadel            port:              number: 80        path: /        pathType: Prefix

Save the yaml content to apisix-zitadel-ingress.yaml and use below command to install zitadel ingress route.

kubectl apply -f apisix-zitadel-ingress.yaml -n local

The Zitadel UI should be accessible via http://zitadel.local from browser. In the login page when prompted the username is zitadel-admin@zitadel.zitadel.local and default password is Password1!. Refer documentation for more info.

Upon first access the UI looks like below.

Create Nginx Backend app access with Apisix and Zitadel configuration

Creating simple Nginx backend application

Below is the manifest file to deploy simple NGINX app with an endpoint /greet which responds back with JSON object.

apiVersion: v1kind: Namespacemetadata:  name: backend-app---apiVersion: v1kind: ConfigMapmetadata:  name: backend-nginx-config  namespace: backend-appdata:  nginx.conf: |    worker_processes auto;    error_log stderr notice;    events {      worker_connections 1024;    }    http {      variables_hash_max_size 1024;      log_format main '$remote_addr - $remote_user [%time_local] "$request" '                      '$status $body_bytes_sent "$http_referer" '                      '"$http_user_agent" "$http_x_forwarded_for"';      access_log off;      real_ip_header X-Real-IP;      charset utf-8;      server {        listen 80;        location /greet {          default_type application/json;          return 200 '{"status":"OK","message":"Greetings!! from server"}';        }      }    }---apiVersion: apps/v1kind: Deploymentmetadata:  name: backend-server  namespace: backend-app  labels:    app: backend-serverspec:  replicas: 1  selector:    matchLabels:      app: backend-server  template:    metadata:      labels:        app: backend-server    spec:      volumes:       - name: nginx-config         configMap:           name: backend-nginx-config           items:           - key: nginx.conf             path: nginx.conf      containers:      - name: backend-server        image: nginx        ports:        - containerPort: 80        volumeMounts:         - name: nginx-config           mountPath: /etc/nginx        resources:          requests:             memory: "128Mi"            cpu: "250m"          limits:            memory: "256Mi"            cpu: "500m"---apiVersion: v1kind: Servicemetadata:  name: backend-svc  namespace: backend-appspec:  selector:    app: backend-server  ports:    - protocol: TCP      port: 8081      targetPort: 80---

Before updating the Apisix route, to configure Zitadel as oidc-connector plugin, create Project and Application in Zitadel for client_id and client_secret.

Using Zitadel UI, create a project and input a project name.

Click the Create an application and input name for application.
Select Web and click continue.

Select code (the screen looks like below)

Input the redirect url, enable the Development Mode since we are using http url. In the Redirect URIshttp://backend.localhost/greet/redirect. This is the URL Zitadel redirects after successful authentication.

Upon saving the screen displays client_id and client_secret make a note of it. Once closed this will not be displayed, else we need to re-create it.
Below is the screen where URLs provided by the Zitadel Identity Provider.

In the Apisix route configuration below the plugins redirect_uri: http://backend.localhost/greet/redirectwhich should match the route config.

Note, if the redirect_uri match the route config exactly, noticed issues in the Apisix logs when accessing the backend app, refer zitadel documentation.

In this case the host is backend.localhost and uri is /* and we configured uri as http://backend.localhost/greet/redirect. The redirect in the uri is not configured in the backend app on Nginx config.

Complete Apisix route configuration using the Apisix dashboard and the completed configuration looks like below. Update the client_id and client_secret from the Zitadel application.

{  "uri": "/*",  "name": "backend-app",  "methods": [    "GET",    "OPTIONS"  ],  "host": "backend.localhost",  "plugins": {    "openid-connect": {      "_meta": {        "disable": false      },      "bearer_only": false,      "client_id": "****clientid created in zitadel app****",      "client_secret": "*** Zitadel app client secret *****",      "discovery": "http://zitadel.local/.well-known/openid-configuration",      "introspection_endpoint": "http://zitadel.local/oauth/v2/introspect",      "realm": "master",      "redirect_uri": "http://backend.localhost/greet/redirect"    }  },  "upstream": {    "nodes": [      {        "host": "backend-svc.backend-app",        "port": 8081,        "weight": 10      }    ],    "timeout": {      "connect": 6,      "send": 6,      "read": 6    },    "type": "roundrobin",    "scheme": "http",    "pass_host": "pass",    "keepalive_pool": {      "idle_timeout": 60,      "requests": 1000,      "size": 320    }  },  "status": 1}

Click the Route section on Apisix Dashboard to create the Route configuration, shown in below snapshot.

Update the upstream configuration, the Host configuration is <service-name>.<namespace> of the backend application service.

Edit the openid-connect plugin and update the configuration

Enable the plugin and click submit to create the route configuration.

Accessing backend app from browser

The http://backend.localhost/greet redirects to zitadel.local Zitadel login page.

After successful authentication the JSON payload will be displayed in the browser.

In certain case if browser throws 500 internal server error check the Apisix container logs to see if the correct redirect url is logged.

]]>

Operator SDK - Create Controller and CRD to deploy Spring app in K8S cluster

Thirumurthi S — Sat, 19 Aug 2023 14:34:21 GMT

Operator pattern to deploy SpringBoot application

In this article we will be using Operator SDK to create Custom Resource Definition (CRD) and Controller logic to deploy a Spring application in Kubernetes cluster.

With reference to my previous operator-sdk blog, we will use the same scaffolded and initialized project structure to create the CRD and reconcile logic. The reconciler logic will programmatically create Kubernetes resources object (using k8s.io/core libraries) like Deployment, ConfigMap, Service and deploy them to the cluster.

Pre-requisites

Basic understanding of Kubernetes
KinD CLI installed and Cluster running
WSL2 installed
GoLang installed in WSL2
Operator-SDK cli installed in WSL2

To demonstrate the use of Operator SDK in design and create CRD, update reconcile logic and deployment workflow - created a SpringBoot application that exposes an end point in 8080 port. This end point will render the response by reading the value from configuration file or environment variable.

Resources required to deploy Spring application?

ConfigMap
- The application.yaml content is deployed as ConfigMap resource. The deployment manifest should mount the ConfigMap as volume, so Spring application can access it. The mounted configMap is passed as external configuration, using --spring.config.location option when starting the Spring application. For more details refer the Kubernetes documentation.
Deployment
- The deployment will deploy the Spring application in the container.
Service
- The service created will forward traffic to 8080 port.

Operator pattern key components

Custom Resource Definition (CRD), Custom Resource (CR) and Controller/Operator are the key components

CRD & CR

Custom Resource Definition (CRD) and Custom Resource (CR) plays a crucial part and should be designed based on requirements.
The Kubernetes API can be extended with Custom Resource Definition (CRD), this defines the schema for the Custom Resource (CR).
Once the CRD is defined we can create an instance of the Custom Resource.

Controller

The controller in Operator SDK is responsible for watching for events and reconciling the Custom Resource (CR) to desire state based on logic defined in reconciliation flow. Without controller just deploying the CRD to cluster doesn't do anything.

Use of Operator SDK

With Operator SDK we can design and define the Customer Resource Definition (CRD) with corresponding reconciler logic. Part of the reconcile logic we can deploy different Kubernetes resources.

Design consideration of CRD and CR

The spec section in the Custom Resource (CR) will be the design starting point at least for beginners, the properties that are required to deploy the application goes under this section. In the Operator SDK project structure this file will be generated under config/samples/
In Custom Resource (CR) code below the spec section includes name and deployment properties. The name will be arbitrary name can be read and applied in the reconciler logic accordingly.
The deployment section further includes name, pod, config and service section.
The pod section includes the properties related to Pod manifest, like image, container port, etc. This will be read in the reconciler logic in Operator SDK project and a deployment object will be created to be deployed in cluster.
The config section will include the data to create the ConfigMap resource.
The service section include name and spec. The spec section in the api/v1alpha1/*type.go uses the k8s.io/core library object, so we can use the regular Service manifest properties like port, targetPort, etc.
By reading further we will see how to deployed the CR to cluster.

apiVersion: greet.greetapp.com/v1alpha1kind: Greetmetadata:  labels:    app.kubernetes.io/name: greet    app.kubernetes.io/instance: greet-sample    app.kubernetes.io/part-of: app-op    app.kubernetes.io/managed-by: kustomize    app.kubernetes.io/created-by: app-op  name: greet-samplespec:  name: first-app  deployment:    name: app    replicas: 1    pod:      image: thirumurthi/app:v1      imagePullPolicy: Always      mountName: app-mount      mountPath: /opt/app      podPort: 8080      command: ["java"]      args: ["-jar","app.jar","--spring.config.location=file:/opt/app/application.yaml"]    config:       name: app      fileName: application.yaml      data: |        env.name: k8s-kind-dev-env        greeting.source: from-k8s-configMap    service:      name: app      spec:        selector:          name: first-app        ports:        - name: svc-port          protocol: TCP          port: 80          targetPort: 8080

Operator SDK type.go file which holds the actual details

With reference to the CR above, we can compare it with the Operator SDK generated api/v1alpha1/*type.go file content where we can easily follow the pattern.

Note:-

The api/v1alpha1/*type.go file includes different Go struct data types defined which will in a way used in the Custom Resource YAML.
pod and config a custom Go struct datatype is defined.
In service, the spec uses the k8s.io/core api corev1.ServiceSpec with which we can specify regular Service manifest properties.

package v1alpha1import (    corev1 "k8s.io/api/core/v1"    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1")// GreetSpec defines the desired state of Greettype GreetSpec struct {    //Name of the resource    // +kubebuilder:validation:MaxLength=25    // +kubebuilder:validation:MinLength=1    Name       string     `json:"name"`    Deployment Deployment `json:"deployment"`}type Deployment struct {    Name      string  `json:"name"`    Pod       PodInfo `json:"pod"`    Replicas  int32   `json:"replicas,omitempty"`    ConfigMap Config  `json:"config,omitempty"`    Service   Service `json:"service,omitempty"`}type Config struct {    Name     string `json:"name,omitempty"`    FileName string `json:"fileName,omitempty"`    Data     string `json:"data,omitempty"`}type PodInfo struct {    Image           string   `json:"image"`    ImagePullPolicy string   `json:"imagePullPolicy,omitempty"`    MountName       string   `json:"mountName,omitempty"`    MountPath       string   `json:"mountPath,omitempty"`    PodPort         int32    `json:"podPort,omitempty"`    Command         []string `json:"command,omitempty"`    Args            []string `json:"args,omitempty"`}type Service struct {    Name string             `json:"name,omitempty"`    Spec corev1.ServiceSpec `json:"spec,omitempty"`}// GreetStatus defines the observed state of Greettype GreetStatus struct {    // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster    // Important: Run "make" to regenerate code after modifying this file    Status string `json:"status,omitempty"`}//Don't leave any space between the marker//+kubebuilder:object:root=true//+kubebuilder:printcolumn:name="APPNAME",type="string",JSONPath=".spec.name",description="Name of the app"//+kubebuilder:printcolumn:name="STATUS",type="string",JSONPath=".status.status",description="Status of the app"//+kubebuilder:subresource:status// +operator-sdk:gen-csv:customresourcedefinitions.displayName="Greet App"// +operator-sdk:gen-csv:customresourcedefinitions.resources="Deployment,v1,\"A Kubernetes Deployment of greet app\""type Greet struct {    metav1.TypeMeta   `json:",inline"`    metav1.ObjectMeta `json:"metadata,omitempty"`    Spec   GreetSpec   `json:"spec,omitempty"`    Status GreetStatus `json:"status,omitempty"`}//+kubebuilder:object:root=true// GreetList contains a list of Greettype GreetList struct {    metav1.TypeMeta `json:",inline"`    metav1.ListMeta `json:"metadata,omitempty"`    Items           []Greet `json:"items"`}func init() {    SchemeBuilder.Register(&Greet{}, &GreetList{})}

Image creation of Spring application

First, build the jar artifact for the SpringBoot application, using maven clean install.
With the Dockerfile defined we can use Docker CLI to build image using docker build and docker push to push the image to Dockerhub.

Approaches to deploy the Spring application

Using individual resource manifests (no Operator SDK)

The conventional way of deploying the app is to create different resource manifest YAML either in single file or different file. In this case we have to create three manifests for ConfigMap, Service and Deployment resources.

Using Operator SDK

With the Operator SDK project we need to define the CRD in here we have used GoLang and build the reconciler logic.
Using the utilities provided in Operator SDK, we generate the CRD files and Controller image.
Once we developed the controller logic, in order to deploy the Spring application we need to deploy the controller image as deployment, then the CRD manifest and the Custom Resource (CR) to the cluster. This CR includes the image of the Spring application.

Note:-

During development the api/v1alpha1/*types.go file will be updated with custom Go struct datatypes, whenever we update this file we need to execute make generate manifests command in WSL2 terminal to generate the CRDs YAML file.

Spring application code

SpringBoot application entry point

package com.app.app;import org.springframework.beans.factory.annotation.Value;import org.springframework.boot.SpringApplication;import org.springframework.boot.autoconfigure.SpringBootApplication;import org.springframework.context.annotation.Bean;import org.springframework.web.bind.annotation.GetMapping;import org.springframework.web.bind.annotation.RequestMapping;import org.springframework.web.bind.annotation.RequestParam;import org.springframework.web.bind.annotation.RestController;@SpringBootApplication@RequestMapping("/api")@RestControllerpublic class AppApplication {    @Bean    GreetConfiguration getGreetConfig(){        return new GreetConfiguration();    }    public static void main(String[] args) {        SpringApplication.run(AppApplication.class, args);    }    @Value("${env.name:no-env-provided}")    private String envName;    public record Greeting(String content,String source) { }    @GetMapping("/hello")    public Greeting hello(@RequestParam(value = "name", defaultValue = "anonymous") String name) {        return new Greeting(String.format("Hello %s!", name),                            String.format("%s-%s",getGreetConfig().getSource().toUpperCase(),envName));    }}

Spring Configuration class

Configuration class to read the application.yaml when application context is loaded

package com.app.app;import lombok.Data;import org.springframework.boot.context.properties.ConfigurationProperties;import org.springframework.context.annotation.Configuration;@Configuration@ConfigurationProperties(prefix="greeting")@Datapublic class GreetConfiguration {    private String source = "from-spring-code";}

Dependencies

<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">    <modelVersion>4.0.0</modelVersion>    <parent>        <groupId>org.springframework.boot</groupId>        <artifactId>spring-boot-starter-parent</artifactId>        <version>3.1.2</version>        <relativePath/> <!-- lookup parent from repository -->    </parent>    <groupId>com.app</groupId>    <artifactId>app</artifactId>    <version>0.0.1-SNAPSHOT</version>    <name>app</name>    <description>Simple app with </description>    <properties>        <java.version>17</java.version>        <spring-cloud.version>2022.0.3</spring-cloud.version>    </properties>    <dependencies>        <dependency>        <groupId>org.springframework.boot</groupId>        <artifactId>spring-boot-starter-web</artifactId>       </dependency>           <dependency>        <groupId>org.projectlombok</groupId>        <artifactId>lombok</artifactId>        <optional>true</optional>        </dependency>            <dependency>        <groupId>org.springframework.boot</groupId>        <artifactId>spring-boot-starter-web</artifactId>        </dependency>        <dependency>        <groupId>org.springframework.boot</groupId>        <artifactId>spring-boot-starter-test</artifactId>        <scope>test</scope>        </dependency>    </dependencies>    <build>        <plugins>            <plugin>                <groupId>org.springframework.boot</groupId>                <artifactId>spring-boot-maven-plugin</artifactId>                <configuration>                    <excludes>                        <exclude>                            <groupId>org.projectlombok</groupId>                            <artifactId>lombok</artifactId>                        </exclude>                    </excludes>                </configuration>            </plugin>            <plugin>                <artifactId>maven-surefire-plugin</artifactId>                <version>3.0.0-M4</version>            </plugin>        </plugins>    </build></project>

Note:-

Leave the application.properties file empty when building the jar.

Dockerfile

FROM eclipse-temurin:17-jdk-alpineARG JAR_FILECOPY ${JAR_FILE} app.jarENTRYPOINT ["java","-jar","/app.jar"]

Image creation of Spring application

Create the jar file usingmvn clean install. The jar file will be created in target\ folder.
Once the Jar file is generated, use below command to build the docker image.

docker build --build-arg JAR_FILE=target/*.jar -t local/app .

We can run the Spring application image in Docker Desktop wiht below command.

docker run --name app -e ENV_NAME="docker-env" -p 8080:8080 -d local/app:latest

Note:-

The env.name value is passed as docker environment variable ENV_NAME.
After the container is ready, the endpoint http://localhost:8080/api/hello?name=test should return response

Output - Running application in Docker

$ curl -i http://localhost:8080/api/hello?name=testHTTP/1.1 200                                                    Content-Type: application/json                                  Transfer-Encoding: chunked                                      Date: Tue, 15 Aug 2023 03:36:14 GMT                             {"content":"Hello test!","source":"FROM-SPRING-CODE-docker-env"}

Passing GREETING_SOURCE environment variable in Docker run command the output looks like below.

docker run --name app -e ENV_NAME="docker-env" -e GREETING_SOURCE="from-docker-cli" -p 8080:8080 -d local/app:latest

Output

$ curl -i http://localhost:8080/api/hello?name=test                                HTTP/1.1 200 Content-Type: application/jsonTransfer-Encoding: chunkedDate: Tue, 15 Aug 2023 03:40:40 GMT{"content":"Hello test!","source":"FROM-DOCKER-CLI-docker-env"}

Push Spring app image to Dockerhub

The images can pushed to the Dockerhub with below command. Use appropriate repository name.

docker build --build-arg JAR_FILE=target/*.jar -t <repoistory-name>/app:v1 .

Deploy Spring app with manifest without Operator SDK

Once the image is pushed to Dockerhub or private registry, we can deploy the Spring application and the Deployment manifest looks like below. Assuming the ConfigMap is created as app-cfg

apiVersion: apps/v1kind: Deploymentmetadata:  labels:    name: first-app  name: app-deploy  namespace: defaultspec:  replicas: 1  selector:    matchLabels:      name: first-app  template:    metadata:      labels:        name: first-app    spec:      containers:      - image: <repository-name>/app:v1        imagePullPolicy: Always        name: app-pod        ports:        - containerPort: 80          name: app-port        volumeMounts:        - mountPath: /opt/app          name: app-mount-vol      volumes:      - configMap:          name: app-cfg        name: app-mount-vol

Deploy Spring application with Operator SDK

CRDs

The fragment of code from api/v1alpha1/*types.go file where we define custom Go struct data types, which will generate the CRDs accordingly.
The Service struct uses the k8s.io/api/core corev1.ServiceSpec so we can use port, targetPort properties like in regular Service manifest file. This is an example to demonstrate that it is always not necessary to define all the properties we can reuse the k8s.io core library object as well.

type Deployment struct {    Name      string  `json:"name"`    Pod       PodInfo `json:"pod"`    Replicas  int32   `json:"replicas,omitempty"`    ConfigMap Config  `json:"config,omitempty"`    Service   Service `json:"service,omitempty"`}type Service struct {    Name string             `json:"name,omitempty"`    Spec corev1.ServiceSpec `json:"spec,omitempty"`}

Reconciler logic

The Reconcile() method calls three different function which creates the Service, ConfigMap and Deployment resource.
- Function checkAndCreateConfigMapResource(), checkAndCreateServiceResource, and checkAndCreateDeploymentResource() follows the same logic to create or update only difference is which object is being initialized in each method.
When the CR is applied, the create event will be triggered and the resources will be deployed. If the CR is deployed with any updates, the update event will update the resources.

package controllersimport (    "context"    "fmt"    "time"    "gopkg.in/yaml.v2"    appsv1 "k8s.io/api/apps/v1"    corev1 "k8s.io/api/core/v1"    "k8s.io/apimachinery/pkg/api/errors"    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"    "k8s.io/apimachinery/pkg/runtime"    "k8s.io/apimachinery/pkg/types"    ctrl "sigs.k8s.io/controller-runtime"    "sigs.k8s.io/controller-runtime/pkg/client"    "sigs.k8s.io/controller-runtime/pkg/log"    "k8s.io/client-go/tools/record"    greetv1alpha1 "github.com/thirumurthis/app-operator/api/v1alpha1")// GreetReconciler reconciles a Greet object// added Recorder to the structtype GreetReconciler struct {    client.Client    Scheme   *runtime.Scheme    Recorder record.EventRecorder}//+kubebuilder:rbac:groups=greet.greetapp.com,resources=greets,verbs=get;list;watch;create;update;patch;delete//+kubebuilder:rbac:groups=greet.greetapp.com,resources=greets/status,verbs=get;update;patch//+kubebuilder:rbac:groups=greet.greetapp.com,resources=greets/finalizers,verbs=update//+kubebuilder:rbac:groups=greet.greetapp.com,resources=deployments,verbs=get;list;watch;create;update;patch;delete//+kubebuilder:rbac:groups=greet.greetapp.com,resources=pods,verbs=get;list;create;update;patchfunc (r *GreetReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {    _ = log.FromContext(ctx)    log.Log.Info("Reconciler invoked..")    instance := &greetv1alpha1.Greet{}    if err := r.Get(ctx, req.NamespacedName, instance); err != nil {        r.Recorder.Event(instance, corev1.EventTypeWarning, "Object", "Failed to read Object")        log.Log.Info("Error while reading the object")        return ctrl.Result{}, client.IgnoreNotFound(err)    }    appName := instance.Spec.Name    if instance.Spec.Name != "" {        log.Log.Info(fmt.Sprintf("appName for CRD is - %s ", instance.Spec.Name))        r.Recorder.Event(instance, corev1.EventTypeWarning, "Greet", fmt.Sprintf("Created - %s ", appName))    } else {        log.Log.Info("instance.Spec.Name - NOT FOUND")    }    checkAndCreateConfigMapResource(r, instance, ctx)    checkAndCreateServiceResource(r, instance, ctx)    checkAndCreateDeploymentResource(r, instance, ctx)    if instance.Status.Status == "" {        instance.Status.Status = "OK"        log.Log.Info("instance.Spec.Name - is set to OK")    }    if err := r.Status().Update(ctx, instance); err != nil {        log.Log.Info("Error while reading the object")        return ctrl.Result{}, client.IgnoreNotFound(err)    }    return ctrl.Result{}, nil}// SetupWithManager sets up the controller with the Manager.func (r *GreetReconciler) SetupWithManager(mgr ctrl.Manager) error {    return ctrl.NewControllerManagedBy(mgr).        For(&greetv1alpha1.Greet{}).        Owns(&appsv1.Deployment{}).        Complete(r)}func checkAndCreateConfigMapResource(r *GreetReconciler, instance *greetv1alpha1.Greet,    ctx context.Context) (ctrl.Result, error) {    fileName := instance.Spec.Deployment.ConfigMap.FileName    data := make(map[string]string)    content := instance.Spec.Deployment.ConfigMap.Data    data[fileName] = content    identifiedConfigMap := &corev1.ConfigMap{}    configMapName := instance.Spec.Deployment.ConfigMap.Name + "-cfg"    log.Log.Info(fmt.Sprintf("data in config %v", data))    configMap := &corev1.ConfigMap{        ObjectMeta: metav1.ObjectMeta{            Name:      configMapName,            Namespace: instance.Namespace,        },        Data: data,    }    if err := r.Get(ctx,        types.NamespacedName{Name: instance.Name, Namespace: instance.Namespace},        identifiedConfigMap); err != nil && errors.IsNotFound(err) {        log.Log.Info("Creating ConfigMap", "ConfigMap", configMapName)        // Error occurred while creating the ConfigMap        //r.Log.Info("Creating ConfigMap", "ConfigMap", configMap)        if err := r.Create(ctx, configMap); err != nil {            // Error occurred while creating the ConfigMap            return ctrl.Result{Requeue: true}, err        }        return ctrl.Result{Requeue: true}, nil    } else {        log.Log.Info(fmt.Sprintf("Updating ConfigMap %v", configMap))        if err := r.Update(ctx, configMap); err != nil {            // Error occurred while updating the ConfigMap            return ctrl.Result{}, err        }        return ctrl.Result{RequeueAfter: time.Duration(60 * time.Second)}, err    }}func checkAndCreateServiceResource(r *GreetReconciler, instance *greetv1alpha1.Greet,    ctx context.Context) (ctrl.Result, error) {    labels := make(map[string]string)    labels["name"] = instance.Spec.Name    serviceSpec := instance.Spec.Deployment.Service.Spec    identifiedService := &corev1.Service{}    name := instance.Spec.Deployment.Service.Name + "-svc"    log.Log.Info(fmt.Sprintf("data in service %v", serviceSpec))    service := &corev1.Service{        ObjectMeta: metav1.ObjectMeta{            Name:      name,            Namespace: instance.Namespace,            Labels:    labels,        },        Spec: serviceSpec,    }    if err := r.Get(ctx, types.NamespacedName{Name: instance.Name, Namespace: instance.Namespace},        identifiedService); err != nil && errors.IsNotFound(err) {        log.Log.Info("Creating Service", "Service", serviceSpec)        // Error occurred while creating the ConfigMap        //r.Log.Info("Creating ConfigMap", "ConfigMap", configMap)        if err := r.Create(ctx, service); err != nil {            // Error occurred while creating the ConfigMap            return ctrl.Result{}, err        }        return ctrl.Result{Requeue: true}, nil    } else {        log.Log.Info(fmt.Sprintf("Updating service %v", service))        if err := r.Update(ctx, service); err != nil {            // Error occurred while updating the ConfigMap            return ctrl.Result{}, err        }        return ctrl.Result{RequeueAfter: time.Duration(60 * time.Second)}, err    }}func checkAndCreateDeploymentResource(r *GreetReconciler, instance *greetv1alpha1.Greet,    ctx context.Context) (ctrl.Result, error) {    labels := make(map[string]string)    labels["name"] = instance.Spec.Name    deploymentName := instance.Spec.Deployment.Name + "-deploy"    replicas := instance.Spec.Deployment.Replicas    imageUrl := instance.Spec.Deployment.Pod.Image    imagePullPolicy := instance.Spec.Deployment.Pod.ImagePullPolicy    mountName := instance.Spec.Deployment.Pod.MountName    mountPath := instance.Spec.Deployment.Pod.MountPath    port := instance.Spec.Deployment.Pod.PodPort    configMapName := instance.Spec.Deployment.ConfigMap.Name + "-cfg"    identifiedDeployment := &appsv1.Deployment{}    log.Log.Info(fmt.Sprintf("Deployment - %s %s %s %d", deploymentName, imageUrl, imagePullPolicy, replicas))    volumeName := mountName + "-vol"    var containers []corev1.Container    var policyNameType corev1.PullPolicy    container := corev1.Container{}    container.Image = imageUrl    var containerPorts []corev1.ContainerPort    cPort := corev1.ContainerPort{}    if port == 0 {        port = 80    }    cPort.ContainerPort = port    var portName string    portName = instance.Spec.Deployment.Name    if len(instance.Spec.Deployment.Name) > 10 {        portName = instance.Spec.Deployment.Name[:10]    }    cPort.Name = portName + "-port"    containerPorts = append(containerPorts, cPort)    container.Ports = containerPorts    container.Name = instance.Spec.Deployment.Name + "-pod"    //pod command    container.Command = instance.Spec.Deployment.Pod.Command    container.Args = instance.Spec.Deployment.Pod.Args    if imagePullPolicy == "" || len(imagePullPolicy) == 0 {        policyNameType = corev1.PullAlways    }    if imagePullPolicy == "Always" {        policyNameType = corev1.PullAlways    }    if imagePullPolicy == "IfNotPresent" {        policyNameType = corev1.PullIfNotPresent    }    container.ImagePullPolicy = policyNameType    container.VolumeMounts = append(container.VolumeMounts, corev1.VolumeMount{        Name:      volumeName,        MountPath: mountPath,    })    containers = append(containers, container)    log.Log.Info(fmt.Sprintf("Container - %#v", containers))    volume := corev1.Volume{        Name: volumeName,        VolumeSource: corev1.VolumeSource{            ConfigMap: &corev1.ConfigMapVolumeSource{                LocalObjectReference: corev1.LocalObjectReference{                    Name: configMapName,                },            },        },    }    var volumes []corev1.Volume    volumes = append(volumes, volume)    deployer := &appsv1.Deployment{        ObjectMeta: metav1.ObjectMeta{            Name:      deploymentName,            Namespace: instance.Namespace,            Labels:    labels,        },        Spec: appsv1.DeploymentSpec{            Replicas: &replicas,            Selector: &metav1.LabelSelector{                MatchLabels: labels,            },            Template: corev1.PodTemplateSpec{                ObjectMeta: metav1.ObjectMeta{                    Labels: labels,                },                Spec: corev1.PodSpec{                    Containers: containers,                    Volumes:    volumes,                },            },        },    }    // Used to deserialize    deployUnstructured, err := runtime.DefaultUnstructuredConverter.ToUnstructured(deployer)    if err != nil {        log.Log.Error(err, "Error occurred in unstructuring")    }    encoder, err := yaml.Marshal(deployUnstructured)    if err != nil {        log.Log.Error(err, "Error occurred in transforming")    }    //prints the yaml format of the deployment object created    log.Log.Info(fmt.Sprintf("%#v", string(encoder)))    log.Log.Info(fmt.Sprintf("yaml: \n%#v\n", deployer))    if err := r.Get(ctx,        types.NamespacedName{Name: instance.Name, Namespace: instance.Namespace},        identifiedDeployment); err != nil && errors.IsNotFound(err) {        log.Log.Info("Creating Deployment", "Deploy", deployer)        // Error occurred while creating the ConfigMap        //r.Log.Info("Creating ConfigMap", "ConfigMap", configMap)        if err := r.Create(ctx, deployer); err != nil {            // Error occurred while creating the ConfigMap            return ctrl.Result{}, err        }        //instance as the owner and controller        ctrl.SetControllerReference(instance, deployer, r.Scheme)        return ctrl.Result{Requeue: true}, nil    } else {        log.Log.Info(fmt.Sprintf("Updating deployment %v", deployer))        if err := r.Update(ctx, deployer); err != nil {            // Error occurred while updating the ConfigMap            return ctrl.Result{}, err        }        //instance as the owner and controller        ctrl.SetControllerReference(instance, deployer, r.Scheme)        return ctrl.Result{RequeueAfter: time.Duration(60 * time.Second)}, err    }}

The Custom Resource file content is show above.
The CRD file can be found from Github

Deployment tips

During local development to run the operator in KinD we can use make generate manifests install run command.
Upon deploying the Custom Resource the logs will be displayed like in below snapshot.

To deploy the Custom Resource manifests use save the content to a file and use the kubectl apply -f <file-name-with-content>.yaml command.
In below the deployment has pod, config, service sections. The reconcile code will use these values to deploy them in Kuberented cluster.
Note the config in the Custom Resource (CR) file

    config:       name: app      fileName: application.yaml      data: |        env.name: k8s-kind-dev-env        greeting.source: from-k8s-configMap`

Once application is deployed, with nginx container we can hit the endpoint and validate the response, which will look like in below snapshot

Creating controller image

From WSL2 we can use below command to create the controller image and push it to Dockerhub.

 make docker-build docker-push IMG=thirumurthi/app-op:v1

Deploying Spring app using Controller Image, CRD and CR

To deploy Spring app in test or production like environment we need to follow below steps
Deploy the Controller in Deployment manifest
Below is the deployment manifest to deploy the controller.

apiVersion: apps/v1kind: Deploymentmetadata:  labels:    name: operator-controller  name: app-controller  namespace: defaultspec:  replicas: 1  selector:    matchLabels:      name: app-controller  template:    metadata:      labels:        name: app-controller    spec:      containers:      - image: thirumurthi/app-op:v1        name: app-controller

Deploy the CRD

kubectl apply -f https://github.com/thirumurthis/projects/blob/main/go-operator/app-op/config/crd/bases/greet.greetapp.com_greets.yaml

Deploy the CR

kubectl apply -f https://github.com/thirumurthis/projects/blob/main/go-operator/app-op/config/samples/greet_v1alpha1_greet.yaml

Note:-
- When I tried to deploy the controller as deployment, there was an due to permission and it requires a ServiceAccount to be created. The ServiceAccount can be created in Operator SDK but for simplicity executed the following command manually.
- The exception message that you would notice in case if ServiceAccount is not created.

E0816 03:51:14.671318       1 reflector.go:148] pkg/mod/k8s.io/client-go@v0.27.4/tools/cache/reflector.go:231: Failed to watch *v1alpha1.Greet: failed to list *v1alpha1.Greet: greets.greet.greetapp.com is forbidden: User "system:serviceaccount:default:default" cannot list resource "greets" in API group "greet.greetapp.com" at the cluster scope

To manually create the ServiceAccount we can use below command

kubectl create clusterrole deployr --verb=get,list,watch,create,delete,patch,update --resource=deployments.appskubectl create clusterrolebinding deployr-srvacct-default-binding --clusterrole=deployr --serviceaccount=default:default

Output after deploying the Controller, CRD and CR

Once Controller, CRD and CR are deployed the list of resources and the output accessing the deployed spring application looks like below snapshot.

]]>

Extend Kubernetes API with operator-sdk

Thirumurthi S — Sat, 29 Jul 2023 16:40:56 GMT

This article gives a brief overview of how to use Operator SDK (Go) to create Custom Resource Definition and Controller to extend Kubernetes API and manage Custom Resources.

This is a hands-on article and requires a good understanding in Kubernetes and how operators work. The operator code can be downloaded from GitHub

Extending Kubernetes API with Operator-SDK

In this blog, have explained how to use operator-sdk to create additional resource in Kubernetes cluster. To start with you we first scaffold the project and create api using the operator-sdk cli, this gives us a starting point to develop the CRD's and Controllers. Note, not all points are explained in detail, only focus on necessary aspect required in developing operators.

I used Windows machine for development. The operator-sdk CLI was installed in WSL2. The scaffolded operator-sdk project can be opened in Visual Studio code for development to update code.

Pre-requisites:

Basic understanding of Kubernetes and Operators
Docker Desktop
KIND CLI (we will use KinD cluster, install in Windows)
kubectl Installed in WSL2 and Windows
Go installed in WSL2
GNU make installed and updated in WSL2
Operator-SDK CLI installed in WSL2

About Operator framework

Operator framework provides tools to extend the Kubernetes API, kubectl api-resource will list the existing resources from the cluster API with operator framework we can create a new resource and deploy it to the cluster.
Operator-SDK CLI provides option to scaffold the project and updates the project with the resource coordinates like Kind, Version and Group used to identify once the new resource is deployed to cluster. Few of the files will be edited, api/<version>/<kind-name>_type.go file contains the go struct where we can define the properties this will be used to generate CRD. The reconciler logic goes into controllers/<kind-name>_controller.go, the entry point will be the main.go file. The operator-sdk project utilizes kubebuilder underneath. The details are explained later. The operator-sdk already has the code to create new manager in main.go and provides the basic code structure for reconciler logic under controllers/ folder.
Once the CRD is deployed to the cluster, we need a controller logic which will monitor for any changes to the resources. For example, when the resource is created the controller will know of this event and in operator will invoke the defined function.

Install GoLang in WSL2

To install GoLang, refer this Go documentation

wget https://dl.google.com/go/go1.20.4.linux-amd64.tar.gzsudo tar -xvf go1.20.4.linux-amd64.tar.gzsudo mv go /usr/local

Open the ~/.bashrc file and add the below variables so it will update as environment variables

export GOROOT=/usr/local/goexport GOPATH=$HOME/goexport PATH=$GOPATH/bin:$GOROOT/bin:$PATH

To find the Go version, issuing go version command we should see the response like below.

$ go versiongo version go1.20.4 linux/amd64

Install KinD CLI in Windows

Download and install KinD CLI, this can be installed via Chocolatey, refer the documentation

The KinD cluster Kubernetes cluster configuration details.

~$ kubectl version -o yamlclientVersion:  buildDate: "2023-05-17T14:20:07Z"  compiler: gc  gitCommit: 7f6f68fdabc4df88cfea2dcf9a19b2b830f1e647  gitTreeState: clean  gitVersion: v1.27.2  goVersion: go1.20.4  major: "1"  minor: "27"  platform: linux/amd64kustomizeVersion: v5.0.1serverVersion:  buildDate: "2023-06-15T00:36:28Z"  compiler: gc  gitCommit: 25b4e43193bcda6c7328a6d147b1fb73a33f1598  gitTreeState: clean  gitVersion: v1.27.3  goVersion: go1.20.5  major: "1"  minor: "27"  platform: linux/amd64

Install GNU make in WSL2

To install GNU make version, simply use sudo apt install make

$ make -versionGNU Make 4.3Built for x86_64-pc-linux-gnuCopyright (C) 1988-2020 Free Software Foundation, Inc.License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

Install Operator-SDK CLI in WSL2

To install operator-sdk refer the Operator SDK documentation, it is self explainotary.
Once installed, we can view the version refer below snippet for command and output response.

$ operator-sdk versionoperator-sdk version: "v1.30.0", commit: "b794fe909abc1affa1f28cfb75ceaf3bf79187e6", kubernetes version: "1.26.0", go version: "go1.19.10", GOOS: "linux", GOARCH: "amd64"

Creating new Operator-SDK project for development

Create a new folder, using the operator-sdk CLI we can initalize the project. The operator-sdk init command will scaffold the necessary files. The command looks like below, the switch options explained below.

operator-sdk init --domain greetapp.com --repo github.com/thirumurthis/app-operator

Output of operator-sdk init command

Writing kustomize manifests for you to edit...Writing scaffold for you to edit...Get controller runtime:$ go get sigs.k8s.io/controller-runtime@v0.14.1Update dependencies:$ go mod tidyNext: define a resource with:$ operator-sdk create api

Info:

The switch option in the operator-sdk command above used to scaffold a project.
--domain in the command is required which will identify the resource when using kubectl api-resource
$ kubectl api-resources | grep -e greet -e NAMENAME        SHORTNAMES   APIVERSION                             NAMESPACED   KINDgreets                   greet.greetapp.com/v1alpha1            true         Greet
--repo is used to specify that this project is outside go path.

Update the scaffolded project with the API with operator-sdk CLI

As mentioned above the API is identified by Kubernetes cluster using Kind, Version and Group, we are using operator-sdk with these coordinates as switch options to update the already created operator-sdk project.
This will update the project structure by updating generated _type.go file and other files.
The output of operator-sdk init command also directs this step to be next.
The kind and version will be used in the CRD manifest.
Command to use operator-sdk CLI to create API on the scaffolded project.

$ operator-sdk create api --group greet --version v1alpha1 --kind Greet --resource --controller

Info:

--resource and --controller switch will update the resources and creates the controller. If not provided in the command, the CLI will prompt from user whether resources and controller required to be created for the project.

Output of operator-sdk create api command

Writing kustomize manifests for you to edit...Writing scaffold for you to edit...api/v1alpha1/deploymentscaler_types.gocontrollers/deploymentscaler_controller.goUpdate dependencies:$ go mod tidyRunning make:$ make generatemkdir -p /mnt/c/goOperator/deploymentscaler/bintest -s /mnt/c/goOperator/deploymentscaler/bin/controller-gen && /mnt/c/goOperator/deploymentscaler/bin/controller-gen --version | grep -q v0.11.1 || \GOBIN=/mnt/c/goOperator/deploymentscaler/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.1/mnt/c/goOperator/deploymentscaler/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."Next: implement your new API and generate the manifests (e.g. CRDs,CRs) with:$ make manifests

Note:

The operator-sdk project includes a Makefile with bunch of useful commands that is useful during development.
In WSL2 terminal, if we are already navigated to the project folder issue make manifests which will generate CRD file. This yaml file will be under config/crd/bases/, in this case the file name will start with greet.

Update the go mod with the latest library version

This step is optional, the go.mod file is updated in the project structure so the latest Kubernetes libraries will be used. These are the latest version at the time of writing this article.
The go.mod file with the updated version looks like below. Once the file is updated, issue go mod tidy to download the latest libraries. go.mod file is more like dependency manager for Go, like maven for Java or pacakge.json in node.js.

go 1.19require (        github.com/onsi/ginkgo/v2 v2.9.5        github.com/onsi/gomega v1.27.7        k8s.io/apimachinery v0.27.4        k8s.io/client-go v0.27.4        sigs.k8s.io/controller-runtime v0.15.0)

Adding new property to the CRD

Let's add new property to the CRD called Name, latter we will see how we can read this value from the reconciler code and print this in log
The Name property should be updated in the api/v1alpha1/*_types.go, where v1alpha1 is the version used in the operator-sdk create api command.
The new property is updated in the GreetSpec struct.
- On top of the new property, we add two markers to perform length validation (comments marked as 1 and 2).
- Other section of the greet_type.go file remains unchanged

Note:-

The comment //Name of the resource above new property will be displayed in the CRD as description.
To create a new CRD from the project use make manifests command in WSL2 the file will be created under 'config/crd/bases/'. Once deployed to the cluster the same can be viewed using kubectl explain <crd-resource-name> command.

package v1alpha1import (    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1")type GreetSpec struct {    //Name of the resource     // +kubebuilder:validation:MaxLength=15   // --->  1 maker for validation    // +kubebuilder:validation:MinLength=1    // --->  2 maker for validation    Name string `json:"name"`    //------------------> 3 new property for the CRD}// GreetStatus defines the observed state of Greettype GreetStatus struct {}//+kubebuilder:object:root=true//+kubebuilder:subresource:statustype Greet struct {    metav1.TypeMeta   `json:",inline"`    metav1.ObjectMeta `json:"metadata,omitempty"`    Spec   GreetSpec   `json:"spec,omitempty"`    Status GreetStatus `json:"status,omitempty"`}//+kubebuilder:object:root=true// GreetList contains a list of Greettype GreetList struct {    metav1.TypeMeta `json:",inline"`    metav1.ListMeta `json:"metadata,omitempty"`    Items           []Greet `json:"items"`}func init() {    SchemeBuilder.Register(&Greet{}, &GreetList{})}

With the above code change, if we issue make manifests command, the CRD YAML file gets created under config/crd/bases/. The newly added properties will look like below.

            properties:              name:                description: Name of the resource                maxLength: 15                minLength: 1                type: string            required:            - name            type: object

Update the controller code to read the new property

Whenever the api/v1alpha1/greet_types1file is updated, we need to use make generate command. This command will perform updates the project with the newly included properties.
Below is the reconciler Go code created by operator-sdk under controllers.
In the Reconcile function, the log.Log is used to log simple message
We initialize an instance of the Greet CRD object which has Name property
The r.Get() will fetch the CRD instance from the Cluster, in case of error when obtaining the object simply return.
If the CRD instance is available, we get the name and log it.
At this point there are NO changes to other files in the project.

func (r *GreetReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {    _ = log.FromContext(ctx)    log.Log.Info("Reconciler invoked..") // -------------> Logs the message passed    instance := &greetv1alpha1.Greet{}   //--------------> initalize an instance of the CRD     err := r.Get(ctx, req.NamespacedName, instance)    if err != nil {        return ctrl.Result{}, nil    }    appName := instance.Spec.Name        // Get the name from the CR deployed     log.Log.Info(fmt.Sprintf("app Name for CRD is - %s ",appName))    if instance.Spec.Name != "" {      log.Log.Info(fmt.Sprintf("appName for CRD is - %s ",instance.Spec.Name))    } else {      log.Log.Info("instance.Spec.Name - NOT FOUND")    }    return ctrl.Result{}, nil}

Create the Custom resource for the Greet CRD

The skeleton of the Custom Resource file will be created when we issue make manifests, where the newly added property is used here name: first-app

apiVersion: greet.greetapp.com/v1alpha1kind: Greetmetadata:  labels:    app.kubernetes.io/name: greet    app.kubernetes.io/instance: greet-sample    app.kubernetes.io/part-of: app-op    app.kubernetes.io/managed-by: kustomize    app.kubernetes.io/created-by: app-op  name: greet-samplespec:  # TODO(user): Add fields here  name: first-app

Deploy operator to local KinD cluster

The operator-sdk includes additional make targets, to help in development. Issuing below command will deploy the controller changes to local kind cluster. The output looks like below.

Note:

In below output first deployed operator changes using make generate install run, then deployed the Custom Resource manifest.
Once the operator sees the CR deployment a create event is received by reconciler and the based on the logic to print the log the log info will be printed.

Info:

If make generate install run from the WSL2 terminal was not able to find the Kubernetes cluster, you can copy the kube config from the windows .kube to WSL2 path.
Form WSL2 terminal, you can use below command to copy,
cp /mnt/c/Users/<username>/.kube/config ~/.kube/config

$ make generate install run....go run ./main.go2023-07-23T13:28:13-07:00       INFO    controller-runtime.metrics      Metrics server is starting to listen    {"addr": ":8080"}2023-07-23T13:28:13-07:00       INFO    setup   starting manager2023-07-23T13:28:13-07:00       INFO    Starting server {"kind": "health probe", "addr": "[::]:8081"}2023-07-23T13:28:13-07:00       INFO    starting server {"path": "/metrics", "kind": "metrics", "addr": "[::]:8080"}2023-07-23T13:28:13-07:00       INFO    Starting EventSource    {"controller": "greet", "controllerGroup": "greet.greetapp.com", "controllerKind": "Greet", "source": "kind source: *v1alpha1.Greet"}2023-07-23T13:28:13-07:00       INFO    Starting Controller     {"controller": "greet", "controllerGroup": "greet.greetapp.com", "controllerKind": "Greet"}2023-07-23T13:28:13-07:00       INFO    Starting workers        {"controller": "greet", "controllerGroup": "greet.greetapp.com", "controllerKind": "Greet", "worker count": 1}2023-07-23T13:28:28-07:00       INFO    Reconciler invoked..2023-07-23T13:28:28-07:00       INFO    app Name for CRD is - first-app2023-07-23T13:28:28-07:00       INFO    appName for CRD is - first-app2023-07-23T13:28:28-07:00       INFO    GET invoked reconcile for resource name - greet-sample

Output once operator deployed

kubectl describe to describe the CRD and looks like below, note there are no event info we can add it later in next section to controller code.

Tracking Event in the Controller

To record the events, first we need to add recorder to the GreetReconciler struct in the controllers/greet_controller.go file and import the library for record.

 import(  ....    "k8s.io/client-go/tools/record"    // Import the library for record  ....)type GreetReconciler struct {    client.Client    Scheme *runtime.Scheme    Recorder record.EventRecorder      // New Recorder event}

We will use the Recorder event object to record events and the code looks like below

func (r *GreetReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {    _ = log.FromContext(ctx)    log.Log.Info("Reconciler invoked..")        instance := &greetv1alpha1.Greet{}    err := r.Get(ctx, req.NamespacedName, instance)    if err != nil {        // If there are errors we record an event.        r.Recorder.Event(instance, corev1.EventTypeWarning, "Object", "Failed to read Object")        return ctrl.Result{}, nil    }       appName := instance.Spec.Name       // Recorder event added       r.Recorder.Event(instance, corev1.EventTypeWarning, "Object", fmt.Sprintf("Created - %s ",appName))       //..... other code    return ctrl.Result{}, nil}

Since the GreetReconciler struct is updated to include a recorder object, the main.go needs to fetch the recorder object from the manager and set to the struct.

   if err = (&controllers.GreetReconciler{               Client: mgr.GetClient(),               Scheme: mgr.GetScheme(),               Recorder: mgr.GetEventRecorderFor("greet-controller"), //Obtain the recorder   }).SetupWithManager(mgr); err != nil {       setupLog.Error(err, "unable to create controller", "controller", "Greet")       os.Exit(1)   }

Output events are displayed when we describe the CRD

If the operator is running in local cluster stop it. make generate install run.
Delete the CR if it already deployed using kubectl delete.
Once deployed the operator will display the log message.

In Recorder.Event the "Object" is the displayed as reason of the event.

 r.Recorder.Event(instance, corev1.EventTypeWarning, "Object", fmt.Sprintf("Created - %s ",appName))

Output where the `Recorder.Event` is updated with different value

Update the status when the Custom Resource deployed

Once the Controller and Custom resources are deployed, we can track the status with kubectl get greet/greet-sample -w, in this section we will see the code changes required to include the APPNAME and STATUS so when using kubectl get <crd-resource-name> will provide the info which like in the snapshot.

In the api/v1alpha1/greet_type.go file we need to add the marker like below over the corresponding Greet struct in this case.

//+kubebuilder:printcolumn:name="APPNAME",type="string",JSONPath=".spec.name",description="Name of the app"//+kubebuilder:printcolumn:name="STATUS",type="string",JSONPath=".status.status",description="Status of the app"

In the code snippet of the greet_type.go file, the GreetStatus struct includes new property Status which is of type string.
In the controllers/greet_controller.go reconciler function we check the status and updated it dynamically based on resource status.

// GreetStatus defines the observed state of Greettype GreetStatus struct {        Status string `json:"status,omitempty"`   // ----------> newly added status property}//Don't leave any new line space between the marker // below marker used to display APPNAME and STATUS//+kubebuilder:object:root=true//+kubebuilder:printcolumn:name="APPNAME",type="string",JSONPath=".spec.name",description="Name of the app"//+kubebuilder:printcolumn:name="STATUS",type="string",JSONPath=".status.status",description="Status of the app"//+kubebuilder:subresource:status// +operator-sdk:gen-csv:customresourcedefinitions.displayName="Greet App"// +operator-sdk:gen-csv:customresourcedefinitions.resources="Deployment,v1,\"A Kubernetes Deployment of greet app\""type Greet struct {    metav1.TypeMeta   `json:",inline"`    metav1.ObjectMeta `json:"metadata,omitempty"`    Spec   GreetSpec   `json:"spec,omitempty"`    Status GreetStatus `json:"status,omitempty"`}

The reconciler logic to validate, set and update the status

func (r *GreetReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {    _ = log.FromContext(ctx)    log.Log.Info("Reconciler invoked..")    instance := &greetv1alpha1.Greet{}    if err := r.Get(ctx, req.NamespacedName, instance); err != nil {             r.Recorder.Event(instance, corev1.EventTypeWarning, "Object", "Failed to read Object")             log.Log.Info("Error while reading the object")             return ctrl.Result{},client.IgnoreNotFound(err)    }       appName := instance.Spec.Name       if instance.Spec.Name != "" {      log.Log.Info(fmt.Sprintf("appName for CRD is - %s ",instance.Spec.Name))          r.Recorder.Event(instance, corev1.EventTypeWarning, "Greet", fmt.Sprintf("Created - %s ",appName))       } else {          log.Log.Info("instance.Spec.Name - NOT FOUND")       }      // update the ok when it is blank      if instance.Status.Status == "" {          instance.Status.Status = "OK"          log.Log.Info("instance.Spec.Name - is set to OK")      }      // update the status with client ------------> Only status will be updated      if err := r.Status().Update(ctx, instance); err != nil {             log.Log.Info("Error while reading the object")             return ctrl.Result{},client.IgnoreNotFound(err)      }      return ctrl.Result{}, nil}

Output for status change

Stop the operator if it was deployed in local cluster.
Delete the Custom Resource if it was deployed to local cluster
Deploy the operator to local cluster using make generate install run
Deploy the custom resource yaml manifest to the local cluster using kubectl apply -f config/samples/.
Withkubectl get greet/greet-sample -w command we can monitor the resource status.

Points that where not covered

Deploying the operator in actual cluster by creating an image.
We can package the CRDs to helm and deploy it via helm CLI

]]>

Automate repetitive tasks using Shell script in Git Bash

Thirumurthi S — Sat, 10 Jun 2023 22:42:02 GMT

Automate repetitive tasks using shell scripts

In day to day application development especially building Microservice architecture based application using Spring Boot and Kubernetes, during development we tend to build artifacts and images multiple times for validating the changes.

In this blog have detail how to automate day to day development tasks like using mvn and kubectl commands, both the commands configured in the Git Bash as alias in the .bashrc file. Demonstrated how to run the commands in bash terminal by passing arguments.

Prerequisites

Basic understanding of Shell scripting
Kind CLI installed (Docker Desktop)
Git Bash Installed

Automate mvn command as a alias in Git Bash

To set an alias command for maven command
- Add below command in $HOME/.bashrc file in windows.

alias runMvn='mvnFunc(){ POMFILE="pom.xml"; if [ -f "$POMFILE" ]; then mvn "$@" clean install; else echo "$POMFILE not found"; fi; unset -f mvnFunc; }; mvnFunc'

If the Git Bash is already opened, to apply the changes to reflect on the open shell issue $ source ~/.bashrc.
About the above code,
- Defines a simple shell function, which checks is the pom.xml file exist in the current directory.
- The usage of "$@" is to use the argument in the mvn command passed as argment.

Command usage in Git Bash

Executing the alias command (without any argument) like below will run mvn clean install
```
$ runMvn
```
Executing alias command (with argument -DskipTest) like below will execute mvn -DskipTests clean install

$ runMvn -DskipTests

Automate kubectl create and delete resource from yaml file

To automate the kubectl apply and delete resources we create a simple shell script.
The shell script will be configured in the ~/.bashrc,

alias runK8SScript='kubeFunc(){ TMPPWD=$(pwd); cd /c/shellScript/; sh runK8SScript.sh "$@" ; cd $TMPPWD; unset -f kubeFunc; }; kubeFunc'

Create a Script (.sh) file named runK8SScript.sh with the below content.
- Below script can be extended say if we have mulitple yaml file we can utilize it.
- Likewise we can use helm to deploy charts as well.
Logic in below script is self explanatory, script uses getopts, to get user option based on which the kubectl apply and kubectl delete command will be executed.

#!/usr/bin/env bashACTION=handleResource(){  INPUT_ACTION=$1  CONTEXT=$2  MANIFEST_PATH=$3  MANIFEST_FILE=$4  echo "[DEBUG] Executing .. kubectl --context $CONTEXT $INPUT_ACTION -f $MANIFEST_PATH$MANIFEST_FILE"  # if namespace not provied within the manifest   # pass it to this function to a new variable  kubectl --context $CONTEXT $INPUT_ACTION $MANIFEST_PATH$MANIFEST_FILE}info() { echo -e "$0 -<option> <action> \nusage:" && grep " .)\ #" $0 | sed 's/)\ ##//g' | sed 's/\([cd] \)/-\1/g';}while getopts ":c:d:" option; do  case $option in    c) ## create      echo "option provided $OPTARG"      ACTION="C"      ;;    d) ## delete      echo "option provided $OPTARG"      ACTION="D"      ;;    *)        echo "not correct usage, use "       info;       exit 0;  esacdoneif [[ -z "$ACTION" ]]; then   echo "other available options"  infofi# using kind and the cluster context is definedCONTEXT_TO_DEPLOY="kind-demo1"# use / at the endMANIFEST_FOLDER="/c/shellScript/manifest/"#yaml fileMANIFEST_FILE_NAME="nginxDeploy.yaml"if [[ "$ACTION" == "C" || -z "$ACTION" ]]; then    SKIP=0    read -p "Do you want to create resource (y/n)? - " input    if [[ ! -z "$input" && !( "$input" == "y" || "$input" == "Y" || "$input" == "yes" || "$input" == "YES" ) ]]; then        echo "Resource creation ignored"       SKIP=1    fi    if (( $SKIP == 0 )); then       echo "Creating resources"      handleResource "apply" $CONTEXT_TO_DEPLOY $MANIFEST_FOLDER $MANIFEST_FILE_NAME    fifiif [[ "$ACTION" == "D" || -z "$ACTION" ]]; then     SKIP=0    read -p "Do you want to delete resource (y/n)? - " input    if [[ ! -z "$input" && !( "$input" == "y" || "$input" == "Y" || "$input" == "yes" || "$input" == "YES" ) ]]; then        echo "Resource deletion ignored"       SKIP=1    fi    if (( $SKIP == 0 )); then         echo "Deleting resource"        handleResource "delete" $CONTEXT_TO_DEPLOY $MANIFEST_FOLDER $MANIFEST_FILE_NAME    fifi

Usage of the alias command

$ runK8SScript

Output with no argument

$ runK8SScript -c create

Output with create argument

$ runK8SScript -d delete

Output with delete argument

]]>

Develop REST API with Javalin and running app using JBang

Thirumurthi S — Sat, 27 May 2023 05:34:46 GMT

REST API with Javalin and JBang

In this blog we will create a REST API using Javalin and running it using JBang.

Pre-requisites:

JBang CLI installed

What is Javalin?

Javalin is a very lightweight web framework for Java
Javalin comes with a Jetty embedded server
Very easy to build REST API

For more details refer the documentation

JBang

With JBang we can run Java class as a script.
With the JBang here there is no specific Java project structure created by Maven or Gradle.
Like Shell script she bang, we do have specify JBang in the Java file.
The dependency are provided at the top of the file within the java comment style.

Possible usage

Since we are not using the Maven or Gradle style project structure, this can be used for spinning up the REST API as a backend with mocked JSON payload and used for building consumer application.

Java code

With the Javalin library we initally configure the app
In this class the example of GET and POST request is demonstrated
Also we read a json file and serve the data in JSON format

///usr/bin/env jbang "$0" "$0" : exit $?//DEPS io.javalin:javalin:5.5.0//DEPS org.projectlombok:lombok:1.18.26//DEPS org.slf4j:slf4j-simple:2.0.7//DEPS commons-io:commons-io:2.11.0//DEPS com.fasterxml.jackson.core:jackson-databind:2.15.1import io.javalin.Javalin;import lombok.AllArgsConstructor;import lombok.Data;import lombok.Getter;import lombok.NoArgsConstructor;import lombok.Setter;import lombok.extern.slf4j.Slf4j;import java.io.File;import java.io.FileInputStream;import java.io.InputStream;import java.io.IOException;import org.apache.commons.io.IOExceptionList;import org.apache.commons.io.IOUtils;import java.nio.charset.StandardCharsets;import java.nio.file.Paths;import java.util.Map;import java.util.List;import java.util.ArrayList;import java.util.HashMap;import com.fasterxml.jackson.core.type.TypeReference;import com.fasterxml.jackson.databind.ObjectMapper;@Slf4jpublic class RESTApi {    static String fileName ="data.json";    static List<Items> items = new ArrayList<>();    public static void main(String[] args){        loadItems();        var app = Javalin.create(config -> {            config.plugins            .enableCors(cors -> cors.add(host -> host.allowHost("localhost")));        })        .get("/data",ctx -> ctx.json(readDataFromFile(fileName)))        .get("/hello/{name}",ctx -> ctx.result("Welcome "+ctx.pathParam("name")))        .get("/items",ctx-> ctx.json(getItems()))        .post("/item", ctx -> {                    ctx.status(200);                    addItem(ctx.bodyAsClass(Items.class));})        .start(7070);    }    private static Items addItem(Items item){        log.info("input data {}",item);        if(item != null){            items.add(item);        }        return item;    }    private static List<Items> getItems(){        return items;    }    private static List<Items> loadItems(){        Items computer = new Items("Computer",2);        Items keyBoard = new Items("KeyBoard",4);        Items mouse = new Items("Mouse",4);        items.add(computer);        items.add(keyBoard);        items.add(mouse);        return items;            }    public static String readDataFromFile(String fileName){        Map<String, Object> valueMap = new HashMap<>();        String data = null;        try{            ObjectMapper mapper = new ObjectMapper();            valueMap = mapper.readValue(Paths.get(fileName).toFile(),                                 new TypeReference<Map<String,Object>>(){});            data = new ObjectMapper().writeValueAsString(valueMap);        }catch(Exception ex){            ex.printStackTrace();        }        return data;    }}@Data@NoArgsConstructor@AllArgsConstructorclass Items{    String product = "";    int quantity =0;}

Sample data json file

{    "companyName" : "xyzABC",    "address" : {        "addressLine1": "Street",        "addressLine2": "Apt Number",        "city" : "City",        "zipCode" : "123456"    },    "departments": [        {"name":"Accounts",         "employees":[            {"name":"employee1","id":"1"},            {"name":"employee2","id":"2"}         ]        }    ] }

Output

$ curl http://localhost:7070/hello/JavalinWelcome Javalin

List the loaded items

$ curl http://localhost:7070/items[{"product":"Computer","quantity":2},{"product":"KeyBoard","quantity":4},{"product":"Mouse","quantity":4}]

List of added item above

$ curl http://localhost:7070/items[{"product":"Computer","quantity":2},{"product":"KeyBoard","quantity":4},{"product":"Mouse","quantity":4},{"product":"Phone","quantity":2}]

Add item

$ curl -X POST -d '{"product":"Phone","quantity":"2"}' http://localhost:7070/item

Javalin startup logs

]]>

Spring Boot NullKeySerializer - Handle Kafka payload with null keys

Thirumurthi S — Mon, 27 Mar 2023 02:56:45 GMT

Spring Boot NullKeySerializer - Handle Kafka payload with null keys

In this article detailed how the default Spring Boot Kafka JSON Serializer can be extended with NullKeySerializer implementation to handle payload when message with map contains null keys. The default JSON Serializer throws exception when payload message to Kafka topic with null keys.

Pre-requisites:

Kafka cluster running in 9092 port

In Message Oriented Middleware (MOM) based distributed application, the payload messages communicate between system over network. There are scenarios where the payload may contain maps with null key. As mentioned above default Spring Boot JSON serializer will throw exception and exception message looks like below.

Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: org.apache.kafka.common.errors.SerializationException: Can't serialize data [MapInputData(messageInfo={null=the key is null explicitly, 1=the key is one non-null}, date=Sun Mar 26 14:40:29 PDT 2023)] for topic [input-topic]] with root cause

Brief steps on how to handle null key in the payload:

First the extend the default spring JSON serializer class org.springframework.kafka.support.serializer.JsonSerializer and implement the NullKeySerializer.
- Next, the extended custom serializer class should be configured in the spring kafka producer spring.kafka.producer.value-serializer property.

Code implementation

The complete spring custom serialzier code that implements NullKeySeralizer:
- In this class the Jackson ObjectMapper is created and configured the implemented NullKeySerializer. The ObjectMapper is also configured to load additional modules, to handle the Date in the payload.
  - The ObjectMapper is passed to the Spring Boot default JsonSerializer.

package com.example.kafka;import com.fasterxml.jackson.core.JsonGenerator;import com.fasterxml.jackson.databind.SerializationFeature;import com.fasterxml.jackson.databind.SerializerProvider;import com.fasterxml.jackson.databind.ser.std.StdSerializer;import com.fasterxml.jackson.databind.util.StdDateFormat;import org.springframework.kafka.support.serializer.JsonSerializer;import com.fasterxml.jackson.databind.ObjectMapper;import java.io.IOException;public class CustomKafkaJsonSerializer extends JsonSerializer<Object> {    public CustomKafkaJsonSerializer() {        super(customizedObjectMapper());    }    private static ObjectMapper customizedObjectMapper() {        ObjectMapper objMapper= new ObjectMapper();        //if the pay load include timestamps we need to use modules        objMapper.findAndRegisterModules();        objMapper.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);        objMapper.setDateFormat(new StdDateFormat().withColonInTimeZone(true));        objMapper.getSerializerProvider().setNullKeySerializer(new NullKeySerializer());        return objMapper;    }    // Null key serializer    static class NullKeySerializer extends StdSerializer<Object> {        public NullKeySerializer() {            this(null);        }        public NullKeySerializer(Class<Object> t) {            super(t);        }        @Override        public void serialize(Object nullKey, JsonGenerator generator, SerializerProvider unused)                throws IOException {            generator.writeFieldName("null");        }    }}

Spring Boot application entry point

package com.example.kafka;import org.springframework.boot.SpringApplication;import org.springframework.boot.autoconfigure.SpringBootApplication;import org.springframework.boot.web.client.RestTemplateBuilder;import org.springframework.context.annotation.Bean;import org.springframework.web.client.RestTemplate;@SpringBootApplicationpublic class KafkaApplication {    public static void main(String[] args) {        SpringApplication.run(KafkaApplication.class, args);    }}

Controller code, which creates the payload message mocking Map with null key.

package com.example.kafka;import com.example.kafka.model.MapInputData;import com.example.kafka.service.ProducerService;import lombok.extern.slf4j.Slf4j;import org.springframework.web.bind.annotation.GetMapping;import org.springframework.web.bind.annotation.RequestMapping;import org.springframework.web.bind.annotation.RestController;import java.util.Date;import java.util.HashMap;import java.util.Map;@RestController@RequestMapping("/api")@Slf4jpublic class SimpleController {    ProducerService producerService;    public SimpleController(ProducerService producerService){        this.producerService=producerService;    }    @GetMapping("/send")    public void sendMessage(){        MapInputData mapInputData = new MapInputData();        Map<String,Object> input = new HashMap<>();        input.put(null,"the key is null explicitly");        input.put("1","the key is one non-null");        mapInputData.setDate(new Date());        mapInputData.setMessageInfo(input);        producerService.sendMessage(mapInputData);    }}

Payload POJO definition

package com.example.kafka.model;import com.fasterxml.jackson.annotation.JsonTypeInfo;import lombok.Data;import java.util.Date;import java.util.Map;@Data@JsonTypeInfo(         use = JsonTypeInfo.Id.NAME,        property = "type")public class MapInputData {    Map<String, Object> messageInfo;    Date date;}

Simple Producer service, that sends the message to Kafka cluster

package com.example.kafka.service;import java.text.MessageFormat;import com.example.kafka.model.MapInputData;import org.springframework.beans.factory.annotation.Value;import org.springframework.kafka.core.KafkaTemplate;import lombok.extern.slf4j.Slf4j;import org.springframework.kafka.support.KafkaHeaders;import org.springframework.messaging.Message;import org.springframework.messaging.support.MessageBuilder;import org.springframework.stereotype.Component;@Component@Slf4jpublic class ProducerService{    @Value("${spring.kafka.topic:demo-topic}")    String topicName;    private final KafkaTemplate<String,Object> kafkaTemplate;    public ProducerService(KafkaTemplate<String,Object> kafkaTemplate){        this.kafkaTemplate = kafkaTemplate;    }    public String sendMessage(MapInputData messageModel){        log.info("Sending message from producer - {}",messageModel);        Message message = constructMessage(messageModel);        kafkaTemplate.send(message);        return MessageFormat.format("Message Sent from Producer - {0}",message);    }    private Message constructMessage(MapInputData messageModel) {        return MessageBuilder.withPayload(messageModel)                .setHeader(KafkaHeaders.TOPIC,topicName)                .setHeader("reason","for-Local-validation")                .build();    }}

Application configuration

spring:  application:    name: service-1  kafka:    topic: input-topic    consumer:      bootstrap-servers: localhost:9092      group-id: input-group-id      auto-offset-reset: earliest      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer      value-deserializer: org.apache.kafka.common.serialization.StringDeserializer    producer:      bootstrap-servers: localhost:9092      key-serializer: org.apache.kafka.common.serialization.StringSerializer      # The custom null serializer added to configuration      value-serializer: com.example.kafka.CustomKafkaJsonSerial

With the above spring configuration the producer can serialize the payload with null key and send message over wire to Kafka broker.

pom.xml with dependencies

<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">    <modelVersion>4.0.0</modelVersion>    <parent>        <groupId>org.springframework.boot</groupId>        <artifactId>spring-boot-starter-parent</artifactId>        <version>3.0.0</version>        <relativePath/>    </parent>    <groupId>com.example</groupId>    <artifactId>kafka</artifactId>    <version>0.0.1-SNAPSHOT</version>    <name>kafka</name>    <description>kafka</description>    <properties>        <java.version>17</java.version>    </properties>    <dependencies>        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-starter</artifactId>        </dependency>        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-starter-web</artifactId>        </dependency>        <dependency>            <groupId>org.springframework.kafka</groupId>            <artifactId>spring-kafka</artifactId>        </dependency>        <dependency>            <groupId>org.projectlombok</groupId>            <artifactId>lombok</artifactId>        </dependency>        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-starter-test</artifactId>            <scope>test</scope>        </dependency>    </dependencies>    <build>        <plugins>            <plugin>                <groupId>org.springframework.boot</groupId>                <artifactId>spring-boot-maven-plugin</artifactId>            </plugin>        </plugins>    </build></project>

]]>

Run Integration in Apache Camel-K

Thirumurthi S — Thu, 09 Feb 2023 05:46:59 GMT

Camel-K

Apache Camel K is a lightweight cloud-integration platform that runs natively on Kubernetes.
In this blog will show how to install Camel-K operator in Kubernetes (KIND) cluster.
Have created a simple integration code (Java integration).
- The integration code expose a REST endpoint using Camel component.
- The message is passed as URL path variable.
- The message is routed to direct endpoint and transform with a constant string.

Pre-requisites

Docker Desktop (Docker daemon running).
Dockerhub account.
Kamel CLI installed.
KIND CLI installed.
Basic understanding on Kubernetes and Operator pattern.

Options to install Camel-K operator

There are different option to install Camel-K. Options are listed below,
- using Helm charts
- using OperatorHub
- using Kamel CLI
In this blog the Camel-K operator is installed with Kamel CLI in KIND cluster.

Note:-During Camel-K installation we need to configure the docker registry (docker.io).In this blog have used Dockerhub, we can use private registry as well.

Install KIND cluster and deploy Nginx Ingress controller

The ingress controller is used to access the REST endpoint from the host machine.
Below is the KIND cluster configuration file which includes patch configuration for ingress controller, save this as kamel-kind-cluster.yaml.

kind: ClusterapiVersion: kind.x-k8s.io/v1alpha4nodes:- role: control-plane  kubeadmConfigPatches:  - |    kind: InitConfiguration    nodeRegistration:      kubeletExtraArgs:        node-labels: "ingress-ready=true"  extraPortMappings:  - containerPort: 80    hostPort: 80    protocol: TCP  - containerPort: 443    hostPort: 443    protocol: TCP

We need to use the above configuration in Kind CLI to create the cluster, the command usage is listed below.

kind create cluster --config kamel-kind-cluster.yaml --name kamel-ingress

In order to apply the ingress controller configuration below command. Refer kind documentation for more info.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml

Install Camel-K operator in KIND cluster

To install the Camel-K operator with Kamel CLI, we need to configure the docker registry (docker.io). For this first we need to create a secret in the cluster with the Docker credentials.

Note:-
Creating a secret with the $HOME/.docker/config.json file directly didn't work for me. When running the integration, the logs indicated unauthorized access.
Below is the command to create secret from docker config file.
kubectl create secret generic regcred \ --from-file=.dockerconfigjson=$HOME/.docker/config.json \ --type=kubernetes.io/dockerconfigjson

Install OLM in the cluster

This is optional step. By installing the OLM (Operator Lifecycle Management) we are not required to use the option --olm=false in Kamel CLI during installation.

To install the OLM, run below command from Gitbash.

curl -sL https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.23.1/install.sh | bash -s v0.23.1

To create the secret with the Docker credentials use below commands
- Storing the Dockerhub credentials in the environment variable will help in case we need to automate the installation process.

# from powershell use below command, to accessing environment variable we use $env:<env-key>kubectl -n default create secret docker-registry external-registry-secret --docker-username $env:DOCKER_USER --docker-password $env:DOCKER_PASS# from shell (like git bash) use below commandkubectl -n default create secret docker-registry external-registry-secret --docker-username $DOCKER_USER --docker-password $DOCKER_PASS

Below command will install the Camel-K operator in default namespace.

# the secret created above is passed in --registry-secret in this commandkamel install -n default --registry docker.io --organization <organization-name> --registry-secret external-registry-secret --wait

If OLM is not installed in the cluster (as mentioned above), in this case then we need to use --olm=false in the command shown below.

kamel install -n default --olm=false --registry docker.io --organization <organization-name> --registry-secret external-registry-secret --wait

Info:
Camel-K operator installation takes sometime to install.Camel-K operator deploys as a pod, so to check the status of the pod in the default namespace use kubectl get pods.

Running a simple integration

We can use Kamel CLI to run integration code developed in Java, groovy, etc.
The Camel-K operator gets deployed to cluster as a pod, we can simply check the status of the pod and make sure it is in running state before creating an integration.
Below is a simple Java integration code, which uses Camel RouteBuilder and REST component to expose REST endpoint and routes the message.

import org.apache.camel.*;import org.apache.camel.builder.RouteBuilder;public class WelcomeRoute extends RouteBuilder {   @Override   public void configure() throws Exception {      restConfiguration().bindingMode("auto");        rest("/api")            .get("/demo/{info}")            .to("log:info")            .to("direct:msg");        from("direct:msg")                .transform().simple("msg received - ${header.info}");   }}

During development we can run the integration using --dev option, the command to create and run the integration looks like below.

kamel run WelcomeRoute.java --dev

Using --dev will print the logs in console.
With this flag any changes to the Java file will reflect immediately in the integration real time.
Below is the console log output of successfully deployed integration.

Condition "Ready" is "True" for Integration welcome-route: 1/1 ready replicas[1] 2023-02-09 02:36:48,267 INFO  [org.apa.cam.k.Runtime] (main) Apache Camel K Runtime 1.16.0[1] 2023-02-09 02:36:48,325 INFO  [org.apa.cam.qua.cor.CamelBootstrapRecorder] (main) Bootstrap runtime: org.apache.camel.quarkus.main.CamelMainRuntime[1] 2023-02-09 02:36:48,334 INFO  [org.apa.cam.mai.MainSupport] (main) Apache Camel (Main) 3.19.0 is starting[1] 2023-02-09 02:36:48,517 INFO  [org.apa.cam.k.lis.SourcesConfigurer] (main) Loading routes from: SourceDefinition{language='java', type='source', location='file:/etc/camel/sources/.\WelcomeRoute.java', }[1] 2023-02-09 02:36:55,022 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main) Apache Camel 3.19.0 (camel-1) is starting[1] 2023-02-09 02:36:55,161 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main) Routes startup (started:2)[1] 2023-02-09 02:36:55,162 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main)     Started route1 (direct://msg)[1] 2023-02-09 02:36:55,162 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main)     Started route2 (rest://get:/api:/demo/%7Binfo%7D)[1] 2023-02-09 02:36:55,163 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main) Apache Camel 3.19.0 (camel-1) started in 732ms (build:0ms init:605ms start:127ms)[1] 2023-02-09 02:36:55,874 INFO  [io.quarkus] (main) camel-k-integration 1.11.1 on JVM (powered by Quarkus 2.14.0.Final) started in 18.400s. Listening on: http://0.0.0.0:8080[1] 2023-02-09 02:36:55,879 INFO  [io.quarkus] (main) Profile prod activated.[1] 2023-02-09 02:36:55,889 INFO  [io.quarkus] (main) Installed features: [camel-attachments, camel-bean, camel-core, camel-direct, camel-java-joor-dsl, camel-k-core, camel-k-runtime, camel-kubernetes, camel-log, camel-platform-http, camel-rest, cdi, kubernetes-client, security, smallrye-context-propagation, vertx]

Kamel CLI can be used to check the status of integration, use below command.

kamel get

If the integration is running on a specific namespace we can use kame get -n <namespace-name>.

By default Camel-K creates a ClusterIP service for this integration.

The endpoints exposed can be checked using the command kubectl get endpoint

By default the Camel-K operator creates service as ClusterIP. In case if we want to expose the service as NodePort or LoadBalancer, we can use Service Traits to configure. The command looks like below.
- There are different traits supported by Camel-K, refer the Camel-K traits documentation.

kamel run --trait service.enabled=true --trait service.node-port=true --trait service.type=NodePort -n <namespace> <file.java> --dev

Apply ingress rule to access the endpoint

To access the REST endpoint from the host machine, we use Nginx Ingress control already deployed in cluster.
We need a ingress rule configuration to route traffic to access the service.
The ingress rule configuration manifest content is listed below save this in a file named ingress-rule.yaml.

apiVersion: networking.k8s.io/v1kind: Ingressmetadata:  name: kamel-welcome-route  annotations:    nginx.ingress.kubernetes.io/rewrite-target: /$2spec:  rules:  - http:      paths:      - pathType: Prefix        path: /(/|$)(.*)        backend:          service:            name: welcome-route            port:              number: 80

To apply the ingress rule use kubectl apply -f ingress-rule.yaml command.
To check the ingress status use kubectl get ingress command.

Access the REST endpoint and output

Once the integration is deployed successfully and in running state, we can use wget to get access the endpoint. Below is the complete command.

wget -q -O - localhost/api/demo/hello

Below is the operator and integration pod running in cluster, have used Lens to connect to the kind cluster.

Passing properties to the integration code

For a simple groovy script, where the my.message is fetched from the properties. Save the below content to a file Message.groovy.

from('timer:props?period=1000')    .log('{{my.message}}')

To pass the property values, since we are not using --dev, the integration will be created

kamel run --property my.message="Example-From-Property" Message.groovy

Output looks like below

[1] 2023-02-09 03:46:30,116 INFO  [route1] (Camel (camel-1) thread #1 - timer://props) Example-From-Property[1] 2023-02-09 03:46:31,090 INFO  [route1] (Camel (camel-1) thread #1 - timer://props) Example-From-Property

Delete an integration using Kamel CLI

Kamel CLI command kamel get can be used to list the integration status.
To delete the integration, we need to use below command

kamel delete <name-of-integration>

Additional information

We can use ConfigMap with the configuration or properties file and pass it in runtime using Kamel CLI to the integration code. Refer the Camel-K runtime configuration documentation
Promoting the integration to different environment, we can do it as well refer the Camel-K promoting documentation

]]>

Execute Java code like a script using JBang

Thirumurthi S — Sun, 29 Jan 2023 07:46:29 GMT

JBang

In this blog have explained the basic usage of JBang CLI command. With additional specification we will be able to run Java code without using any source code structure.

What is JBang?

JBang is a CLI tool which can run java code directly from a file.
With JBang CLI we can execute java code from file by passing it as a argument.
- In order to execute the Java file by JBang CLI, the file shouls include special defintions within java comments, which is explained in the example.
JBang can also execute .jsh (java shell) file. The .jsh file is similar to .java file, but it doesn't require class definition or main method.
- Any code that executes in JShell REPL editor, can be places in .jsh file.
JBang can also be used to install JDK's and list installed JDK's, using command jbang jdk list.

Install JBang

JBang CLI can be installed in different ways refer installation docs.
In my Windows machine I used chocolatey manager.

Execute first Java code in JBang

JBang CLI requires the environment to be defined in very first line of the java file, like below.
This is similar to shell shebang, where we use #!.

///usr/bin/env jbang "$0" "$@" ; exit $?

Save below code snippet as addTwoNumbers.java

///usr/bin/env jbang "$0" "$@" ; exit $?class addTwoNumbers {    public static void main(String[] args) {        if(args.length==0) {            System.out.println("No arguments passed!!");        } else {            try{              int num1 = Integer.parseInt(args[0]);              int num2 = Integer.parseInt(args[1]);              System.out.println("Sum of "+num1+" + "+num2+" = "+(num1+num2));             } catch (NumberFormatException ex){                 System.out.println("ERROR:- only whole number is supported");             }        }    }}

JBang command to run Java file

> jbang addTwoNumbers.java 5 5Sum of 5 + 5 = 10

JBang can resolve Maven dependencies

JBang CLI can resolve Maven or Gradle dependencies from the Java file.
JBang requires the dependencies to be specified in a specific format which should start with //DEPS and looks like below.

//DEPS groupId:artifactId:version

Defining Maven BOM dependencies in the Java file. Note, the suffix @pom, is required when defining the bom dependencies.
- Points to note when using bom dependencies,
  - The bom definition should be specified before using the related dependencies.
  - Single Java file that is executed by JBang can only contain one bom definition.

//DEPS groupId:artifactId:version@pom

Example for using camel bom dependencies, not nee to specify the versions for the realted dependencies.

//DEPS org.apache.camel:camel-bom:3.20.1@pom//DEPS org.apache.camel:camel-core//DEPS org.apache.camel:camel-main

Sample code defining maven dependencies for JBang

The java code below uses camel dependencies. It simply generates random number every 2.5 seconds.
Save below code snippet in CamelDemo.java file.

///usr/bin/env jbang "$0" "$0" : exit $?//DEPS org.apache.camel:camel-bom:3.20.1@pom//DEPS org.apache.camel:camel-core//DEPS org.apache.camel:camel-main//DEPS org.apache.camel:camel-stream//DEPS org.slf4j:slf4j-nop:2.0.6//DEPS org.slf4j:slf4j-api:2.0.6import org.apache.camel.*;import org.apache.camel.builder.*;import org.apache.camel.main.*;import org.apache.camel.spi.*;import static org.apache.camel.builder.PredicateBuilder.*;import java.util.Random;import static java.lang.System.*;class CamelDemo{    public static void main(String ... args) throws Exception{        out.println("Starting camel route...");        setProperty("org.slf4j.simpleLogger.logFile", "System.out");        Random random = new Random();        Main main = new Main();        main.configure().addRoutesBuilder(new RouteBuilder(){            public void configure() throws Exception{                from("timer:hello?period=2500")               .process(exchange -> exchange.getIn().setBody(random.nextInt(500)))               .setBody(simple("rNum: = ${body}"))               .to("stream:out");            }         });        main.run();    }}

Output of JBang CLI execution

To execute the java file using jbang use the command jbang CamelDemo.java
Initial run will download the defined jar dependencies and output will look like below,
We can use the -verbose switch to print additional logs of jbanglike, jbang --verbose CamelDemo.java

> jbang CamelDemo.java[jbang] Resolving dependencies...[jbang]    org.apache.camel:camel-bom:3.20.1@pom[jbang]    org.apache.camel:camel-core:3.20.1[jbang]    org.apache.camel:camel-main:3.20.1[jbang]    org.apache.camel:camel-stream:3.20.1[jbang]    org.slf4j:slf4j-nop:2.0.6[jbang]    org.slf4j:slf4j-api:2.0.6[jbang] Dependencies resolved[jbang] Building jar...Starting camel route...rNum: = 482rNum: = 289

Use JBang in Docker to run Java file

# in windows when attaching the volume better to use the complete path of the folder where CamelDemo.java existsdocker run -v C:\\simple-camel\\:/ws -w/ws -ti jbangdev/jbang-action --verbose CamelDemo.java

Note:Based on the JBang documentation, the command might looke like below, but note pwd in volume didn't work in windows
docker run -v `pwd`:/ws -w/ws -ti jbangdev/jbang-action --verbose CamelDemo.java

Use JBang to execute Java code directly using URL

We can use JBang CLI to execute the java code directly from the internet as well

> jbang https://github.com/jbangdev/jbang-examples/blob/main/examples/helloworld.java

Consideration

There are certain limitation in using JBang CLI, like it can't be used for complex Java programs.
It can refer mulitple java file under pacakges, refer documentation
To use JBang in production totally depends on the usage, it can be used for small automation process but not for any complex code still this community is growing.

]]>

Deploy Jaeger instance in KIND cluster using Jaeger operator

Thirumurthi S — Wed, 11 Jan 2023 06:06:20 GMT

This blog will explain how to deploy the Jaeger all-in-one image in KIND cluster using the Jaeger operator.

Pre-requisites

Basic understanding of Jaeger architecture
Docker Desktop installed and running
KIND CLI configured

About this article

First, we will see instructions on how to deploy Jaeger instance in the Docker Desktop KIND cluster
Next, we will run the SpringBoot application locally to send traces with OpenTracing.

what is Jaeger?

Jaeger is an open-source project used for distributed tracing. Jaeger project was developed by Uber and is now a CNCF project.

Say, when a request is being processed by multiple microservice the request flow between services can be traced in the Jaeger. The span tracing can be used to analyze the performance since the Jaeger UI provides details of how much time is spent on each service.

Jaeger installation steps

Install KIND cluster with necessary ports exposed
Install cert-manager, using cert-manager yaml
- Based on the Jaeger Installation document it requires cert-manager to be installed in the Kubernetes cluster before installing the Jaeger operator.
Create Namespace and Install Jaeger operator yaml
Create Jaeger instance using CRD manifest
- In this case, have used all-in-one mode Jaeger instance

NOTE:
There are different modes the Jaeger can be deployed, the default mode is all-in-one mode where the trace data will not be persisted.
all-in-one mode is mostly used for local development and data is held in memory.
For production, it is better to use production mode. Refer Jaeger documentation for more details.

KIND configuration

KIND cluster configuration below exposes multiple ports export, it is not necessary to expose all the ports, latter only ports 16686 and 14268 are forwarded.
Save the YAML content as jaeger_kind_cluster.yaml file.

kind: ClusterapiVersion: kind.x-k8s.io/v1alpha4networking:  ipFamily: ipv4  apiServerAddress: 127.0.0.1  apiServerPort: 6443nodes:- role: control-plane  extraPortMappings:  - containerPort: 6831    hostPort: 6831    listenAddress: "127.0.0.1"    protocol: UDP  - containerPort: 6832    hostPort: 6832    listenAddress: "127.0.0.1"    protocol: UDP  - containerPort: 16686    hostPort: 16686    listenAddress: "127.0.0.1"    protocol: TCP  - containerPort: 16685    hostPort: 16685    listenAddress: "127.0.0.1"    protocol: TCP  - containerPort: 9411    hostPort: 9411    listenAddress: "127.0.0.1"  - containerPort: 4317    hostPort: 4317    listenAddress: "127.0.0.1"    protocol: TCP  - containerPort: 4318    hostPort: 4318    listenAddress: "127.0.0.1"    protocol: TCP  - containerPort: 5778    hostPort: 5778    listenAddress: "127.0.0.1"    protocol: TCP  - containerPort: 14250    hostPort: 14250    listenAddress: "127.0.0.1"    protocol: TCP  - containerPort: 14268    hostPort: 14268    listenAddress: "127.0.0.1"    protocol: TCP  - containerPort: 14269    hostPort: 14269    listenAddress: "127.0.0.1"    protocol: TCP  - containerPort: 80    hostPort: 80    protocol: TCP  - containerPort: 443    hostPort: 443    protocol: TCP

cert-manager and jaeger-operator yaml file reference

Save the content from the Cert-Manager/GitHub file to cert-manager.yaml
Save the content from the Jaegertracing/GitHub file to jaeger-opertor.yaml
Below is the Jaeger CRD manifest YAML, which deploys Jaeger instance in KIND cluster.

apiVersion: jaegertracing.io/v1kind: Jaegermetadata:  name: simplest

Shell script to install Jaeger in KIND cluster

Once the yaml files are stored under a directory, run below script in Git Bash to install the Jaeger Instance.

#! /bin/sh# Create kind cluster in Docker desktopkind create cluster --name jaeger-cluster --config jaeger_kind_cluster.yaml# install the cert manager kubectl apply -f cert-manager.yaml# create namespace tracekubectl create ns trace# sleep for 10 secsleep 10# install the jaeger operatorkubectl -n trace apply -f jaeger-operator.yaml# pause for 100 secondssleep 100# install the simple clusterkubectl -n trace apply -f simple-jaeger.yaml

After execution of the shell script, with the below command status and strategy of deployed Jaeger instance can be verified.

$ kubectl -n trace get jaegersNAME       STATUS    VERSION   STRATEGY   STORAGE   AGEsimplest   Running   1.40.0    allinone   memory    10d

Below is the output of status of the deployed Jaeger resources in the KIND cluster

# get the pods name of the jaeger$ kubectl -n trace get pods# output NAME                               READY   STATUS    RESTARTS       AGEjaeger-operator-6787f4df85-ww7mx   2/2     Running   7 (147m ago)   10dsimplest-6b8dbb67c5-kpz5j          1/1     Running   0              146m

# ge the service status$ kubectl -n trace get svc# output sampleNAME                            TYPE        CLUSTER-IP   PORT(S)                                                 jaeger-operator-metrics         ClusterIP   10.96.230.32 8443/TCP                                                jaeger-operator-webhook-service ClusterIP   10.96.137.93 443/TCP                                                 simplest-agent                  ClusterIP   None         5775/UDP,5778/TCP,6831/UDP,6832/UDP                     simplest-collector              ClusterIP   10.96.135.45 9411/TCP,14250/TCP,14267/TCP,14268/TCP,4317/TCP,4318/TCPsimplest-collector-headless     ClusterIP   None         9411/TCP,14250/TCP,14267/TCP,14268/TCP,4317/TCP,4318/TCPsimplest-query                  ClusterIP   10.96.212.68 16686/TCP,16685/TCP

Port forward the UI and the Jaeger collector service port

In the spring application, we will send the traces to the 14268 port, which is the Jaeger collector service running in the cluster.
Based on the Jaeger architecture, the collector service will collect the traces.

# port forward the jaeger pod (the pod name can be fetched from above command)kubectl -n trace port-forward pod/simplest-6b8dbb67c5-kpz5j 16686:16686# port forward the jaeger service kubectl -n trace port-forward svc/simplest-collector 14268:14268

NOTE:
SpringBoot application is developed using 2.7.7 version with OpenTracing dependencies.
SpringBoot 3.0.0 doesn't support OpenTracing, it is deprecated in favor of OpenTelementry.

SpringBoot Application to send traces

The pom.xml file with OpenTracing dependencies.

<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">    <modelVersion>4.0.0</modelVersion>    <parent>        <groupId>org.springframework.boot</groupId>        <artifactId>spring-boot-starter-parent</artifactId>        <version>2.7.7</version>        <relativePath/>    </parent>    <groupId>com.example</groupId>    <artifactId>app</artifactId>    <version>0.0.1-SNAPSHOT</version>    <name>app</name>    <description>application</description>    <properties>        <java.version>17</java.version>    </properties>    <dependencies>        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-starter</artifactId>        </dependency>        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-starter-web</artifactId>        </dependency>        <dependency>            <groupId>io.opentracing.contrib</groupId>            <artifactId>opentracing-spring-jaeger-cloud-starter</artifactId>            <version>3.3.1</version>        </dependency>        <dependency>            <groupId>org.projectlombok</groupId>            <artifactId>lombok</artifactId>        </dependency>        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-starter-test</artifactId>            <scope>test</scope>        </dependency>    </dependencies>    <build>        <plugins>            <plugin>                <groupId>org.springframework.boot</groupId>                <artifactId>spring-boot-maven-plugin</artifactId>            </plugin>        </plugins>    </build></project>

The entry point of SpringBoot application.

package com.example.app;import org.springframework.boot.SpringApplication;import org.springframework.boot.autoconfigure.SpringBootApplication;import org.springframework.boot.web.client.RestTemplateBuilder;import org.springframework.context.annotation.Bean;import org.springframework.web.client.RestTemplate;@SpringBootApplicationpublic class JaegerApplication {    public static void main(String[] args) {        SpringApplication.run(KafkaApplication.class, args);    }    // Used by the Jeager dependencies to intercept http header    @Bean    public RestTemplate restTemplate(RestTemplateBuilder builder){        return builder.build();    }}

Simple Spring controller below exposes two endpoint /api/generate and api/fetch

INFO:
We will execute the same SpringBoot application in different ports (8080 and 8090) on the same host machine.
The endpoint http://localhost:8080/api/fetch will invoke the application running on http:/localhost:8090/api/generate. The traces will be sent to the Jaeger instance by the application.
We can update the Jaeger configuration to send traces in the sample rather than sending all the traces for each request which is not shown here.

package com.example.kafka;import io.jaegertracing.Configuration;import io.jaegertracing.internal.samplers.ConstSampler;import io.jaegertracing.internal.samplers.ProbabilisticSampler;import lombok.extern.slf4j.Slf4j;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.beans.factory.annotation.Value;import org.springframework.boot.web.client.RestTemplateBuilder;import org.springframework.context.annotation.Bean;import org.springframework.http.ResponseEntity;import org.springframework.web.bind.annotation.GetMapping;import org.springframework.web.bind.annotation.RequestMapping;import org.springframework.web.bind.annotation.RestController;import org.springframework.web.client.RestTemplate;import java.util.Random;@RestController@RequestMapping("/api")@Slf4jpublic class AppController {   private RestTemplate restTemplate;   public AppController(RestTemplate restTemplate){       this.restTemplate = restTemplate;   }    Random rand = new Random();    @GetMapping("/generate")    public long calculate(){        long number = rand.nextInt(1000);        log.info("api /generate invoked {}",number);        return number;    }    @Value("${app2.port:8090}")    private String appPort;    @Value("${app2.host:localhost}")    private String hostName;    @GetMapping("/fetch")    public ResponseEntity getNum(){        String url = "http://"+hostName+":"+appPort+"/api/generate";        String resp = restTemplate.getForObject(url, String.class);        log.info("api/fetch invoked - {} = {}",url,resp);        return ResponseEntity.ok("random num : "+resp);    }}

application.yml file defining the OpenTracing Jaeger URL configuration.
The spring application name is not passed during runtime with Java command.

  opentracing:    jaeger:      http-sender:        url: http://localhost:14268/api/traces

Execute the spring boot application services

Use mvn clean install command to create the jar file.
Once jar file is generated, execute two instances of application with the below commands.

java -jar .\target\app-0.0.1-SNAPSHOT.jar --spring.application.name=service-1 --server.port=8080 --debugjava -jar .\target\app-0.0.1-SNAPSHOT.jar --spring.application.name=service-2 --server.port=8090 --debug

After the application is up and running, use curl command to access the REST end-point like below

curl -i http://localhost:8080/api/fetchHTTP/1.1 200 Content-Type: text/plain;charset=UTF-8Content-Length: 16Date: Wed, 11 Jan 2023 02:09:58 GMTrandom num : 771

Once the curl command access the endpoint the traces are sent to Jaeger and we can view the spans from Jaeger UI at http://localhost:16686
INFO
- In Jaeger UI the service name jaeger-query is the default.
- After accessing the REST endpoint and traces are collected the Jaeger UI spans can be viewed under service. This service is the SpringBoot application name. In this case, service-1 and service-2.

Snapshot of the Jaeger UI after shipping the traces

]]>

Deploy Strimzi Apache Kafka Operator in KIND cluster with KRaft mode

Thirumurthi S — Sat, 03 Dec 2022 08:15:34 GMT

Deploy Kafka cluster in KIND cluster

In this article, I will explain to show how we can deploy Strimzi Kafka enabling KRaft mode in KIND cluster within the Docker desktop.
The configuration defined in this article also shows how we can access Kafka broker from the host machine.

Prerequisites :

Basic understanding of Kubernetes and Operator patterns
Basic understanding of Apache Kafka
Docker Desktop installed and running
KIND CLI installed
GNU Make 4.0 installed (for automation)

Apache Kafka Zookeeper Vs KRaft:

Kafka community currently recommends using Zookeeper for leader election in the production environment. Due to lots of limitations Kafka community is moving out of Zookeeper.
KRaft is new way by which Kafka manages quoram between multiple clusters.
Kraft is still an experimental feature and NOT to be used in production yet.

Deploying Kafka in Kubernetes?

To deploy Kafka in Kubernetes the easiest option is Strimzi Apache Kafka Operator.
Strimzi Operators can be configured to deploy Kafka brokers in production.
Strimzi is CNCF supported project.
Persistent datastore in Strimzi has different options like,
- Ephemeral
- JBOD Just a Bunch Of Disks
- Persistent Volumes provided by any Cloud providers
strimzi.io details the steps to deploy a persistent Kafka cluster in KIND.

What does this article include?

We will deploy Strimzi Kafka cluster with 3 brokers and Ephemeral type datastore. With ephemeral configuration, if the broker is destroyed the data will be lost.
How to enables the experimental KRaft mode in Strimzi Operator.
How we can access the Kafka brokers running in KIND cluster from the host machine.
- In Windows machine to access the containers running KIND cluster we need to forward the port, by updating the Strimzi cluster configuration.

> **INFO**> > *   Accessing `Strimzi Apache Kafka` brokers from the outside of kubernetes cluster it requires bootstrap service to be exposed as NodePort. [Refer to this article that explains accessing Strimzi Kafka cluster from an external client](https://strimzi.io/blog/2019/04/23/accessing-kafka-part-2/)>

Why KRaft mode?

The reason to use Kraft mode is on my laptop, deploying the Strimzi Kafka with Zookeeper keeps on restarting the Zookeeper and Kafka Broker pods very often most of the time not accessible. I think this is because of memory limitations my laptop only had 8GB of memory.
KRaft feature does create Zookeeper pods.
Deploying Kafka in KRaft mode with 3 brokers in KIND cluster was always running successfully.
Noticed that when sending messages to the cluster with the console producer bat script there are retry error messages printed in console.
Since it is easy to spin up in a local machine we can use this to develop Kafka based applications locally.

NOTE:-
The same Strimzi Cluster configuration will work for non- kRaft mode where Zookeeper is used, the configuration requires including the Entity operator and Topic operator which can have empty values.

Steps to install Strimzi Apache Kafka in KIND cluster

1. Spin up the KIND cluster with the required port forward configuration
2. Create a namespace called kafka
3. Install the Strimzi Operator (This can be done using Helm chart as well)
4. Enable KRaft feature
5. Install the Strimzi cluster using the CRD configuration
6. Create Topic using Strimzi CRD in the Kafka broker
7. Connect to Strimzi Kafka broker using Apache Kafka binary bat script

INFO
Later in this article, we will see how to automate the process using GNU make .

Step 1: Create KIND cluster with port forward configuration

To create Kind cluster we can use the below configuration which includes a list of port that forwards the network traffic to the cluster. Since we use GNU makesave this in a file named kafka_kind_cluster.yaml.

kind: ClusterapiVersion: kind.x-k8s.io/v1alpha4networking:  ipFamily: ipv4  apiServerAddress: 127.0.0.1  apiServerPort: 6443nodes:- role: control-plane  extraPortMappings:  - containerPort: 31092    hostPort: 9092    listenAddress: "127.0.0.1"    protocol: TCP  - containerPort: 31234    hostPort: 31234    listenAddress: "127.0.0.1"    protocol: TCP  - containerPort: 31235    hostPort: 31235    listenAddress: "127.0.0.1"  - containerPort: 31236    hostPort: 31236    listenAddress: "127.0.0.1"

If Docker Desktop is running, execute the below command to create KIND cluster

kind create cluster --name=kafka --config=kafka_kind_cluster.yaml

docker ps will display the ports that forwards the traffic like below

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES e5c4d36547af kindest/node:v1.25.2 "/usr/local/bin/entr" About an hour ago Up About an hour 127.0.0.1:6443->6443/tcp, 127.0.0.1:31234-31236->31234-31236/tcp, 127.0.0.1:9092->31092/tcp kafka-control-plane

Step 2: Create a namespace

We can create the namespace in the cluster Imperatively using kubectl command or Declaratively using YAML manifest.

kubectl create ns kafka

Step 3: Install Strimzi operator

Strimzi can also be installed using Helm chart as well, the steps are available in Strimzi documentation.

# namespace used is kafkakubectl create -f 'https://strimzi.io/install/latest?namespace=kafka' -n kafka

Step 4: Enable KRaft feature

Kraft mode requires the feature UseStimziPodSets also to be enabled, since this feature is already enabled in Strimzi Version 0.32.0 no action is required.
The enable Kraft mode we need to set the environment variable in Strimzi Operator. Before enabling the KRaft feature we ned to make sure the Operator is in Running status using kubectl -n kafka get pods,svc,deployment command.
Executing the command below will enable the KRaft feature in the operator eventually restarts it.

kubectl -n kafka set env deployment/strimzi-cluster-operator STRIMZI_FEATURE_GATES=+UseKRaft

Step 5: Install Kafka cluster using Strimzi CRD definition

This section will detail the Strimzi CRD to define Kafka cluster configuration that will spin up 3 brokers with Ephemeral type storage.

INFO
The listener configuration exposes the service as NodePort with the specified port.
name: externaltype: NodePortport: 9094
When the Kafka client connects to the broker the bootstrap server sends the list of brokers with the leader info, by using advertisedHost: 127.0.0.1 the host machine can connect to the broker.
Additionally, these broker ports are used in the KIND cluster configuration to forward the traffic to Kafka broker.
The configuration below still specifies the Zookeeper details since KRaft is an experimental feature. We don't need to specify the Entity Operator and Topic Operator for KRaft mode.

The configuration below defines 3 brokers, expose service as Nodeport and Storage type as Ephemeral. Save the content to file kraftmode.yaml

apiVersion: kafka.strimzi.io/v1beta2kind: Kafkametadata:  name: my-cluster  namespace: kafkaspec:  kafka:    version: 3.2.3    replicas: 1    listeners:      - name: plain        port: 9092        type: internal        tls: false      - name: external        port: 9094        type: nodeport        tls: false        configuration:          bootstrap:            nodePort: 31092          brokers:            - broker: 0              advertisedHost: 127.0.0.1              nodePort: 31234            - broker: 1              advertisedHost: 127.0.0.1              nodePort: 31235            - broker: 2              advertisedHost: 127.0.0.1              nodePort: 31236    config:      offsets.topic.replication.factor: 3      transaction.state.log.replication.factor: 3      transaction.state.log.min.isr: 2      default.replication.factor: 3      min.insync.replication: 2      inter.broker.protocol.version: "3.2"    storage:      type: ephemeral  zookeeper:    replicas: 3    storage:      type: ephemeral

Once Kraft feature is enabled and the Strimzi operator is in Running state, execute the command below to deploy the Kafka cluster.

kubectl -n kafka apply -f kraftMode.yaml

To track the status of the resource execute the command below.

kafka -n kafka get pods,deploy,svc

Step 6: Create Topic using Strimzi CRD

The configuration to create a Topic using the CRD is listed below, save the content to a file named topic.yaml

apiVersion: kafka.strimzi.io/v1beta2kind: KafkaTopicmetadata:    name: example-demo    namespace: kafka    labels:      app: example-app      strimzi.io/cluster: kafkaspec:   topicName: example-demo   partitions: 3   replicas: 3

To create the topic and view the details of the created topic use the commands below

# to create the topickubectl -n kafka apply -f topic.yaml# to view the created topickubectl -n kafka get kafkatopics

Step 7: Access the Kafka cluster

We can access the kafka broker using any Kafka client, the bootstrap server now is accessible via localhost:9092
Download the binaries from Apache kafka website, and use the kafka-console-producer.bat, kafka-topic.bat, etc. to connect the brokers.
The command below uses the producer script to send messages.

kafka-console-producer.bat --bootstrap-server localhost:9092 --topic example-demo aks=all

# once messages are sent we can list the topic with below commandkafka-topic.bat --bootstrap-server localhost:9092 --list

Using the Makefile

The Makefile below automates part of the process of deploying the operator, creating the cluster and deleting the kind cluster.

Note:
Save below content in a file called Makefile, make sure the recipe under the target starts with a tab space else make command will error out.
This Makefile contains additional targets like installing kind cli and install kubectl, etc. but this only works on Linux machines.

SHELL :=/bin/bashNAMESPACE ?= kafkaCLUSTERNAME ?= kafkaSLEEPSEC ?= 5000#KAFKAEPHEMERALCONFIG := kafka-ephemeral-single.yamlKAFKAEPHEMERALCONFIG := kraftmode.yamlKAFKACLUSTER := kafka_kind_cluster.yaml.DEFAULT_GOAL :=info.PHONY: install-kind-linuxinstall-kind-linux:     @curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.17.0/kind-linux-amd64     @chmod +x ./kind     @mv ./kind /usr/local/bin/kind     @chmod +x /usr/local/bin/kind  .PHONY: install-kubectlinstall-kubectl:     @curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"    @chmod +x kubectl     @mkdir -p ~/.local/bin     @mv ./kubectl /usr/local/bin/kubectl     @chmod +x /usr/local/bin/kubectl.PHONY: install-clusterinstall-cluster:    @kind create cluster --name=${CLUSTERNAME} --config=./${KAFKACLUSTER}.PHONY: create-namespacecreate-namespace:    @kubectl create ns ${NAMESPACE}.PHONY: install-operatorinstall-operator:    @kubectl create -f 'https://strimzi.io/install/latest?namespace=${NAMESPACE}' -n ${NAMESPACE}.PHONY: list-resourcelist-resource:    @kubectl -n kafka get pods,svc.PHONY: wait-zzwait-zz: # sleep 5 second    @echo "waiting...." \    sleep ${SLEEPSEC}.PHONY: delete-kind-clusterdelete-kind-cluster: ; @kind delete cluster --name=${CLUSTERNAME}.PHONY: install-kafka-clusterinstall-kafka-cluster: enable-kraft-feature wait-zz    @kubectl -n ${NAMESPACE} apply -f ${KAFKAEPHEMERALCONFIG}    @sleep ${SLEEPSEC}.PHONY: enable-kraft-featureenable-kraft-feature:     kubectl -n ${NAMESPACE} set env deployment/strimzi-cluster-operator STRIMZI_FEATURE_GATES=+UseKRaft.PHONY: deploy-operatordeploy-operator: install-cluster create-namespace install-operator wait-zz list-resource    @echo "installing kafka".PHONY: kafka-producerkafka-producer:     echo "kafka producer starting up..."    @kubectl -n kafka run kafka-producer -ti --image=quay.io/strimzi/kafka:0.32.0-kafka-3.3.1 --rm=true --restart=Never -- bin/kafka-console-producer.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --topic my-topic.PHONY: kafka-consumerkafka-consumer:     echo "kafka consumer starting to listen..."    @kubectl -n kafka run kafka-consumer -ti --image=quay.io/strimzi/kafka:0.32.0-kafka-3.3.1 --rm=true --restart=Never -- bin/kafka-console-consumer.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --topic my-topic --from-beginninginfo:    @awk '/^[a-zA-Z_-]+: / { print $$0; print "\n"}' $(MAKEFILE_LIST) | \    awk -F":" 'BEGIN {print "targets" } /^[a-zA-Z_-]+/ {print "    "$$1}'

Using Makefile Targets

Executing the command make will list the target names.
Step 1: To deploy the operator use command, make deploy-operator,
Step 2: Once the Strimzi operator status is Running. Execute make install-kafka-cluster command to install the cluster. This will enable the KRaft feature and install the cluster with the configuration.
Note: Make sure the file name matches and all files are in the same directory.

Output

make deploy-operator

make install-kafka-cluster

Created topic details

CODE:- Code can be downloaded from Github.

Additional info on using Zookeeper configuraion

If the KRaft mode is disabled (environment variable UseKraft is turned off), the configuration below can be used to deploy Kafka cluster that uses Zookeeper to manage leader elections.
In that case Makefile requires changes, remove the target that enables Kraft mode.

apiVersion: kafka.strimzi.io/v1beta2kind: Kafkametadata:  name: my-cluster  namespace: kafkaspec:  kafka:    version: 3.2.3    replicas: 1    listeners:      - name: plain        port: 9092        type: internal        tls: false      - name: external        port: 9094        type: nodeport        tls: false        configuration:          bootstrap:            nodePort: 31092          brokers:            - broker: 0              advertisedHost: 127.0.0.1              nodePort: 31234    config:      offsets.topic.replication.factor: 1      transaction.state.log.replication.factor: 1      transaction.state.log.min.isr: 1      default.replication.factor: 1      min.insync.replicas: 1      inter.broker.protocol.version: "3.2"    storage:      type: ephemeral  zookeeper:    replicas: 3    storage:      type: ephemeral  entityOperator:    topicOperator: {}    userOperator: {}

]]>

Build Custom Kustomize Plugin With Containerized KRM function

Thirumurthi S — Sat, 26 Nov 2022 22:54:18 GMT

Develop Kustomize custom plugin with containerized KRM function using Go

In this blog we will see how to develop custom plugin for Kustomize using containers KRM function.
The customized plugin in this blog, will add annotation with specified key and value to set of resources mentioned under the target: tag of the functionCondig:. The annotation information will be defined in a YAML file and referred in the kustomization.yaml configuration using transformers: tag. Some of this information might be not clear at this step.

Pre-requisites:

Basic understanding of kubernetes and deploying resources using YAML manifest
Basic understanding of GO Lang. Installed locally for development. VS Code with Go plugin is easy to configure an use as IDE.
Docker desktop installed
Kustomize CLI installed

To understand what Kustomize is and how to use to mange manifest check my previous blog link here

How Kustomize plugin works?

With containerized KRM function kustomize framework will read the manifest and generate all resource into as Kubernetes resource ResourceList.
- The ResourceList is further utilized by the framework by passing it to Generators and transformer pipeline finally rendering the manifest.
Kustomize provides kyaml framework which is used to parse the ResourceList manifest in Go Language

Work flow

Using kyaml Go framework build the custom plugin with necessary logic.
Create container image, and push to docker hub or private image registry.

Create a transformer yaml file with the configuration required for the plugin logic. The container reference will be provided in this file as annotation.

Below is sample transformers file were the image is referenced.

apiVersion: transformers.example.co/v1 # any name that is uniquekind: ValueAnnotator                   # any name to identify as transformermetadata:  name: notImportantHere  annotations:    config.kubernetes.io/function: |      container:        image: example.docker.com/my-functions/valueannotator:1.0.0  # container image namevalue: 'important-data'

The transformer file should be reference in the kustomization.yaml configuration using transformer: tag
The Kustomize framework will use this file and send it as stdin to the container, the output manifest will be updated with the logic defined in containre code.

Kustomize KRM function

The Kustomize documentation explains an example on adding annotation value to resources. This blog uses the same approach but shows how to debug the code during development with a test case.

Kustomize provides a kyaml framework in GO language, with which we can build the custom plugin.

Plugin development:

For local development we use a sample YAML which defines Kubernetes ResourceList type.
Since the kustomize framework uses the data under the functionConfig: tag of the ResourseList resource as input to the container in stdin, we define the inputs to which set of resources the annotation value needs to be added.
In the below yaml, we have defined the annotation value holder: sample-io/add this will be added to ConfigMap and Service resources only, not to kind: Deployment listed in the items tag

apiVersion: config.kubernetes.io/v1kind: ResourceListfunctionConfig:  annotationList:  - target:       applyAnnotationTo:        - ConfigMap        - Service    key: holder    value: sample-io/additems: - kind: ConfigMap   apiVersion: v1   metadata:      name: webconfig      namespace: demo   data:      key: sample - apiVersion: v1   kind: Service   metadata:      name: websvc      namespace: demo   spec:      selector:        app: web      type: ClusterIP      ports:      - name: svcport        protocol: TCP        port: 80        targetPort: 5000 - apiVersion: apps/v1   kind: Deployment   metadata:      name: fe-deployment      labels:        app: web   spec:      replicas: 1      selector:        matchLabels:          app: web      template:        metadata:          labels:            app: web        spec:          containers:          - name: web            image: nginx:1.14.2            ports:            - containerPort: 80

Go project setup

To initialize Go workspace from scratch, create a folder and issue go mod init command. This will create go.mod and go.sum files.
In below example I created a folder named project and the source code under folder named source.

# from the project directory issued below command# for my project structure> go mod init source > go mod tiny

For developing test case installed testing and yaml package, for which issued below command

# from the project directory> go get gopkg.in/yaml.v3> go get github.com/stretchr/testify/assert> go mod tidy

Code Logic
- The struct type defined in the code represents information under the functionConfig property.
- The kyaml framework parse and injects the YAML data in the function argument that is passed to the Filter function. Our core logic should defined in this function and passed to the processor.
- With the list of items passed by the kyaml framework, with the information under the functionConfig we filter the matching resource and add the annotation with the specified key and value.

Code

Note:-
The code main() function has runAsCommand with true will execute the code block that can generates Dockerfile automatically with the command go run kustomizePlugin.go gen .
Below code can generate the Dockerfile for us
cmd := command.Build(p, command.StandaloneDisabled, false)//automatically generates the Dockerfile for uscommand.AddGenerateDockerfile(cmd)if err := cmd.Execute(); err != nil {     os.Exit(1)}
framework.Execute() takes reader and writer object and the the ResourceList YAML file can be read and passed as input. Make development easy
  if error := framework.Execute(p, byteReadWriter); error != nil {   panic(error) }

Code with the custom plugin logic, save this file as kustomizePlugin.go

package mainimport (    "fmt"    "os"    "sigs.k8s.io/kustomize/kyaml/fn/framework"    "sigs.k8s.io/kustomize/kyaml/fn/framework/command"    "sigs.k8s.io/kustomize/kyaml/kio"    "sigs.k8s.io/kustomize/kyaml/yaml")type AnnotationConfig struct {    Target framework.Selector `yaml:"target,omitempty"`    Key    string             `yaml:"key,omitempty"`    Value  string             `yaml:"value,omitempty"`}type AnnotationConfigs struct {    AnnotationList []AnnotationConfig `yaml:"annotationList,omitempty"`}func process(byteReadWriter *kio.ByteReadWriter, runAsCommand bool) {    config := new(AnnotationConfigs)    //function that will be passed to the kustomize framework execute function    fn := func(items []*yaml.RNode) ([]*yaml.RNode, error) {        // string data type null check        if config.AnnotationList == nil || len(config.AnnotationList) == 0 {            return nil, fmt.Errorf("value is required and can't be empty")        }        for _, annotationConfig := range config.AnnotationList {            filterItemsByResourceType, err := annotationConfig.Target.Filter(items)            if err != nil {                return nil, err            }            // the range returns index, actual object            // PipeE function is from the kyaml framework            for _, filterResource := range filterItemsByResourceType {                err := filterResource.PipeE(yaml.SetAnnotation(annotationConfig.Key, annotationConfig.Value))                if err != nil {                    return nil, err                }            }        }        return items, nil    }    // create a processor using the which will hold the actual logic of filtering the    // resources in the -items of the resourceList and creates annotation to matching     // resource defined in the functionConfig section    p := framework.SimpleProcessor{Config: config, Filter: kio.FilterFunc(fn)}    /*        framework.Execute() is the entrypoint for invoking configuration functions built        with this framework from code. Execute reads a ResourceList        from the given source, passes it to a ResourceListProcessor,        and then writes the result to the target.        STDIN and STDOUT will be used if no reader or writer respectively is provided.    */  if !runAsCommand {    if error := framework.Execute(p, byteReadWriter); error != nil {      panic(error)    }  }        /*      With the below code it would be difficult to debug the code in local (IMO)      We can pass in the input directly, with execute we can pass the reader and writer      So I have used a flag to NOT run this block of code       But the kustomize documentation demonstrates this approach    */    if runAsCommand {        /*         command.Build() returns a cobra.Command to run a function.         The cobra.Command reads the input from STDIN, invokes the provided processor,         and then writes the output to STDOUT.        */        cmd := command.Build(p, command.StandaloneDisabled, false)        //automatically generates the Dockerfile for us        command.AddGenerateDockerfile(cmd)        if err := cmd.Execute(); err != nil {            os.Exit(1)        }    }}func check(e error) {    if e != nil {        fmt.Print(e)        panic(e)    }}func main() {    runAsCommand := false    byteReadWriter := &kio.ByteReadWriter{}    process(byteReadWriter, runAsCommand)}

Simple Test case to validate the output

package mainimport (    "bytes"    "fmt"    "log"    "os"    "strings"    "testing"    "github.com/stretchr/testify/assert"    "gopkg.in/yaml.v3"    "sigs.k8s.io/kustomize/kyaml/kio")/*to install yaml packagego get gopkg.in/yaml.v3go get github.com/stretchr/testify/assertgo mod tidy*/func TestAnnotationPlugin(t *testing.T) {    runAsCommand := false    inputFile, err := os.Open("resources.yaml")    outputFile := bytes.Buffer{}    check(err)    fmt.Print("reading file..")    byteReadWriter := &kio.ByteReadWriter{        Reader: inputFile,        Writer: &outputFile,    }    process(byteReadWriter, runAsCommand)    outputAsStr := strings.TrimSpace(outputFile.String())    fmt.Println(outputAsStr)    readYamlStr := make(map[interface{}]interface{})    err2 := yaml.Unmarshal([]byte(outputAsStr), &readYamlStr)    if err2 != nil {        log.Fatal(err2)    }    substr := "holder: sample-io/add"    configMap := "ConfigMap"    service := "Service"    deployment := "Deployment"    // initialize a map    output := make(map[string]bool)    output[configMap] = false    output[service] = false    output[deployment] = false    for k, v := range readYamlStr {        if k == "items" {            for _, value := range v.([]interface{}) {                manifest, err := yaml.Marshal(&value)                if err != nil {                    log.Fatalf("error: %v", err)                }                //fmt.Printf("---:\n%s\n\n", string(manifest))                if strings.Contains(string(manifest), substr) &&                    (strings.Contains(string(manifest), service) ||                        strings.Contains(string(manifest), configMap)) {                    if strings.Contains(string(manifest), configMap) {                        output[configMap] = true                    }                    if strings.Contains(string(manifest), service) {                        output[service] = true                    }                }            }        }    }    assert.True(t, output[service])    assert.True(t, output[configMap])    assert.False(t, output[deployment])    defer inputFile.Close()}

Run Go program

Use below command to run the Go code, which will also generate the Dockerfile only when the function main() runAsCommand is set to true.
```
> go run kustomizePlugin.go gen .
```
Dockerfile generated and updated to create the image

Additional note :-
Move the Dockerfile generated under source folder to project folder.
The go.mod and go.sum is under the project directory and these files are move to the image during creation process
Update the Docker COPY command to copy only the file source/kustomizePlugin.go

FROM golang:1.19-alpine as builderENV CGO_ENABLED=0WORKDIR /go/src/COPY go.mod go.sum /go/src/RUN go mod download# update the path where the kustmizePlugin.go code is presentCOPY ./source/kustomizePlugin.go .RUN go build -ldflags '-w -s' -v -o /usr/local/bin/function ./FROM alpine:latestCOPY --from=builder /usr/local/bin/function /usr/local/bin/functionENTRYPOINT ["function"]

Docker command to generate the image, in below example it mage is only pushed to local Docker Desktop

> docker build -t kustomize_dev:1.0.0 .

Using the KRM function custom plugin container

Once the image is created successfully, we can add a prod folder under overlay check my previous blog link
- overlay\prod\annotationTransformer.yaml defining the input to wich resource the annotation should be added. In this case to service only

apiVersion: transformers.customplugin.co/v1kind: ResourceAnnotatormetadata:  name: demo-plugin  annotations:    config.kubernetes.io/function: |      container:        image:  kustomize_dev:1.0.0annotationList:  - target:       applyToResources:        - Service    key: holder    value: sample-io/add

overlay\prod\kustomization.yaml reference the transformer file path

apiVersion: kustomize.config.k8s.io/v1beta1kind: Kustomizationresources: - ../../base/nameSuffix: -prod# reference the path of the transformer file that uses the containertransformers:  - annotationTransformer.yaml

Output

To render the manifest file for prod with below command
By default the custom plugins can't be used with kustomize CLI, we need to use --enable-alpha-plugins option

> kustomize build --enable-alpha-plugins prod\

Rendered prod yaml file

apiVersion: v1data:  APP_NAME: demo_app  ENV_CODE: DEVkind: Secretmetadata:  annotations:    holder: sample-io/add  name: demo-app-secret-prodtype: Opaque---apiVersion: apps/v1kind: Deploymentmetadata:  annotations:    holder: sample-io/add  labels:    app: nginx  name: demo-app-prodspec:  replicas: 3  selector:    matchLabels:      app: nginx  template:    metadata:      labels:        app: nginx    spec:      containers:      - image: nginx:latest        name: nginx        ports:        - containerPort: 80

Bonus

Basics of Go Language

Go Lang program should be defined within main package, function main() is the entry point
Go Lang statement doesn't require any delimiter like semicolon used in Java
- By including the brackets at end of the function or block of code compiler will be identifying it as function or block. Refer below code example
The function name first char uppercase indicates it is public function, lowercase is private function.
struct is a used to define custom data type in Go

The code below also includes a testcase which uses YAML Go library

We don't define any data type to unmarshall the generated Kubernetes manifest, we use map of interface and iterate and assert based on the required output.

package main// we need to import the package using below syntax,// fmt package contains utilities to print to stdout like printimport( "fmt")// as mentioned above the function main should end with bracket { // else Go will report errorfunction main(){// alternate use of below code is var output = 0 // use := to initialize variable with valueoutput := 0for i := 1; i <= 10; i++ {    // use = to reassign values    output = output + i}// in order to make any function in package to be public//the first char shoud be upper casefmt.Println("sum :-", output)fmt.Printf("sum:- %d", output)  }

]]>

Manage Kubernetes Manifest with Kustomize

Thirumurthi S — Sat, 26 Nov 2022 08:32:27 GMT

Manage Kubernetes manifest using Kustomize

In this blog have demonstrated, how we can use Kustomize to manage Kubernetes manifest with simple example and also how to render Kubernetes manifest using Kustomize CLI.

Pre-requisites:

Basic understanding of Kubernetes and how resources are deployed using yaml
- Kustomize CLI installed in the machine. To install refer the documentation for more details

What is Kustomize?

Kustomize provides a solution for customizing Kubernetes resource configuration free from templates and DSLs. Refer kustomize.io
- Kustomize lets you customize raw, template-free YAML files for multiple purposes, leaving the original YAML untouched and usable as is.
kubectl client already supports kustomize
Kustomize configuration can applied directly to Kubernetes cluster using kubectl apply -k <directory\to\kustomization.yaml>

High level overview

In enterprise project to deploy the application into Kubernetes cluster the resources will be defined in many of YAML manifest file. If there are lots of manifest file and in case we need to apply different configuration for each environment like dev, test or production we can use helm in order to render and deploy the manifest we need helm command. Kustomize can helps in managing the environment specific configuration in a manageable way.
With Kustomize we not need to template the manifest, the environment specific configuration can be applied to cluster directly using the kubectl command. The manifest can also be rendered using kustomize cli.

With the kustomize structure the base or raw manifest will never change.

How kustomize works?

Say, we have a manifest for our application in a directory named base, kustomize uses kustomization.yaml file that configures the path these manifest under the same directory using resources tag

Info:-The resources tag is used to specify the path of the application manifest yaml this tag is identified by kustomize framework

In order to configure the environment specific configuration overlay folder is created, which contains folders specific to environment. With reference to the image below, we use dev and test folders refers the environment.

Info:-Per Kustomize documentation example have used the folder name as overlay, this folder name can be different.

overlay\dev folder contains kustomization.yaml file which defines the path to the base folder (in resources tag), and dev environment specific changes. For dev the configuration adds "-dev" to the name using nameSuffix and also updates the deployment replicas which is defined inline in the kustomization.yaml
overlay\test folder contains kustomization.yaml file which defines the path to the base folder (in resources tag), and path of the patch file using patches tag. The deployment_patch.yaml use patch transform to update the replicas of the deployment. There are other options to update the configuration of resources, using inline patch configuration where the deployment image tag and the secret properties are updated.
In order to render the environment specific manifest we can use the kustomize CLI command kustomzie build /environment/directory/of/kustomization.yaml or use kubectl apply -k /directory/of/kustomization.yaml.In both case the command takes in the environment specific folder where the kustomization.yaml exists.

The framework will use the kustomize.yaml configuration to identify the base manifest.

Kustomize project folder structure

Dev environment specific representation

Test environment specific representation

With reference to the dev, the base folder is same for the test as well.

Code sample

Below are the YAML file used for this demonstration
base/deployment.yaml

apiVersion: apps/v1kind: Deploymentmetadata:  name: demo-app  labels:    app: nginxspec:  replicas: 3  selector:    matchLabels:      app: nginx  template:    metadata:      labels:        app: nginx    spec:      containers:      - name: nginx        image: nginx:latest        ports:        - containerPort: 80

base/secret.yaml

apiVersion: v1kind: Secretmetadata:  name: demo-app-secrettype: Opaquedata:  APP_NAME: demo_app  ENV_CODE: DEV   # For test we need to update this to TEST

base/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1kind: Kustomizationresources:   - deployment.yaml   - secret.yaml

Dev environment specific configuration

overlay/dev/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1kind: Kustomizationresources: - ../../base/nameSuffix: -dev# example of updating deployment replica from kustomization# Note:- Kustomize supports patch update where the same # changes can be created as a patch, will see this example in # overlay/test directory for demo# when using the replica in this kustomization.yaml, any change # to deployment replica will not be applied, noticed this update# takes precedencereplicas:- name: demo-app  count: 1

Render dev environment manifest

Command to generate dev specific manifest

# navigate to specific directory where the manifest is stored# in this case kustomization foder contains the base and overlay folder# refer the image above where the project structure is deplicted (fig 1)> kustomize build overlay/dev

INFO:-The manifest will be displayed in the stdout, we can redirect it to specific directory using the -o switch, usage kustomize build overlay/dev -o output/

Output of rendered dev manifest

apiVersion: v1data:  APP_NAME: demo_app  ENV_CODE: DEVkind: Secretmetadata:  name: demo-app-secret-devtype: Opaque---apiVersion: apps/v1kind: Deploymentmetadata:  labels:    app: nginx  name: demo-app-devspec:  replicas: 1  selector:    matchLabels:      app: nginx  template:    metadata:      labels:        app: nginx    spec:      containers:      - image: nginx:latest        name: nginx        ports:        - containerPort: 80

Test environment specific configuration

overlay/test/deployment_patch.yaml

- op : replace  path: /spec/replicas  value: 3

INFO:-There are couple ways to specify the patch files
The target can be specified directly in the deployment_patch.yaml file like
target: group: appsversion: v1kind: Deploymentname: demo-app
This patch file can directly define the deployment manifest yaml, with the test specific configuration. The kustomization.yaml file under test, will refer to the pathc using patches tag.
kustomize framework will automatically infer the type of patch to be used when using patches tag, like patchesStrategicMerge or patchTransform

overlay/test/kustomization.yaml
- Below yaml file defines different options to apply the patch, using external file and inline in the configuration yaml

apiVersion: kustomize.config.k8s.io/v1beta1kind: Kustomizationresources: - ../../base/nameSuffix: -testpatches:  - path: deployment_patch.yaml    target:       group: apps      version: v1      kind: Deployment      name: demo-app  - patch: |-     - op: replace       path: /spec/template/spec/containers/0/image       value: nginx:1.21.0    target:      kind: Deployment      name: demo-app  - patch: |-      apiVersion: v1      kind: Secret      metadata:        name: demo-app-secret        type: Opaque      data:        APP_NAME: demo_app        ENV_CODE: TEST ## UPDATE TO TEST IN THIS PATCH

Render test environment manifest

To render test environment specific manifest, the command is kustomzie build overlay/test

Output of rendered test environment manifest

apiVersion: v1data:  APP_NAME: demo_app  ENV_CODE: TESTkind: Secretmetadata:  name: demo-app-secret-test  type: Opaquetype: Opaque---apiVersion: apps/v1kind: Deploymentmetadata:  labels:    app: nginx  name: demo-app-testspec:  replicas: 3  selector:    matchLabels:      app: nginx  template:    metadata:      labels:        app: nginx    spec:      containers:      - image: nginx:1.21.0        name: nginx        ports:        - containerPort: 80

Info:-The rendered YAML file can be directly deployed to the cluster using kubectl command. We can also directly use CD (continuous delivery) tool like Argo CD to use the kustomize configuration to deploy the resources.

Additional Info

Kustomize do supports different options to customize the configuration like

replacements
ConfigMap generator
SecretGenerator
built-in plugin
etc...

Refer the kustomize documentation for details.

]]>

Reactive App built with SpringBoot and Apache Camel to stream data using Apache Kafka

Thirumurthi S — Sat, 12 Nov 2022 07:11:07 GMT

Simple reactive application with SpringBoot, Apache Camel using Kafka Broker to stream the data.

In this blog have demonstrated how to integrate Apache Camel with Apache Kafka and SpringBoot, by building a simple Reactive application.
The Apache Camel components are used to generate random numbers every 2 seconds, this random number is sent to Kafka and consumed by different camel route finally connecting with SpringBoot Flux to stream the data via controller endpoint.

About the application

Apache camel route definition details are as follows
- Route set 1:
  - from: timer component used to poll every 2 seconds in this case
  - processor (this will generate Random number between 0-500 and set in camel exchange)
  - to: direct endpoint (direct is specific to Apache Camel)
- Route set 2:
  - from: direct endpoint
  - to: kafka broker using the Camel kafka component
- Route set 3:
  - This will be
    - from: kafka broker consumes the message
    - to: send to direct endpoint
- Route set 4:
  - from: direct endpoint
  - to: reactive-stream endpoint, named numbers
To integrate the Apache Camel reactive-stream with the SpringBoot Flux, the publisher is retrieved from camel-context and subscribed using Flux.
With Camel reactive-streams component it is easy to use Project Reactor or RxJava or other reactive framework. In this case Spring Flux is used to subscribe to the publisher.

Code flow representation:

Pre-requisites:

Kafka setup installed and running, accessible at http://localhost:9092
Basic understanding of Apache Camel

Code details

Required dependencies for the project

Create springboot project with Apache Camel and WebFlux dependencies, pom.xml details as follows

<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">    <modelVersion>4.0.0</modelVersion>    <parent>        <groupId>org.springframework.boot</groupId>        <artifactId>spring-boot-starter-parent</artifactId>        <version>2.7.5</version>        <relativePath/>    </parent>    <groupId>com.camel.kafka</groupId>    <artifactId>app</artifactId>    <version>0.0.1-SNAPSHOT</version>    <name>app</name>    <description>Demo project for Spring Boot</description>    <properties>        <java.version>17</java.version>    </properties>    <dependencies>        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-starter-webflux</artifactId>        </dependency>        <dependency>            <groupId>org.apache.camel.springboot</groupId>            <artifactId>camel-spring-boot-starter</artifactId>            <version>3.19.0</version>        </dependency>        <dependency>            <groupId>org.apache.camel.springboot</groupId>            <artifactId>camel-kafka-starter</artifactId>            <version>3.19.0</version>        </dependency>        <dependency>            <groupId>io.projectreactor</groupId>            <artifactId>reactor-test</artifactId>            <scope>test</scope>        </dependency>        <dependency>            <groupId>org.apache.camel.springboot</groupId>            <artifactId>camel-reactive-streams-starter</artifactId>            <version>3.19.0</version>        </dependency>        <dependency>            <groupId>org.apache.camel.springboot</groupId>            <artifactId>camel-reactor-starter</artifactId>            <version>3.19.0</version>        </dependency>        <dependency>            <groupId>org.projectlombok</groupId>            <artifactId>lombok</artifactId>            <optional>true</optional>        </dependency>        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-starter-test</artifactId>            <scope>test</scope>        </dependency>    </dependencies>    <build>        <plugins>            <plugin>                <groupId>org.springframework.boot</groupId>                <artifactId>spring-boot-maven-plugin</artifactId>                <configuration>                    <excludes>                        <exclude>                            <groupId>org.projectlombok</groupId>                            <artifactId>lombok</artifactId>                        </exclude>                    </excludes>                </configuration>            </plugin>        </plugins>    </build></project>

Camel Routing configuration details

Below code shows the route configuration mentioned above, we define it by extending the RouteBuilder of Camel.

package com.camel.kafka.app;import org.apache.camel.builder.RouteBuilder;import org.apache.camel.component.kafka.KafkaConstants;import org.springframework.stereotype.Component;@Componentpublic class AppCamelBasedProducerConsumer extends RouteBuilder {    String kafkaProducerURI = "kafka:camel-demo?brokers=localhost:9092";    String kafkaConsumerURI = kafkaProducerURI;    @Override    public void configure() throws Exception {        //route set 1        from("timer://demoapp?fixedRate=true&period=2000")                .process(new RandomGenerationProcess())                .to("direct:message");        //route set 2        from("direct:message")                .setHeader(KafkaConstants.HEADERS, constant("FROM-CAMEL"))                .to(kafkaProducerURI);        //route set 3        from(kafkaConsumerURI + "&groupId=app&autoOffsetReset=earliest&seekTo=BEGINNING")                .log("message - ${body} from ${headers[kafka.TOPIC]}")                .to("direct:outputStream");        //route set 4        from("direct:outputStream")                .to("reactive-streams:numbers");    }}

Camel Processor configuration that generates random number

Below is a implementation of Camel Processor which generates the random number.
In Camel with processor we can transform the messages retrieved from one endpoint to another.
- For example, we can use file component to read the contents of the file from a directory and use processor to transform all into uppercase.

package com.camel.kafka.app;import org.apache.camel.Exchange;import org.apache.camel.Processor;import org.springframework.stereotype.Component;import java.util.Random;@Componentpublic class RandomGenerationProcess implements Processor {    private Random random = new Random();    @Override    public void process(Exchange exchange) throws Exception {        Integer rand = random.nextInt(500);        //Set the random number to the Camel exchange in the body        //This will be sent to the next endpoint        exchange.getIn().setBody(rand);    }}

Info:- We can pass the processor as lambda expression as well like below, so we don't need separate class for implement Processor

   //.....   Random random = new Random();   @Override   public void configure() throws Exception {       from("timer://foo?fixedRate=true&period=2000")       //pass in hte lamda directly       .process(exchange -> exchange.getIn().setBody(random.nextInt(500)))       .to("direct:message");   //....

Below is the service layer where the Camel reactive-streams and the Spring Flux are chained.

package com.camel.kafka.app;import org.apache.camel.CamelContext;import org.apache.camel.ConsumerTemplate;import org.apache.camel.component.reactive.streams.api.CamelReactiveStreams;import org.apache.camel.component.reactive.streams.api.CamelReactiveStreamsService;import org.reactivestreams.Publisher;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Component;import reactor.core.publisher.Flux;@Componentpublic class AppService{    //Fetch the camel context from container    @Autowired    CamelContext camelContext;    //Used to fetch the reactive stream publisher    CamelReactiveStreamsService camel;    public Flux<Integer> randomIntStream(){        camel = CamelReactiveStreams.get(camelContext);        Publisher<Integer> numbers = camel.fromStream("numbers", Integer.class);        return Flux.from(numbers);    }}

Controller code to create an event stream endpoint

Below is a simple Controller, where we define the endpoint as a stream by defining a MediaType, so browsers can access the endpoint as stream of data

package com.camel.kafka.app;import lombok.extern.slf4j.Slf4j;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.http.MediaType;import org.springframework.web.bind.annotation.GetMapping;import org.springframework.web.bind.annotation.RequestMapping;import org.springframework.web.bind.annotation.RestController;import reactor.core.publisher.Flux;@RestController@RequestMapping("/api")@Slf4jpublic class AppController {    @Autowired    AppService appService;    //Including the Media Type TEXT_EVENT_STREAM_VALUE enables browser     //to connect to the endpoint as event stream so data will be streamed continuously    // accessing this endpoint with Chrome the data will be streamed    // at this time when I tried with FireFox it downloads the stream as file    @GetMapping(value="/stream",produces= MediaType.TEXT_EVENT_STREAM_VALUE)    public Flux<Integer> getRandomIntegerStream(){        log.info("invoked controller stream uri /stream");          return appService.randomIntStream();    }}

Output:

Running the above code will throws exception message until an active subscriber is connected, in this case had to hit the http://localhost:8080/api/stream from a browser browser. The console output once connected using browser starts streaming the data to the subscriber.

2022-11-11 22:31:11.779  WARN 17004 --- [mer[camel-demo]] o.a.camel.component.kafka.KafkaConsumer  : Error during processing. Exchange[9D3C45E152C9A66-0000000000000437]. Caused by: [org.apache.camel.component.reactive.streams.ReactiveStreamsNoActiveSubscriptionsException - The stream has no active subscriptions]org.apache.camel.component.reactive.streams.ReactiveStreamsNoActiveSubscriptionsException: The stream has no active subscriptions    at org.apache.camel.component.reactive.streams.engine.CamelPublisher.publish(CamelPublisher.java:111) ~[camel-reactive-streams-3.19.0.jar:3.19.0]    at org.apache.camel.component.reactive.streams.engine.DefaultCamelReactiveStreamsService.sendCamelExchange(DefaultCamelReactiveStreamsService.java:151) ~[camel-reactive-streams-3.19.0.jar:3.19.0]    at org.apache.camel.component.reactive.streams.ReactiveStreamsProducer.process(ReactiveStreamsProducer.java:52) ~[camel-reactive-streams-3.19.0.jar:3.19.0]    at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:172) ~[camel-core-processor-3.19.0.jar:3.19.0]    at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:477) ~[camel-core-processor-3.19.0.jar:3.19.0]    at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:181) ~[camel-base-engine-3.19.0.jar:3.19.0]    at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:59) ~[camel-base-engine-3.19.0.jar:3.19.0]    at org.apache.camel.processor.Pipeline.process(Pipeline.java:175) ~[camel-core-processor-3.19.0.jar:3.19.0]    at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:392) ~[camel-base-engine-3.19.0.jar:3.19.0]    at org.apache.camel.impl.engine.DefaultAsyncProcessorAwaitManager.process(DefaultAsyncProcessorAwaitManager.java:83) ~[camel-base-engine-3.19.0.jar:3.19.0]    at org.apache.camel.support.AsyncProcessorSupport.process(AsyncProcessorSupport.java:41) ~[camel-support-3.19.0.jar:3.19.0]    at org.apache.camel.component.kafka.consumer.support.KafkaRecordProcessor.processExchange(KafkaRecordProcessor.java:109) ~[camel-kafka-3.19.0.jar:3.19.0]    at org.apache.camel.component.kafka.consumer.support.KafkaRecordProcessorFacade.processRecord(KafkaRecordProcessorFacade.java:124) ~[camel-kafka-3.19.0.jar:3.19.0]    at org.apache.camel.component.kafka.consumer.support.KafkaRecordProcessorFacade.processPolledRecords(KafkaRecordProcessorFacade.java:77) ~[camel-kafka-3.19.0.jar:3.19.0]    at org.apache.camel.component.kafka.KafkaFetchRecords.startPolling(KafkaFetchRecords.java:318) ~[camel-kafka-3.19.0.jar:3.19.0]    at org.apache.camel.component.kafka.KafkaFetchRecords.run(KafkaFetchRecords.java:158) ~[camel-kafka-3.19.0.jar:3.19.0]    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[na:na]    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[na:na]    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na]    at java.base/java.lang.Thread.run(Thread.java:833) ~[na:na]2022-11-11 22:31:12.708  INFO 17004 --- [ctor-http-nio-2] com.camel.kafka.app.AppController        : invoked controller stream uri /stream2022-11-11 22:31:13.774  INFO 17004 --- [mer[camel-demo]] route3                                   : message - 442 from camel-demo2022-11-11 22:31:15.783  INFO 17004 --- [mer[camel-demo]] route3                                   : message - 205 from camel-demo2022-11-11 22:31:17.784  INFO 17004 --- [mer[camel-demo]] route3                                   : message - 53 from

From Chrome browser below is the output where the data will be streamed continously

Additional notes

In this pipeline instead of timer component we can use file, jms, etc. Camel component to fetch data and push to kafka.
The Camel Kafka component has additional configuration that can be found in the documentation like enabling transaction, reading from the offset, etc.

]]>

Deploy KIND cluster in Docker Desktop and Access the container from Windows host

Thirumurthi S — Mon, 07 Nov 2022 02:46:55 GMT

Access container running in KIND Kubernetes cluster in Window Docker Desktop

What is Kind?

With Kind (Kubernetes IN Docker) it is easy to spin up a local kubernetes cluster within Docker Desktop. The Kind runs as a container by itself.
Kind documentation is easy to understand, for more details and understanding refer documentation link.

Motivation

KIND cluster deployed in Windows Docker Desktop with default configuration we will not be able to access the container running in the cluster from the host machine.
In order to access the container from the host machine we need to create KIND cluster using extraPortMappings cluster configuration.
This is not the case if we create the KIND cluster in Linux based system, based on the documentation this is network based limitation in Windows and Mac.

Pre-requisite tools installed

1. Docker Desktop installed and running.
1. Install KIND using Chocolatey package manager refer Kind documentation or Chocolatey documentation package manager.
1. Install Kubectl installed using chocolatey package manager.
1. Install GNU make in Windows using Chocolatey

In this blog we will also see how to use the make utility to automate the steps to create the Kind cluster and deploy simple nginx container in the cluster. Once deployed we can use a target to verify whether we are able to access the container.

Instruction to create KIND cluster in Docker Desktop and deployed a Nginx container

1. Create KIND cluster with CLI

KIND CLI used to create cluster with default configuration and specify the configuration in yaml file

# command to create cluster with default config. # the default cluster name is kind> kind create cluster

# --name switch can be used to provide name for cluster > kind create cluster --name=test01

KIND cluster configuration exposes the port

In order to expose the port, we can use extraPortMappings option in the cluster config yaml file. The content of the cluster config, save it as kind_cluster.yaml.

kind: ClusterapiVersion: kind.x-k8s.io/v1alpha4nodes:- role: control-plane  # port forward 8010 on the host to 80 on this node  extraPortMappings:  - containerPort: 80    hostPort: 8010    listenAddress: "127.0.0.1"    protocol: TCP

KIND CLI command to use the config file

> kind create cluster --name=test --config=.\kind_cluster.yaml

List of context created after the KIND cluster is created

INFO:-
After cluster deployed the KIND kube config is automatically merged to the kubectl kube config file, so we can use kubectl command directly to access the context.

Command that lists the contexts after the KIND cluster is created
* indicates the kubectl is using it as current context

$ kubectl config get-contextsCURRENT   NAME              CLUSTER           AUTHINFO          NAMESPACE          docker-desktop    docker-desktop    docker-desktop*         kind-test         kind-test         kind-test          kind-test01       kind-test01       kind-test01

With the above configuration once the cluster is deployed if we check the docker ps we should see the 8010 port exposed.

2. Create a nginx container with the config file

Instead of running nginx container using kubectl run command, we use the deployment manifest file below with ports specified.
- We can see the containerPort and the hostPort in the deployment manifest both match cluster container port value 80.
- The service type ClusterIP expose the container 80 port, so the container can be accessed from the host machine.
- Save the deployment content to the file named app_deployment.yaml

INFO:-
To access the nginx container from the host laptop, the service needs to be created.

apiVersion: apps/v1kind: Deploymentmetadata:  labels:    app: web  name: webspec:  replicas: 1  selector:    matchLabels:      app: web  strategy: {}  template:    metadata:      labels:        app: web    spec:      containers:      - image: nginx        name: nginx        ports:        - containerPort: 80          hostPort: 80        resources: {}status: {}---apiVersion: v1kind: Servicemetadata:  name: web-svcspec:  selector:    app: web  ports:    - protocol: TCP      port: 80      targetPort: 80

Create the deployment with kubectl -n web apply -f app_deployment.yaml.

3. Access the container from host machine

With the Git Bash use the curl command to hit the 8010 port.

curl http://localhost:8010

Or from the browser on host machine which displays the default nginx home page like below

Makefile to deploy the KIND cluster

make is a build tool used in Linux/Unix systems to build C/C++ code earlier
The make file included all the above steps,
- Create the KIND cluster,
- Create a deployment to run a nginx container
- Target to run the curl command to access the container
GNU make also supports lots of features, refer makde documentation.

# Makefile simple representationtarget:<tab-space>recipe

INFO:-
The recipe in the Makefile followed by the target name should start with tab.else it won't make will not be able to execute the Makefile

Below make file content should be created as Makefile

SHELL :=/bin/bashNAMESPACE ?= webCLUSTERNAME ?= testCLUSTERCONFIG := kind_cluster.yamlDEPLOYMENT_MANIFEST := app_deployment.yaml.DEFAULT_GOAL :=info#------------------------------#  Note:-#     - PHONY - usually the targets will be a file in make, in order to indicate it as a task we use PHONY#     - using @ before the commands in the recipe indicate the statement not to be printed in the stdout#       without @, the statement will be printed and command will get executed#     - $$ in the recipe is way to escape the character between the make and the shell command. (refer info target)#.PHONY: create-namespace create-namespace: # create namespace    @kubectl create ns ${NAMESPACE}.PHONY: install-clusterinstall-cluster: # install cluster    @kind create cluster --name=${CLUSTERNAME} --config=./${CLUSTERCONFIG}.PHONY: list-resourcelist-resource: # list resources deployment, pods, service    @kubectl -n ${NAMESPACE} get deploy,pods,svc.PHONY: display-logdisplay-log: # display pod logs (first index is used)    @kubectl -n ${NAMESPACE} logs pod/$(shell kubectl -n ${NAMESPACE} get pods --no-headers -o=jsonpath='{.items[0].metadata.name}')# target within single line alternate approach.PHONY: delete-clusterdelete-cluster: ; @kind delete cluster --name=${CLUSTERNAME}.PHONY: apply-manifestapply-manifest: # apply the manifest    @kubectl -n ${NAMESPACE} apply -f ${DEPLOYMENT_MANIFEST}.PHONY: deploy # creates cluster and deploy appdeploy: install-cluster create-namespace apply-manifest list-resource     @echo "Deploy cluster".PHONY: check-container-accesscheck-container-access: # checks the container after deployment    @curl http://localhost:8010info: # display the target names    @awk '/^[a-zA-Z_-]+: / { print $$0; print "\n"}' $(MAKEFILE_LIST) | \    awk -F":" 'BEGIN {print "targets" } /^[a-zA-Z_-]+/ {print "    "$$1}'

Place the cluster config (as kind_cluster.yaml) and manifest content (as app_deployment.yaml) in same directory as the Makefile.

From Git Bash editor, navigate to the directory and issue the below command

> make deploy

Output - using make targets

The output of using the make deploy, deploy target which deploys the cluster and Nginx container.

Output of make check-container-access once the deployment is complete

Output of make or or make info will list the targets like below

Exposing multiple port from cluster

Cluster configuration to expose multiple port in this case 8012 and 8013
Once exposed we can access the container from host via these ports.

kind: ClusterapiVersion: kind.x-k8s.io/v1alpha4nodes:- role: control-plane  extraPortMappings:  - containerPort: 8000    hostPort: 8012    listenAddress: "127.0.0.1"    protocol: TCP  - containerPort: 8001    hostPort: 8013    listenAddress: "127.0.0.1"    protocol: TCP

Below are the two nginx deployment using different ports
- 8012.yaml
  - The nginx containerPort : 80, the service definition targetPort is 80 which handles the traffic to the web-pod1 container

apiVersion: apps/v1kind: Deploymentmetadata:  labels:    app: web-pod1  name: web-pod1spec:  replicas: 1  selector:    matchLabels:      app: web-pod1  strategy: {}  template:    metadata:      labels:        app: web-pod1    spec:      containers:      - image: nginx        name: nginx        ports:        - containerPort: 80          hostPort: 8000        resources: {}status: {}---apiVersion: v1kind: Servicemetadata:  name: web-pod1spec:  selector:    app: web-pod1  ports:    - protocol: TCP      port: 8000      targetPort: 80

8013.yaml

apiVersion: apps/v1kind: Deploymentmetadata:  labels:    app: web-pod2  name: web-pod2spec:  replicas: 1  selector:    matchLabels:      app: web-pod2  strategy: {}  template:    metadata:      labels:        app: web-pod2    spec:      containers:      - image: nginx        name: nginx        ports:        - containerPort: 80          hostPort: 8001        resources: {}status: {}---apiVersion: v1kind: Servicemetadata:  name: web-pod2spec:  selector:    app: web-pod2  ports:    - protocol: TCP      port: 8001      targetPort: 80

]]>

Integration test Kafka using Awaitality library

Thirumurthi S — Sun, 30 Oct 2022 22:34:06 GMT

Using Awaitality in integration test Kafka Broker

In this blog will briefly explain how to can use Awaitility library to test Kafka in integration test.

Why use Awaitility dependency

Awaitility library can be used for testing external services like Kafka, RabbitMQ, etc. mostly this can be used when application handling asynchronized calls and wait for responses.
Awaitility has different method to support application built in asynchronized way, for more details about Awaitlity refer the documentation
In below have created a simple SpringBoot application with a Producer to send message to Kafka Broker and Consumer to consumes the message. The Listener in the Consumer, will wait for message s to be received from the topic.
The Consumer has to wait till the message is received and this mostly happens in asynchronous application. The application has to wait for a response from external service which might be immediate or delayed after few seconds in such case we can use Awaitility library to perform integration test.
Note, there are cases where we can mock the external service like in this case we use embedded Kafka broker, this might not be always possible since some projects might have dedicated environment. In such scenarios we can use Awaitlity library.
For integration testing in this example have used the embedded Kafka Broker to test the ConsumerService.

Code

Create SpringBoot application with lombok and kafka dependency form start.spring.io or IDE.

Required maven dependencies

Include awaitility dependency in pom.xml

    <dependency>        <groupId>org.awaitility</groupId>        <artifactId>awaitility</artifactId>        <scope>test</scope>    </dependency>

Kafka testing dependency should already be included if not add it to pom.xml

    <dependency>        <groupId>org.springframework.kafka</groupId>        <artifactId>spring-kafka-test</artifactId>        <scope>test</scope>    </dependency>

ProducerService Code that sends message to broker

Simple producer code, where the Kafka broker configuration are defined in application.properties, SpringBoot will use it to create the KafkaTemplate

package com.kafka.example.kafkademo.code;import java.text.MessageFormat;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.kafka.core.KafkaTemplate;import org.springframework.stereotype.Component;import lombok.extern.slf4j.Slf4j;@Component@Slf4jpublic class ProducerService{    @Autowired    KafkaTemplate<String,String> kafkaTemplate;    public String sendMessage(String topic, String message){        log.info("Sending message from producer - {}",message);        kafkaTemplate.send(topic,message);        return MessageFormat.format("Message Sent from Producer - {0}",message);    }}

ConsumerService Code that consumes message from broker

Simple Consumer code which will use the @KafakListener configuration defined in the application.properties.

package com.kafka.example.kafkademo.code;import lombok.Getter;import lombok.Setter;import lombok.extern.slf4j.Slf4j;import org.apache.kafka.clients.consumer.ConsumerRecord;import org.springframework.kafka.annotation.KafkaListener;import org.springframework.stereotype.Component;@Component@Slf4jpublic class ConsumerService {    @Getter    @Setter    private String payload;    @KafkaListener(topics="${test.app.topic}")    public void consumeMessageAppTopic(ConsumerRecord<?,?> consumerRecord){        log.info("payload consuming {}",consumerRecord.toString());        payload = consumerRecord.value().toString();    }}

Kafka broker configuration details

application.properties file with the Kafka broker configuration
- Using 9094 as Kafka port instead of default 9092.

# topic nametest.app.topic=test-topic# Consumer configurationspring.kafka.consumer.bootstrap-servers= localhost:9094spring.kafka.consumer.group-id= consume-test-idspring.kafka.consumer.auto-offset-reset= earliestspring.kafka.consumer.key-deserializer= org.apache.kafka.common.serialization.StringDeserializerspring.kafka.consumer.value-deserializer= org.apache.kafka.common.serialization.StringDeserializer# Producer configurationspring.kafka.producer.bootstrap-servers= localhost:9094spring.kafka.producer.key-serializer= org.apache.kafka.common.serialization.StringSerializerspring.kafka.producer.value-serializer= org.apache.kafka.common.serialization.StringSerializer

Integration test case code using Awaitality library

Below code should be placed in the test folder in the SpringBoot project structure

package com.kafka.example.kafkademo.code;import org.awaitility.Awaitility;import org.junit.jupiter.api.Test;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.beans.factory.annotation.Value;import org.springframework.boot.test.context.SpringBootTest;import org.springframework.kafka.test.context.EmbeddedKafka;import org.springframework.test.annotation.DirtiesContext;import java.time.Duration;import java.util.Objects;import java.util.concurrent.TimeUnit;@SpringBootTest@DirtiesContext/*properties to the EmbeddedKafka brokerpartitions  # of partitions to be used per topic, default 2.brokerProperties  Kafka broker configuration, using plain text listener.*/@EmbeddedKafka(partitions = 1,        brokerProperties = { "listeners=PLAINTEXT://localhost:9094", "port=9094" })class ConsumerServiceTest {    @Autowired    private ConsumerService consumer;    @Autowired    private ProducerService producer;    @Value("${test.app.topic}")    private String topic;    @Test    void testUsingAwaitility_messageConsumed(){        String expectedData = "message data";        producer.sendMessage(topic,expectedData);        /*          Awaitality which will wait for 15 seconds to receive the           message, if message is received before 15 seconds          the test case will be passed and assertion gets validated          until() requires a callable to return the Boolean here,          where the payLoad is obtained        */        Awaitility.await().atMost(Duration.ofSeconds(15))                .until(()->Objects.nonNull(consumer.getPayload()));    }    @Test    void testUsingAwaitility_messageEquality(){        String expectedData = "message data";        producer.sendMessage(topic,expectedData);        /*          The Awaitality until() method callable to validate the           expected message with the consumed message        */        Awaitility.await().atMost(Duration.ofSeconds(15))                .until(()->Objects.equals(expectedData,consumer.getPayload()));    }}

Alternate approach to test Kafka

Integration test case using CountDownLatch

ConsumerService code changes to use CountDownLatch

Alternatively we can use CountDownLatch to wait till the consumer listener recieves the message. Only the ConsumerService code will change in this case, the ProducerService remains the same.
The consumer code looks like below when using CountDownLatch

package com.kafka.example.kafkademo.code;import lombok.Getter;import lombok.Setter;import lombok.extern.slf4j.Slf4j;import org.apache.kafka.clients.consumer.ConsumerRecord;import org.springframework.kafka.annotation.KafkaListener;import org.springframework.stereotype.Component;import java.util.concurrent.CountDownLatch;@Component@Slf4jpublic class ConsumerService {    @Getter    @Setter    private String payload;    @Getter    @Setter    private CountDownLatch latch = new CountDownLatch(1);    @KafkaListener(topics="${test.app.topic}")    public void consumeMessageAppTopic(ConsumerRecord<?,?> consumerRecord){        log.info("payload consuming {}",consumerRecord.toString());        payload = consumerRecord.value().toString();        //once the message is consumed we call the countdown latch to         // decrement the count        latch.countDown();    }    public void resetLatch(){        latch = new CountDownLatch(1);    }}

Test case code using CountDownLatch

The consumer.getLatch().await() in test case will wait the thread till the message is consumed by the ConsumerService class consumerMessageAppTopic() method, since this method invokes countDown() after the message is received from the Listener releasing the thread to proceed further.

package com.kafka.example.kafkademo.code;import org.awaitility.Awaitility;import org.junit.jupiter.api.Test;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.beans.factory.annotation.Value;import org.springframework.boot.test.context.SpringBootTest;import org.springframework.kafka.test.context.EmbeddedKafka;import org.springframework.test.annotation.DirtiesContext;import java.time.Duration;import java.util.Objects;import java.util.concurrent.TimeUnit;@SpringBootTest@DirtiesContext@EmbeddedKafka(partitions = 1,        brokerProperties = { "listeners=PLAINTEXT://localhost:9094", "port=9094" })class ConsumerServiceTest {    @Autowired    private ConsumerService consumer;    @Autowired    private ProducerService producer;    @Value("${test.app.topic}")    private String topic;    @Test    public void testConsumeMessage_countDownLatch() throws Exception {        String expectedData = "message data";        producer.sendMessage(topic,expectedData);        // CountDownLatch await() method will wait for 10 seconds or till the        // ConsumerService consumeMessageAppTopic() invokes countDown() method.        boolean messageConsumed = consumer.getLatch().await(10, TimeUnit.SECONDS);        assertTrue(messageConsumed);                assertTrue(expectedData.equals(consumer.getPayload()));    }}

Output

Running the test cases should succeed

]]>

Validate input JSON payload with SpringBoot AOP

Thirumurthi S — Sun, 23 Oct 2022 16:40:58 GMT

Validate input JSON payload with SpringBoot AOP

In this blog will demonstrate an use case to use SpringBoot AOP to perform input validation in the @Controller or @Service layer level.

Pre-requisites

Basic understanding of AOP concept like Point Cut, ProceedingJointPoint, Adivces - Around, Before , After, etc.

Use case - Validate payload in controller or service layer

When building REST based application at circumstance we might need to perform customized validation on the JSON payload before passing request to the repository layer or further next layer.

Info
Defining JSON schema, is another possible option this might requires more coding effort.
The validation can be performed using the hibernate-validator, with built-in annotation @NotNull, @NotEmpty, etc. defined in the entity class.

Create SpringBoot project using start.spring.io with spring web and lombok dependencies.
Add the Spring starter AOP dependencies in the pom.xml, like below

        <dependency>            <groupId>org.springframework.boot</groupId>            <artifactId>spring-boot-starter-aop</artifactId>        </dependency>

In this blog have included few additional points,
- How to easily create builder pattern on POJO using @Build annotation from Lombok project
- Extending RuntimeException to create custom exception used to handle Input validation in this case
- Custom Spring annotation creation and registering it in AOP @Around advice which will be invoked and used to intercept the method calls
- java.utils.Objects to validate null. Objects contains few other methods which is not detailed here
- Java feature to perform instanceof check and directly store the converted data to variable
```
 // Code where the Object instanceof Customer and the converted data stored in customers if ( requestObject.length >0  && requestObject[0] instanceof Customer customers ){        // first object in the arguments        if (Objects.isNull(customers)){            throw new InvalidInputException("Input invalid - Cannot be null");        }  //.....
```

Info:-
SpringBoot AOP only used to intercept methods. For advanced feature we need to use AspectJ, for example to check if fields have changed.

Code

Create Spring Annotation

Create Spring Annotation, which will be configured in the Aspect @Around advice class.

package com.app.demo.config;import java.lang.annotation.ElementType;import java.lang.annotation.Retention;import java.lang.annotation.RetentionPolicy;import java.lang.annotation.Target;@Target(ElementType.METHOD)@Retention(RetentionPolicy.RUNTIME)public @interface ValidateInputRequest {}

Create Aspect to intercept the method calls

Below code defines the Aspect using @Aspect annotation (note, this aspect needs to be regiestered as a bean using @Component).
The custom annotation @ValidateInputRequest in the previous code snippet is registered in the @Around advice.
The Aspect class validateInput() method will be invoked if the custom annotation @ValidateInputRequest is annotated on the method defined in @Controller or @Service layer.

package com.app.demo.config;import com.app.demo.InvalidInputException;import com.app.demo.dto.Customer;import lombok.extern.slf4j.Slf4j;import org.aspectj.lang.ProceedingJoinPoint;import org.aspectj.lang.annotation.Around;import org.aspectj.lang.annotation.Aspect;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Component;import java.util.Arrays;import java.util.Objects;import java.util.Optional;@Aspect@Component@Slf4jpublic class ValidationAspectConfig {    //Using the Around advice, with the Annotation created and place the annotation     //over the method will invoke this call    @Around("@annotation(ValidateInputRequest)")    public Object validateInput(ProceedingJoinPoint  pjp) throws Throwable {        log.info("AOP intercept for method - "+pjp.getSignature());        Object[] requestObject = pjp.getArgs();        if ( requestObject.length >0  &&requestObject[0] instanceof Customer customers ){            // first object in the arguments            if (Objects.isNull(customers)){                throw new InvalidInputException("Input invalid - Cannot be null");            }            //Performing validation for the input request intercepted and received            ValidateInput.validateRequest(customers);        }else{            log.info("input object is not of customer instance");        }        //just obtain the obj from the ProceedingJoinPoint        Object obj = pjp.proceed();        log.info("Completed executing the AOP intercept");        return obj;    }}

Custom validation logic to validate input JSON payload

Class where we perform the basic validation, to demonstrate basic validation
- Where we check if the input Customer JSON payload received contains name attribute and should NOT be empty.
- Also checking if the address attributes should be available.

package com.app.demo.config;import com.app.demo.InvalidInputException;import com.app.demo.dto.Customer;import com.fasterxml.jackson.core.JsonProcessingException;import com.fasterxml.jackson.databind.ObjectMapper;import org.springframework.stereotype.Component;import java.util.Objects;@Componentpublic class ValidateInput {    public static void validateRequest(Customer customer){       if( Objects.isNull(customer.getName()) || customer.getName().isEmpty()){           throw new InvalidInputException("Invalid input - name cannot be null");       }       if( Objects.isNull(customer.getAddresses())               || (Objects.nonNull(customer.getAddresses()) && customer.getAddresses().isEmpty())){           throw new InvalidInputException("Invalid input - address cannot be null");       }    }}

Customer and Address POJO class

Simple POJO class defining Customer related info

package com.app.demo.dto;import lombok.AllArgsConstructor;import lombok.Builder;import lombok.Data;import lombok.NoArgsConstructor;@Data@NoArgsConstructor@AllArgsConstructor@Builderpublic class Address {    public String addressLine1;    public String addressLine2;    public String aptNo;    public String city;    public String state;    public long zipcode;}

package com.app.demo.dto;import java.util.List;import lombok.AllArgsConstructor;import lombok.Builder;import lombok.Data;import lombok.NoArgsConstructor;@Data@NoArgsConstructor@AllArgsConstructor@Builderpublic class Customer {    public String name;    public long id;    public String department;    public List<Address> addresses;}

Controller class

A simple controller object

package com.app.demo.controller;import java.util.List;import com.app.demo.config.ValidateInputRequest;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.http.ResponseEntity;import org.springframework.web.bind.annotation.*;import com.app.demo.dto.Customer;import com.app.demo.service.CustomerService;@RestController@RequestMapping("/api")public class CustomerController {    @Autowired    private CustomerService customerService;    @GetMapping("/customer")    // the annotation can be used at controller method, commented for now    //@ValidateInputRequest    public ResponseEntity<List<Customer>> getCustomers(){        return ResponseEntity.ok(customerService.getAllCustomers());    }    @PostMapping("/customer")    //@ValidateInputRequest    public ResponseEntity<Customer> addCustomer(@RequestBody Customer customer){        return ResponseEntity.ok(customerService.addCustomer(customer));    }}

Service class

Service layer where we have included the custom annotation @ValidateInputRequest
Have used List of items to load when the application is successfully started

package com.app.demo.service;import com.app.demo.config.ValidateInputRequest;import com.app.demo.dto.Address;import com.app.demo.dto.Customer;import org.springframework.stereotype.Service;import java.util.ArrayList;import java.util.List;@Servicepublic class CustomerService {    static List<Customer> inMemoryDb = new ArrayList<>();    public List<Customer> getAllCustomers() {        return fetchCustomers();    }    //The annotation which will be intercepted by AOP    @ValidateInputRequest    public Customer addCustomer(Customer customer) {        inMemoryDb.add(customer);        return customer;    }    protected List<Customer> fetchCustomers() {        return inMemoryDb;    }    static {        Address address1 = Address.builder()                .addressLine1("work adddress of customer1")                .city("city1").build();        Customer customer1 = Customer.builder()                .name("user1")                .addresses(List.of(address1))                .build();        Address address20 = Address.builder()                .addressLine1("work address customer2")                .city("city1").build();        Address address21 = Address.builder()                .addressLine1("home address customer2")                .city("city1").build();        Customer customer2 = Customer.builder()                .name("user2")                .addresses(List.of(address20, address21))                .build();        inMemoryDb.add(customer1);        inMemoryDb.add(customer2);    }}

Custom Exception

Creating custom Exception extending RuntimeException, used to throw this exception when validating the input JSON payload.

package com.app.demo;public class InvalidInputException extends RuntimeException{    public InvalidInputException(String message){        super(message);    }}

SpringBoot Application entry point

package com.app.demo;import org.springframework.boot.SpringApplication;import org.springframework.boot.autoconfigure.SpringBootApplication;import org.springframework.context.annotation.EnableAspectJAutoProxy;@SpringBootApplicationpublic class DemoApplication {   public static void main(String[] args) {      SpringApplication.run(DemoApplication.class, args);   }}

Project structure snapshot

Output

Curl command to send an JSON payload with empty Customer name

The output with the below response will trigger an exception in output

curl -X POST -H "Content-Type:application/json" http://localhost:8080/api/customer -d '{"name":"","id":0,"department":null,"addresses":[{"addressLine1":"work adddress of customer3","addressLine2":null,"aptNo":null,"city":"city1","state":null,"zipcode":0}]}'

Exception message

{"timestamp":"2022-10-23T04:42:35.931+00:00","status":500,"error":"Internal Server Error","trace":"com.app.demo.demo.InvalidInputException: Invalid input - name cannot be null\r\n\tat com.app.demo.demo.config.ValidateInput.validateRequest(ValidateInput.java:19)com.app.demo.demo.config.ValidationAspectConfig.validateInput(ValidationAspectConfig.java:38)java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)java.base/java.lang.reflect.Method.invoke(Method.java:568)org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)com.app.demo.demo.service.CustomerService$$EnhancerBySpringCGLIB$$e764f31c.addCustomer(<generated>)com.app.demo.demo.controller.CustomerController.addCustomer(CustomerController.java:29)\r\n\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\r\n\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)java.base/java.lang.reflect.Method.invoke(Method.java:568) ........org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1789)org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)\r\n\tat java.base/java.lang.Thread.run(Thread.java:833)\r\n","message":"Invalid input - name cannot be null","path":"/api/customer"}

]]>

Parse part of JSON string using Java

Thirumurthi S — Sun, 09 Oct 2022 00:17:24 GMT

Given a string in JSON format parse only part of the JSON using Java library without mapping to POJO's

When working with JSON objects, there are possible scenarios where we only need a part of the JSON.
Mostly in this scenario, we also don't want to create dedicated POJO's to map the properties.
For example,
- In a case when working with third party API's returning JSON repsonse, and if our requirement is to only use part of that JSON.
- Say, below is the sample JSON from the third-party API, and we need to check only the status property.

{  "app" : {          "name" : "demo-app",          "version" : "1.0",          "description" : "simple app demo"          },   "request" : {        "status" : "ACCEPTED"     },      "details" : [          {"endpoint" : "http://domain.com/api/v1/user"},          {"trackId" : "1234554321" },   ]   }

To fetch the status we can represent the Json path like request.status.
- In JsonPath library, the path can be represented $.request.status
- In Fasterxml library, the path can be represented /request/status

In this blog have demonstrated how to parse part of the JSON using two types java Json parsing library
- 1. Jayway Jsonpath
- 1. Fasterxml Jackson

1. Using `Jayway Jsonpath` library to fetch part of the json value provided the path till the key property

Add dependency in `pom.xml`

The Jayway jsonpath requires slf4j jars

<dependency>      <groupId>com.jayway.jsonpath</groupId>      <artifactId>json-path</artifactId>      <version>2.7.0</version></dependency><dependency>      <groupId>org.slf4j</groupId>      <artifactId>slf4j-api</artifactId>      <version>1.7.36</version></dependency><dependency>      <groupId>org.slf4j</groupId>      <artifactId>slf4j-simple</artifactId>      <version>1.7.30</version></dependency>

Java code using the jsonpath library to fetch part of the json

We specify the path part of the command, JsonPath.compile().
Using the JsonPath.compile().read(), will return the value of the json specified in the path.
Additionally, have used BiFunction<> Lambda functions as example usage for points.

package org.example;import com.jayway.jsonpath.JsonPath;import java.util.List;import java.util.Map;import java.util.function.BiFunction;public class JaywayJsonPathBasedJSONParser {    public static void main(String ... args){        String inputJson = getJsonString();        // define the path the $ - represents the root        // path is represented with `.`        JsonPath jsonPath = JsonPath.compile("$.store.bicycle.color");        // parse json string for the path        String bicycleColor = jsonPath.read(inputJson);        // print the parsed string        System.out.println(bicycleColor);        // additionally below is a representation just parsing hte         // in this case the path returns List<> which is json Array         // the list of book will be returned as List of map        String bookJsonPathExpression = "$.store.book";        List<Map<String,Object>> result = fetchMapUsingPath.apply(inputJson,bookJsonPathExpression);        result.forEach( (item)->{            item.forEach((key,value)->{System.out.println(key+" : "+value);});            System.out.println("--------------------------");        });    }    //Below is a Function definition to convert     static BiFunction<String, String , List<Map<String,Object>>> fetchMapUsingPath = (jsonString, pathExpression)->{        JsonPath jsonPath = JsonPath.compile(pathExpression);        return jsonPath.read(jsonString);    } ;    /**     * The method to return the simple json as string     * @return     */    public static String getJsonString(){        String inputJson = """                {                    "store": {                        "book": [                            {                                "category": "sci-fi",                                "author": "Douglas Adams",                                "title": "The long dark tea-time of the soul",                                "price": 14.25                            },                            {                                "category": "fiction",                                "author": "Stella Gibbons",                                "title": "Cold comfort farm",                                "isbn" : "978-1476783000",                                "price": 13.79                            }                        ],                        "bicycle": {                            "color": "blue",                            "price": 39.99                        }                    },                    "expensive": 10                }                """;        return inputJson;    }}

Output:

bluecategory : sci-fiauthor : Douglas Adamstitle : The long dark tea-time of the soulprice : 14.25-----------------category : fictionauthor : Stella Gibbonstitle : Cold comfort farmisbn : 978-1476783000price : 13.79-----------------

2. Using `Fasterxml Jackson` library to parse part of the json using the provided json key as path.

Add below dependency in `pom.xml` to load the fasterxml jackson library

<dependency>      <groupId>com.fasterxml.jackson.core</groupId>      <artifactId>jackson-core</artifactId>      <version>2.13.4</version></dependency><dependency>      <groupId>com.fasterxml.jackson.core</groupId>      <artifactId>jackson-databind</artifactId>      <version>2.13.4</version></dependency><dependency>      <groupId>com.fasterxml.jackson.core</groupId>      <artifactId>jackson-annotations</artifactId>      <version>2.13.4</version></dependency>

Java code uses the jackson library to fetch the part of JSON

Using ObjectMapper read the complete json as Jsonnode
Defining a JsonPointer with path to the JSON specified properties
With the returned JsonNode, use the at(<pass the JsonPointer) method to read the value

package org.example;import com.fasterxml.jackson.core.JsonPointer;import com.fasterxml.jackson.core.JsonProcessingException;import com.fasterxml.jackson.core.type.TypeReference;import com.fasterxml.jackson.databind.JsonNode;import com.fasterxml.jackson.databind.ObjectMapper;import java.util.List;import java.util.Map;public class JacksonBasedJSONParser {    public static void main(String ... args) {        // json input string        String inputJson = getJsonString();        // jackson ObjectMapper object        ObjectMapper objectMapper = new ObjectMapper();        //path represented using / to fetch the color        JsonPointer bicycleColorPointer = JsonPointer.compile("/store/bicycle/color");        //path represented to fetch book list        JsonPointer bookJsonPathPointer = JsonPointer.compile("/store/book");        try {            JsonNode completeJsonNode = objectMapper.readValue(inputJson,JsonNode.class);            String bicycleColor = completeJsonNode.at(bicycleColorPointer).toString();            //print bicycelcolor             System.out.println(bicycleColor);            List<Map<String,Object>> bookList = objectMapper.readValue(completeJsonNode.at(bookJsonPathPointer).toString(),new TypeReference<>(){});            bookList.forEach( (item)->{                item.forEach((key,value)->{                    System.out.println(key+" : "+value);                });                System.out.println("------------------------");            });        } catch (JsonProcessingException e) {            throw new RuntimeException(e);        }    }    /**     * This method returns the json string sample     * @return     */    public static String getJsonString(){        String inputJson = """                {                    "store": {                        "book": [                            {                                "category": "sci-fi",                                "author": "Douglas Adams",                                "title": "The long dark tea-time of the soul",                                "price": 14.25                            },                            {                                "category": "fiction",                                "author": "Stella Gibbons",                                "title": "Cold comfort farm",                                "isbn" : "978-1476783000",                                "price": 13.79                            }                        ],                        "bicycle": {                            "color": "blue",                            "price": 39.99                        }                    },                    "expensive": 10                }                """;        return inputJson;    }}

Output:

"blue"category : sci-fiauthor : Douglas Adamstitle : The long dark tea-time of the soulprice : 14.25------------------------category : fictionauthor : Stella Gibbonstitle : Cold comfort farmisbn : 978-1476783000price : 13.79------------------------

]]>