Java Spring – Microservices - Roman Kryvolapov

Open all questions about Spring

➤ How microservices can interact
➤ How to use Docker in Spring
➤ How to Use Docker Compose in Spring
➤ How to use Kubernetes in Spring
➤ How to use Elasticsearch in Spring
➤ How to use RabbitMQ in Spring
➤ How to use Kafka in Spring
➤ What is Eureka Service
➤ How to implement Spring Cloud Gateway
➤ How to implement redirection using Spring Cloud Gateway
➤ What is Service Discovery
➤ What is Dockerfile and Docker Compose
➤ What tools are there for monitoring microservices
➤ What is Trace ID and Span ID

➤ How microservices can interact

In a microservice architecture, microservices can interact with each other in several ways, such as synchronous and asynchronous calls. Let's look at the main approaches to microservice interaction:

Synchronous calls (HTTP/REST):
This is the most common way for microservices to interact. One microservice makes an HTTP request to another microservice. In Spring Boot, this can be done using RestTemplate or WebClient.

Example of using RestTemplate:

import org.springframework.stereotype.Service
import org.springframework.web.client.RestTemplate


@Service
class UserService {
  
    private val restTemplate = RestTemplate()
    
    fun getUserDetails(userId: Long): User {
        val url = "http://order-service/orders/user/$userId"
        val response = restTemplate.getForObject(url, User::class.java)
        return response ?: throw RuntimeException("User not found")
    }
    
}

Example of using WebClient:

import org.springframework.stereotype.Service
import org.springframework.web.reactive.function.client.WebClient


@Service
class UserService {
  
    private val webClient = WebClient.create("http://order-service")
    
    fun getUserDetails(userId: Long): User {
        return webClient.get()
            .uri("/orders/user/$userId")
            .retrieve()
            .bodyToMono(User::class.java)
            .block() ?: throw RuntimeException("User not found")
    }
    
}

Asynchronous calls (messages):
Microservices can communicate asynchronously by exchanging messages through message brokers such as RabbitMQ, Apache Kafka, and others.

Example of using RabbitMQ:

import org.springframework.amqp.core.Queue
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration


@Configuration
class RabbitMQConfig {
  
    @Bean
    fun queue(): Queue {
        return Queue("example.queue", false)
    }
    
}

import org.springframework.amqp.rabbit.core.RabbitTemplate
import org.springframework.stereotype.Component


@Component
class MessageSender(private val rabbitTemplate: RabbitTemplate) {
  
    fun sendMessage(message: String) {
        rabbitTemplate.convertAndSend("example.queue", message)
    }
    
}

import org.springframework.amqp.rabbit.annotation.RabbitListener
import org.springframework.stereotype.Component


@Component
class MessageReceiver {
  
    @RabbitListener(queues = ["example.queue"])
    fun receiveMessage(message: String) {
        println("Received message: $message")
    }
    
}

Interaction via database:
Microservices can interact via a common database, but this is not recommended, as it violates the principle of data isolation in the microservice architecture. It is better to use the database only for storing data, and interact via API.

Using discovery and routing services:
Eureka (service discovery)

Example Eureka configuration:

import org.springframework.beans.factory.annotation.Autowired
import org.springframework.cloud.client.discovery.DiscoveryClient
import org.springframework.stereotype.Service
import org.springframework.web.client.RestTemplate


@Service
class UserService {
  
    @Autowired
    private lateinit var discoveryClient: DiscoveryClient
  
    private val restTemplate = RestTemplate()
    
    fun getUserDetails(userId: Long): User {
        val instances = discoveryClient.getInstances("order-service")
        val orderServiceUri = instances[0].uri
        val url = "$orderServiceUri/orders/user/$userId"
        val response = restTemplate.getForObject(url, User::class.java)
        return response ?: throw RuntimeException("User not found")
    }
    
}

Spring Cloud Gateway (routing):
used to route requests to the appropriate microservices.
Example Gateway configuration:

Using gRPC:
gRPC is a modern RPC (Remote Procedure Call) framework that uses Protocol Buffers and supports asynchronous calls.

Example of using gRPC (Protocol (user.proto)):

syntax = "proto3";
option java_package = "com.example.demo";
option java_multiple_files = true;
service UserService {
    rpc GetUserDetails (UserRequest) returns (UserResponse);
}
message UserRequest {
    int64 userId = 1;
}
message UserResponse {
    int64 id = 1;
    string username = 2;
    string password = 3;
    string role = 4;
}

import io.grpc.stub.StreamObserver
import net.devh.boot.grpc.server.service.GrpcService


@GrpcService
class UserServiceImpl : UserServiceGrpc.UserServiceImplBase() {
  
    override fun getUserDetails(request: UserRequest, responseObserver: StreamObserver<UserResponse>) {
        val user = UserResponse.newBuilder()
            .setId(request.userId)
            .setUsername("testuser")
            .setPassword("password")
            .setRole("ROLE_USER")
            .build()
        responseObserver.onNext(user)
        responseObserver.onCompleted()
    }
    
}

import net.devh.boot.grpc.client.inject.GrpcClient
import org.springframework.stereotype.Service


@Service
class UserServiceClient {
  
    @GrpcClient("user-service")
    private lateinit var userServiceStub: UserServiceGrpc.UserServiceBlockingStub
  
    fun getUserDetails(userId: Long): UserResponse {
        val request = UserRequest.newBuilder().setUserId(userId).build()
        return userServiceStub.getUserDetails(request)
    }
    
}

➤ What is Dockerfile and Docker Compose

Dockerfile:
is used to create customized Docker images. It is the primary way of describing how to build a container image with the required dependencies and configurations.

# Using a base image
FROM openjdk:11-jre-slim
# Setting the working directory
WORKDIR /app
# Copy jar file to container
COPY target/myapp.jar /app/myapp.jar
# Specifying the command to launch the application
CMD ["java", "-jar", "myapp.jar"]

Docker Compose:
is used to manage multi-container applications. It is an orchestration tool that allows you to run and manage multiple containers as a single entity, defining their interactions and dependencies.

version: '3.8'
services:
  web:
    build: .
    ports:
      - "8080:8080"
    depends_on:
      - db
  db:
    image: postgres:13
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
      POSTGRES_DB: mydatabase
    volumes:
      - db-data:/var/lib/postgresql/data
volumes:
  db-data:

Both tools are often used together:
First, a Dockerfile is created for each service, then a Docker Compose file is created to manage all of these services in one application.

➤ How to use Docker in Spring

Docker is a platform for developing, delivering, and running applications in containers. Containers allow you to package an application with its dependencies and ensure its isolation and portability.

Key concepts of Docker:

Containers:
Isolated environments in which applications run.

Images:
Templates used to create containers.

Dockerfile:
A script describing how to create a Docker image.

Docker Hub:
A cloud registry where you can store and share Docker images.

Basic Docker commands:

docker build:
Create a Docker image from a Dockerfile.

docker run:
Launching a container from a Docker image.

docker pull:
Loading a Docker image from a registry.

docker push:
Uploading a Docker image to the registry.

docker ps:
List of running containers.

docker stop/start:
Stop/start container.

# Step 1: Building the application
FROM openjdk:21-jdk-slim AS build
WORKDIR /app
# Copy the Gradle build files and source code
COPY build.gradle settings.gradle gradlew gradlew.bat ./
COPY gradle gradle
COPY src src
# Set permissions to execute the gradlew file
RUN chmod +x ./gradlew
# Download dependencies and build the project
RUN ./gradlew bootJar
# Step 2: Create a minimal image to run the application
FROM openjdk:21-jdk-slim
WORKDIR /app
# Copy the assembled jar file from the build stage
COPY --from=build /app/build/libs/*.jar /app/app.jar
# Specify the command to launch the Spring Boot application
ENTRYPOINT ["java", "-jar", "/app/app.jar"]

Assembling the project:
Build the project using Maven or Gradle to get an executable jar file in the target folder.

./gradlew build

Building a Docker image:
Use the docker build command to create a Docker image from a Dockerfile.

docker build -t your-dockerhub-username/demo .

Launching a container:
Run the container using the docker run command.

docker run -p 8080:8080 your-dockerhub-username/demo

Your application is now available at http://localhost:8080.

Uploading the image to Docker Hub:
Log in to Docker Hub and upload your image.

docker login
docker push your-dockerhub-username/demo

Useful Docker commands:

docker images:
List of all images.

docker ps -a:
List of all containers.

docker stop:
Container stop.

docker start:
Starting a stopped container.

docker rm:
Removing the container.

docker rmi:
Deleting an image.

➤ How to Use Docker Compose in Spring

Docker Compose is a tool for defining and managing multi-container Docker applications. With Docker Compose, you can describe services, networks, and volumes in a single YAML file, then easily deploy them with a single command.

Example of using Docker Compose:
Let's say we have a Spring Boot application that uses a PostgreSQL database. We want to deploy these two services together using Docker Compose.

spring.datasource.url=jdbc:postgresql://db:5432/mydatabase
spring.datasource.username=postgres
spring.datasource.password=postgres
spring.jpa.hibernate.ddl-auto=update

Creating a Dockerfile:

# Use the official image as a parent image
FROM openjdk:11-jre-slim
# Set the working directory in the container
WORKDIR /app
# Copy the jar file to the container
COPY target/demo-0.0.1-SNAPSHOT.jar app.jar
# Run the jar file
ENTRYPOINT ["java", "-jar", "app.jar"]

Create a docker-compose.yml file in the project root directory:

version: '3.8' # Specifies the version of the Docker Compose file format
services: # Defines a list of services to be deployed
  app: # Service definition for Spring Boot application
    image: your-dockerhub-username/demo # Docker image name
    build: # Define the Docker image build process
      context: . # Build context, current directory
      dockerfile: Dockerfile # The name of the Dockerfile that will be used for the build
    ports:
      - "8080:8080" # Forwarding port 8080 of the host to port 8080 of the container
    environment: # Environment variables for application configuration
      SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/mydatabase # Database connection URL
      SPRING_DATASOURCE_USERNAME: postgres # Username for the database
      SPRING_DATASOURCE_PASSWORD: postgres # Database password
    depends_on: # Defines dependencies on other services
      - db # Depends on the db service
  db: # Service definition for PostgreSQL database
    image: postgres:13 # PostgreSQL version 13 Docker image name
    environment: # Environment variables for PostgreSQL configuration
      POSTGRES_DB: mydatabase # Name of the database to be created
      POSTGRES_USER: postgres # Database user name
      POSTGRES_PASSWORD: postgres # Password for the database user
    ports:
      - "5432:5432" # Forwarding port 5432 of the host to port 5432 of the container
    volumes:
      - postgres_data:/var/lib/postgresql/data # Create a volume to store PostgreSQL data
volumes: # Defines a list of volumes that will be used by services
  postgres_data: # Volume to store PostgreSQL data so that it persists across container restarts

Explanation for each section:

version:
Specifies the version of the Docker Compose file format. In this case, version 3.8 is used.

services:
Defines all services to be deployed.

app:
Service for Spring Boot application.

image:
The name of the Docker image that will be used to run the container.

build:
Defines the Docker image build parameters.

context:
The directory where the Dockerfile is located.

dockerfile:
The name of the Dockerfile.

ports:
Forwarding port 8080 of the host to port 8080 of the container.

environment:
Environment variables for configuring the database connection.

depends_on:
Specifies that this service depends on the db service.

db:
Service for PostgreSQL database.

image:
The name of the PostgreSQL version 13 Docker image.

environment:
Environment variables for database configuration.

ports:
Forwarding port 5432 of the host to port 5432 of the container.

volumes:
Create a volume to store PostgreSQL data.

volumes:
Specifies the volume to be used to store PostgreSQL data.

How does this work:

Assembly and launch:
When you run the docker-compose up –build command, Docker Compose first builds a Docker image for the app service using the specified Dockerfile. Then both services (app and db) are started, with the app service depending on the db service, so db is started first.

Environment variables:
Environment variables are used to configure database connection settings in a Spring Boot application.

Port forwarding:
Ports are forwarded from containers to the host machine so that the application and database can be accessed.

Data volume:
A postgres_data volume is created to store PostgreSQL database data so that the data is persistent across container restarts.

This docker-compose.yml file makes it easy to deploy a multi-container application using Docker Compose, providing consistency and ease of container management.

Assembly and launch:
Run docker-compose to build and start all services.

docker-compose up --build

This command:
Build a Docker image for your Spring Boot application.
Launch containers for the PostgreSQL application and database.
Set up communication between containers.

➤ How to use Kubernetes in Spring

Kubernetes is an open-source container orchestration system that enables automated deployment, scaling, and management of containerized applications. It provides a platform for running, managing, and scaling containers.

Here's a step-by-step guide on how to use Kubernetes to deploy a Spring Boot application.

Prerequisites:

Docker:
Make sure Docker is installed and configured.

Kubernetes:
Install and configure Kubernetes (e.g. using Minikube for local development).

kubectl:
Install the kubectl command-line utility to interact with the Kubernetes cluster.

Creating a Docker image:

# Use the official image as a parent image
FROM openjdk:11-jre-slim
# Set the working directory in the container
WORKDIR /app
# Copy the jar file to the container
COPY target/demo-0.0.1-SNAPSHOT.jar app.jar
# Run the jar file
ENTRYPOINT ["java", "-jar", "app.jar"]

Build the Docker image and upload it to Docker Hub (or any other Docker image registry).

# Building a Docker image
docker build -t your-dockerhub-username/demo .
# Login to Docker Hub
docker login
# Uploading a Docker image to Docker Hub
docker push your-dockerhub-username/demo

Creating Kubernetes Manifests:
Create Kubernetes manifests for your Deployment, Service, and Ingress configuration.

deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-deployment
  labels:
    app: demo
spec:
  replicas: 3
  selector:
    matchLabels:
      app: demo
  template:
    metadata:
      labels:
        app: demo
    spec:
      containers:
      - name: demo
        image: your-dockerhub-username/demo:latest
        ports:
        - containerPort: 8080

service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: demo-service
spec:
  selector:
    app: demo
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080
  type: LoadBalancer

ingress.yaml:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: demo.local
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: demo-service
            port:
              number: 80

Using Kubernetes Manifests:
Use the kubectl command to apply manifests to your Kubernetes cluster.

kubectl apply -f deployment.yaml
kubectl apply -f service.yaml
kubectl apply -f ingress.yaml

DNS setup:
For local development with Minikube, add the following entry to the /etc/hosts file:

<MINIKUBE_IP> demo.local

Get Minikube IP using the command:

minikube ip

Checking the deployment:
Check the status of your deployment, service, and ingress using the commands:

kubectl get deployments
kubectl get services
kubectl get ingress

Open your browser and navigate to http://demo.local to see your deployed Spring Boot application.

➤ How to use Apache Kafka in Spring

Apache Kafka is a distributed streaming platform used to build real-time data processing systems. Kafka was originally developed at LinkedIn and opened as an open source project in 2011. It is used to publish, store, and process real-time streams of records.

The main components of Kafka are:

Producer:
Sends records (messages) to Kafka topics.

Consumer:
Reads records (messages) from Kafka topics.

Broker:
A Kafka server that accepts data from producers, stores it, and distributes it to consumers. A Kafka cluster consists of one or more brokers.

Topic:
A logical category or channel where producers send data and consumers read it from. A topic is partitioned to provide parallelism and scalability.

Partition:
A subdivision of a topic. Each partition is an ordered and immutable log where producers add messages.

Zookeeper:
The coordination system used by Kafka to manage cluster metadata and track the status of brokers and topics.

How Kafka works:

Posting messages:
Productors publish messages to specific topics. The messages are stored in the topic partitions as logs.

Storing messages:
Messages are stored in topic partitions on disk and can be configured to be stored for a specified time or until a specified data volume is reached.

Reading messages:
Consumers subscribe to topics and read messages from partitions. Kafka allows consumers to control where they start reading messages, which provides great flexibility and the ability to re-read data.

Scalability:
Partitions allow you to distribute the load across multiple brokers, providing horizontal scaling.

Folltolerance:
Replicating partitions across multiple brokers provides high availability and fault tolerance.

Setting up Kafka:
Install and configure Apache Kafka and Zookeeper using Docker Compose.
Create a docker-compose.yml file

version: '3.8'
services:
  zookeeper:
    image: bitnami/zookeeper:latest
    container_name: zookeeper
    ports:
      - "2181:2181"
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
  kafka:
    image: bitnami/kafka:latest
    container_name: kafka
    ports:
      - "9092:9092"
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092
      - ALLOW_PLAINTEXT_LISTENER=yes

Go to the file directory in the command line and run:

docker-compose up -d

Check the status

docker-compose ps

Creating an event producer:
Create a Spring Boot application with Kafka dependencies.

dependencies {
    implementation("org.springframework.boot:spring-boot-starter")
    implementation("org.springframework.kafka:spring-kafka")
}

spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=my-group
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer

or via configuration class

import org.apache.kafka.clients.producer.ProducerConfig
import org.apache.kafka.common.serialization.StringSerializer
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import org.springframework.kafka.core.DefaultKafkaProducerFactory
import org.springframework.kafka.core.KafkaTemplate
import org.springframework.kafka.core.ProducerFactory
import org.springframework.kafka.support.serializer.JsonSerializer


@Configuration
class KafkaProducerConfig {
  
    @Bean
    fun producerFactory(): ProducerFactory<String, String> {
        val configProps = HashMap<String, Any>()
        configProps[ProducerConfig.BOOTSTRAP_SERVERS_CONFIG] = "localhost:9092"
        configProps[ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG] = StringSerializer::class.java
        configProps[ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG] = JsonSerializer::class.java
        return DefaultKafkaProducerFactory(configProps)
    }
    
    @Bean
    fun kafkaTemplate(): KafkaTemplate<String, String> {
        return KafkaTemplate(producerFactory())
    }
    
}

import org.apache.kafka.clients.consumer.ConsumerConfig
import org.apache.kafka.common.serialization.StringDeserializer
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import org.springframework.kafka.annotation.EnableKafka
import org.springframework.kafka.config.ConcurrentKafkaListenerContainerFactory
import org.springframework.kafka.core.ConsumerFactory
import org.springframework.kafka.core.DefaultKafkaConsumerFactory
import org.springframework.kafka.support.serializer.ErrorHandlingDeserializer
import org.springframework.kafka.support.serializer.JsonDeserializer


@EnableKafka
@Configuration
class KafkaConsumerConfig {
  
    @Bean
    fun consumerFactory(): ConsumerFactory<String, String> {
        val configProps = HashMap<String, Any>()
        configProps[ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG] = "localhost:9092"
        configProps[ConsumerConfig.GROUP_ID_CONFIG] = "group_id"
        configProps[ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG] = ErrorHandlingDeserializer::class.java
        configProps[ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG] = ErrorHandlingDeserializer::class.java
        configProps[ErrorHandlingDeserializer.VALUE_DESERIALIZER_CLASS] = JsonDeserializer::class.java.name
        return DefaultKafkaConsumerFactory(configProps, StringDeserializer(), JsonDeserializer())
    }
    
    @Bean
    fun kafkaListenerContainerFactory(): ConcurrentKafkaListenerContainerFactory<String, String> {
        val factory = ConcurrentKafkaListenerContainerFactory<String, String>()
        factory.consumerFactory = consumerFactory()
        return factory
    }
    
}

Producer and Consumer:

import org.springframework.kafka.core.KafkaTemplate
import org.springframework.stereotype.Service


@Service
class EventProducer(private val kafkaTemplate: KafkaTemplate<String, String>) {
  
    fun sendMessage(topic: String, message: String) {
        kafkaTemplate.send(topic, message)
    }
    
}

import org.springframework.kafka.annotation.KafkaListener
import org.springframework.stereotype.Service


@Service
class EventConsumer {
  
    @KafkaListener(topics = ["topic_name"], groupId = "group_id")
    fun consume(message: String) {
        println("Consumed message: $message")
    }
    
}

➤ How to use RabbitMQ in Spring

RabbitMQ is a message broker software that allows applications to exchange messages and perform tasks asynchronously. It supports multiple messaging protocols and is widely used to build distributed and scalable systems.

Basic concepts of RabbitMQ:

AMQP Protocol:
RabbitMQ supports the AMQP (Advanced Message Queuing Protocol), which defines the rules for exchanging messages between clients and brokers.

Queues:
A queue is a buffer for storing messages. Messages are sent to the queue where they wait to be processed by the recipient.

Exchangers:
The exchanger receives messages from the producer and routes them to one or more queues depending on the established rules.

Bindings:
A binding connects a queue to an exchanger and defines the rules for routing messages from the exchanger to the queue.

Messages:
Messages are data that are passed between applications through queues. Each message consists of a header and a body (payload).

Producers:
Manufacturers send messages to exchangers.

Consumers:
Consumers receive messages from queues and process them.

Example of RabbitMQ in operation:
The Producer sends a message to the Exchange.
The exchanger routes the message to the appropriate queue (Queue) based on the established rules.
The Consumer receives a message from the queue and processes it.

To use RabbitMQ in Kotlin using Spring Boot, you can use the Spring AMQP library (Spring for RabbitMQ)

dependencies {
    implementation 'org.springframework.boot:spring-boot-starter-amqp'
    implementation 'org.jetbrains.kotlin:kotlin-reflect'
    implementation 'org.jetbrains.kotlin:kotlin-stdlib-jdk8'
}

import org.springframework.amqp.core.Queue
import org.springframework.amqp.rabbit.connection.ConnectionFactory
import org.springframework.amqp.rabbit.core.RabbitTemplate
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration


@Configuration
class RabbitConfig {
  
    @Bean
    fun queue(): Queue {
        return Queue("myQueue", false)
    }
    
    @Bean
    fun rabbitTemplate(connectionFactory: ConnectionFactory): RabbitTemplate {
        return RabbitTemplate(connectionFactory)
    }
    
}

import org.springframework.amqp.rabbit.core.RabbitTemplate
import org.springframework.stereotype.Service

  
@Service
class MessageSender(private val rabbitTemplate: RabbitTemplate) {
  
    fun sendMessage(message: String) {
        rabbitTemplate.convertAndSend("myQueue", message)
    }
    
}

import org.springframework.amqp.rabbit.annotation.RabbitListener
import org.springframework.stereotype.Service


@Service
class MessageListener {
  
    @RabbitListener(queues = ["myQueue"])
    fun receiveMessage(message: String) {
        println("Received message: $message")
    }
    
}

import org.springframework.web.bind.annotation.GetMapping
import org.springframework.web.bind.annotation.RequestParam
import org.springframework.web.bind.annotation.RestController


@RestController
class MessageController(private val messageSender: MessageSender) {
  
    // http://localhost:8080/send?message=Hello
    @GetMapping("/send")
    fun send(@RequestParam message: String) {
        messageSender.sendMessage(message)
        return "Message sent: $message"
    }
    
}

➤ How Kafka differs from RabbitMQ

Kafka and RabbitMQ are two popular messaging systems used to transfer messages between components of distributed systems. However, they have different architectural approaches, target use cases, and features.

Apache Kafka:

Architecture:

Log-based pub/sub:
Kafka stores messages as logs, allowing subscribers to read messages from a specific offset.

Message broker:
Kafka consists of clusters of brokers that store data in a distributed manner.

Storing messages:
Messages are stored on disk, and each message has a unique offset. This allows messages to be re-read and the log to be rewinded.
Support for long-term data storage.

Performance:
High performance and throughput suitable for processing large amounts of data in real time.

Scaling support:
Easily scales horizontally by adding new brokers and partitions.

Target use cases:
Stream processing.
Implementation of event systems.
Real-time data analytics.

Ecosystem:
Includes components such as Kafka Streams for stream processing and Kafka Connect for integration with various data sources.

RabbitMQ:

Architecture:

Message queues:
RabbitMQ uses the concept of message queues and routing.

Message broker:
Messages are transmitted through exchanges and queues.

Storing messages:
Messages can be stored in RAM or on disk.
Supports reliable message delivery through acknowledgements and retry.

Performance:
Good performance for a wide range of scenarios, but may be less efficient when processing large amounts of data compared to Kafka.

Scaling support:
Supports clustering and federation for horizontal scaling, but can be more complex to set up and manage than Kafka.

Target use cases:
Classic task queue for asynchronous processing.
Implementation of a messaging system with complex routing.
Integration and data exchange between heterogeneous systems.

Ecosystem:
Support for various protocols (AMQP, MQTT, STOMP).
A large number of plugins to extend functionality.

Kafka and RabbitMQ solve different messaging problems and have their own advantages and limitations. Kafka is suitable for processing large volumes of data and stream processing, while RabbitMQ is better at handling asynchronous processing tasks and complex message routing. The choice between them depends on the specific requirements and use cases in your project.

➤ What is Eureka Service

Eureka Service is part of Netflix’s microservices development toolkit called Netflix OSS. It is used to register and discover services in distributed systems. Eureka Service is the core of the service discovery service, which allows different microservices to find and interact with each other.

The main components of Eureka:

Eureka Server:
A central server that acts as a registrar for all microservices. Microservices register with Eureka Server and report their status (healthy, down, etc.).

Eureka Client:
Eureka clients that register themselves with Eureka Server and can use it to discover other services.

eureka-service-project
├── eureka-server
│   ├── build.gradle
│   ├── src
│   │   ├── main
│ │ │ ├── java (or kotlin)
│   │   │   │   └── com
│   │   │   │       └── example
│   │   │   │           └── eurekaserver
│   │   │   │               └── EurekaServerApplication.kt
│   │   │   ├── resources
│   │   │   │   └── application.properties
│   │   └── test
│ │ ├── java (or kotlin)
│   │       │   └── com
│   │       │       └── example
│   │       │           └── eurekaserver
│   │       │               └── EurekaServerApplicationTests.kt
├── eureka-client
│   ├── build.gradle
│   ├── src
│   │   ├── main
│ │ │ ├── java (or kotlin)
│   │   │   │   └── com
│   │   │   │       └── example
│   │   │   │           └── eurekaclient
│   │   │   │               ├── EurekaClientApplication.kt
│   │   │   │               └── ServiceInstanceRestController.kt
│   │   │   ├── resources
│   │   │   │   └── application.properties
│   │   └── test
│ │ ├── java (or kotlin)
│   │       │   └── com
│   │       │       └── example
│   │       │           └── eurekaclient
│   │       │               └── EurekaClientApplicationTests.kt
└── settings.gradle

How Eureka Service works:

Registration of services:
Each microservice that is a Eureka client registers itself with Eureka Server when it starts, providing its information such as address, port, and service ID.

Status update:
Clients regularly send "pings" (heartbeats) to the Eureka Server to confirm that they are still active.

Service discovery:
Microservices can use Eureka Client to get a list of all available services registered in Eureka Server and interact with them.

Basic annotations and their usage in Eureka:

@EnableEurekaServer:
Includes Eureka Server.

@EnableEurekaClient:
Includes Eureka Client for registration and service discovery.

@EnableDiscoveryClient:
Includes a generic service discovery mechanism (can be used with various discovery systems, including Eureka).

@LoadBalanced:
Marks a RestTemplate or WebClient to use client load balancing.

Example of using Eureka Service:
In settings.gradle you need to add includeBuild all modules

Example for Eureka Server:

dependencies {
    implementation 'org.springframework.boot:spring-boot-starter-web'
    implementation 'org.springframework.cloud:spring-cloud-starter-netflix-eureka-server'
    testImplementation 'org.springframework.boot:spring-boot-starter-test'
}

server.port=8080
eureka.client.register-with-eureka=false
eureka.client.fetch-registry=false
eureka.server.enable-self-preservation=false

server.port=8761:
Specifies the port on which Eureka Server will run. By default, Eureka Server runs on port 8080.

eureka.client.register-with-eureka=false:
Specifies that the Eureka Server should not register itself as a client. This is a Eureka client configuration parameter that specifies that this Eureka Server will not register with another Eureka Server.

eureka.client.fetch-registry=false:
Specifies that Eureka Server should not fetch the registry of other services. This is also a Eureka client configuration parameter that disables attempts to obtain a list of all registered services.

eureka.server.enable-self-preservation=false:
Disables self-preservation mode on Eureka Server. This mode is designed to protect against the loss of service registrations when the server does not receive pings from clients for a long time. Disabling this mode allows services to be immediately removed from the registry if they do not send pings, which can be useful for testing, but is not recommended for production environments.

@EnableEurekaServer:
This annotation is used to enable the Eureka Server. It is specified on the main class of the application to mark it as a Eureka Server.

import org.springframework.boot.autoconfigure.SpringBootApplication
import org.springframework.boot.runApplication
import org.springframework.cloud.netflix.eureka.server.EnableEurekaServer


@SpringBootApplication
@EnableEurekaServer
class EurekaServerApplication


fun main(args: Array<String>) {
    runApplication<EurekaServerApplication>(*args)
}

Eureka Client example:

dependencies {
    implementation 'org.springframework.boot:spring-boot-starter-web'
    implementation 'org.springframework.cloud:spring-cloud-starter-netflix-eureka-client'
    testImplementation 'org.springframework.boot:spring-boot-starter-test'
}

server.port=0
eureka.client.service-url.defaultZone=http://localhost:8761/eureka/

server.port=0:
Specifies the port on which Eureka Client will be launched. In this case, the client application will run on the port specified automatically.

eureka.client.service-url.defaultZone=http://localhost:8761/eureka/:
Specifies the URL for registering the client with the Eureka Server. The defaultZone parameter specifies the address of the Eureka Server that the client will interact with for registration and service discovery. In this case, the client will register with the Eureka Server running at http://localhost:8761/eureka/.

Additional parameters that can be used in application.properties:

For Eureka Server:

eureka.instance.hostname:
Sets the hostname for Eureka Server.

eureka.instance.prefer-ip-address:
Specifies whether to use the IP address instead of the hostname for registration.

For Eureka Client:

eureka.instance.hostname:
Sets the hostname for Eureka Client.

eureka.instance.prefer-ip-address:
Specifies whether to use an IP address instead of a host name for client registration.

eureka.client.initial-instance-info-replication-interval-seconds:
Specifies the interval in seconds between retries of sending instance metadata to Eureka Server.

eureka.client.registry-fetch-interval-seconds:
Specifies the interval in seconds between client attempts to obtain an updated service register from Eureka Server.

eureka.client.instance-info-replication-interval-seconds:
Specifies the interval in seconds between sending information about a service instance to Eureka Server.

@EnableEurekaClient:
This annotation is used to enable a Eureka client. It is specified on the main application class to mark it as a Eureka client. The client automatically registers with the Eureka Server and can use it to discover other services.

@EnableDiscoveryClient:
This annotation is a more general annotation for enabling service discovery. It allows an application to use different service discovery mechanisms, including Eureka. It is used similarly to the @EnableEurekaClient annotation.

import org.springframework.boot.autoconfigure.SpringBootApplication
import org.springframework.boot.runApplication
import org.springframework.cloud.netflix.eureka.EnableEurekaClient


@SpringBootApplication
@EnableEurekaClient
class EurekaClientApplication


fun main(args: Array<String>) {
    runApplication<EurekaClientApplication>(*args)
}

import org.springframework.cloud.client.discovery.DiscoveryClient
import org.springframework.web.bind.annotation.GetMapping
import org.springframework.web.bind.annotation.RestController


@RestController
class ServiceInstanceRestController(private val discoveryClient: DiscoveryClient) {

  
    @GetMapping("/service-instances")
    fun serviceInstances(): List<String> {
        return discoveryClient.services
    }
    

}

@LoadBalanced:
This annotation is used to mark a RestTemplate or WebClient to use Ribbon-based client load balancing (if used). This allows a RestTemplate or WebClient to make HTTP requests to named services registered with Eureka, with load balancing across service instances.

import org.springframework.boot.autoconfigure.SpringBootApplication 
import org.springframework.boot.runApplication
import org.springframework.cloud.client.loadbalancer.LoadBalanced
import org.springframework.context.annotation.Bean
import org.springframework.web.client.RestTemplate


@SpringBootApplication
class LoadBalancedApplication {

  
    @Bean
    @LoadBalanced
    fun restTemplate(): RestTemplate {
        return RestTemplate()
    }

}


fun main(args: Array<String>) {
    runApplication<LoadBalancedApplication>(*args)
}

➤ How to implement Spring Cloud Gateway

Spring Cloud Gateway is a modern API gateway built on Spring Framework 5, Spring Boot 2, and the Spring WebFlux project. It provides powerful capabilities for routing and managing API traffic in a microservices architecture. Spring Cloud Gateway is designed to provide easy integration with other Spring Cloud components and to work in an asynchronous and reactive environment.

Key features of Spring Cloud Gateway:

Request Routing:
Spring Cloud Gateway can route requests to different microservices based on various criteria such as path, headers, request parameters, and more.

Filters:
Allows you to apply filters to requests and responses. Filters can modify requests, add or change headers, perform authentication and authorization, logging, and more.

Load Balancing:
Built-in load balancing support allows you to distribute requests across multiple microservice instances.

Integration with Eureka:
Spring Cloud Gateway integrates easily with Eureka and other service discovery systems to dynamically route requests to registered services.

Support for asynchronous processing:
Built on the Spring WebFlux reactive stack, allowing it to handle large numbers of parallel requests with high performance.

Safety:
Support for various authentication and authorization mechanisms, including OAuth2.

Example of using Spring Cloud Gateway:
Create a new Spring Boot project with dependencies for Spring Cloud Gateway and Spring Boot Actuator.

eureka-service-project
├── eureka-server
│   ├── build.gradle
│   ├── src
│   │   ├── main
│   │   │   └── ...
│   │   └── ...
├── eureka-client
│   ├── build.gradle
│   ├── src
│   │   ├── main
│   │   │   └── ...
│   │   └── ...
├── api-gateway
│   ├── build.gradle
│   ├── src
│   │   ├── main
│   │   │   └── ...
│   │   └── ...
└── settings.gradle

Configuring settings.gradle

rootProject.name = 'eureka-service-project'
include 'eureka-server', 'eureka-client', 'api-gateway'

api-gateway/build.gradle

dependencies {
    implementation 'org.springframework.boot:spring-boot-starter-webflux'
    implementation 'org.springframework.cloud:spring-cloud-starter-gateway'
    implementation 'org.springframework.cloud:spring-cloud-starter-netflix-eureka-client'
    testImplementation 'org.springframework.boot:spring-boot-starter-test'
}
dependencyManagement {
    imports {
        mavenBom "org.springframework.cloud:spring-cloud-dependencies:2021.0.1"
    }
}

api-gateway/src/main/resources/application.properties

server.port=8080
spring.cloud.gateway.discovery.locator.enabled=true
spring.cloud.gateway.discovery.locator.lower-case-service-id=true
eureka.client.service-url.defaultZone=http://localhost:8761/eureka/

api-gateway/src/main/kotlin/com/example/apigateway/ApiGatewayApplication.kt

import org.springframework.boot.autoconfigure.SpringBootApplication
import org.springframework.boot.runApplication
import org.springframework.cloud.client.discovery.EnableDiscoveryClient

@SpringBootApplication
@EnableDiscoveryClient
class ApiGatewayApplication

fun main(args: Array<String>) {
    runApplication<ApiGatewayApplication>(*args)
}

Explanations of the settings:

server.port=8080:
API Gateway will run on port 8080.

spring.cloud.gateway.discovery.locator.enabled=true:
Enables automatic route discovery for microservices registered in Eureka.

spring.cloud.gateway.discovery.locator.lower-case-service-id=true:
Converts service identifiers to lowercase to simplify routing.

eureka.client.service-url.defaultZone=http://localhost:8761/eureka/:
Eureka Server URL for registration and service discovery.

Request Routing:
Spring Cloud Gateway automatically creates routes for all microservices registered with Eureka. For example, if you have a microservice named eureka-client, requests to API Gateway at /eureka-client/** will be routed to the corresponding microservice.
Routing can be configured in a configuration file or programmatically.

src/main/resources/application.properties

server.port=8080
spring.cloud.gateway.discovery.locator.enabled=true
spring.cloud.gateway.discovery.locator.lower-case-service-id=true
eureka.client.service-url.defaultZone=http://localhost:8761/eureka/
# Defining routes for eureka-client
spring.cloud.gateway.routes[0].id=eureka-client
spring.cloud.gateway.routes[0].uri=lb://eureka-client
spring.cloud.gateway.routes[0].predicates[0]=Path=/eureka-client/**
spring.cloud.gateway.routes[0].filters[0]=RewritePath=/eureka-client/(?<remaining>.*), /${remaining}

Configuration notes:

server.port=8080:
Specifies that Spring Cloud Gateway will run on port 8080.

spring.cloud.gateway.discovery.locator.enabled=true:
Enables automatic route discovery for services registered in Eureka.

spring.cloud.gateway.discovery.locator.lower-case-service-id=true:
Converts service identifiers to lowercase to simplify routing.

eureka.client.service-url.defaultZone=http://localhost:8761/eureka/:
Specifies the Eureka Server URL for registration and service discovery.

routes:
Setting up routes. In this case, requests starting with /eureka-client/** will be redirected to the eureka-client service.

➤ How to implement redirection using Spring Cloud Gateway

Let's assume we have two microservices registered in Eureka: order-service and inventory-service.
We want to configure routing in Spring Cloud Gateway such that requests starting with /orders/ are redirected to order-service, and requests starting with /inventory/ are redirected to inventory-service.

server.port=8080
# Enable automatic route discovery for services registered in Eureka
spring.cloud.gateway.discovery.locator.enabled=true
spring.cloud.gateway.discovery.locator.lower-case-service-id=true
# Specify the Eureka Server URL for registration and service discovery
eureka.client.service-url.defaultZone=http://localhost:8761/eureka/
# Defining routes for order-service
spring.cloud.gateway.routes[0].id=order-service
spring.cloud.gateway.routes[0].uri=lb://order-service
spring.cloud.gateway.routes[0].predicates[0]=Path=/orders/**
spring.cloud.gateway.routes[0].filters[0]=RewritePath=/orders/(?<remaining>.*), /${remaining}
# Defining routes for inventory-service
spring.cloud.gateway.routes[1].id=inventory-service
spring.cloud.gateway.routes[1].uri=lb://inventory-service
spring.cloud.gateway.routes[1].predicates[0]=Path=/inventory/**
spring.cloud.gateway.routes[1].filters[0]=RewritePath=/inventory/(?<remaining>.*), /${remaining}

Explanation of parameters:

server.port:
Specifies that Spring Cloud Gateway will run on port 8080.

spring.cloud.gateway.discovery.locator.enabled:
Enables automatic route discovery for services registered in Eureka.

spring.cloud.gateway.discovery.locator.lower-case-service-id:
Converts service identifiers to lowercase to simplify routing.

eureka.client.service-url.defaultZone:
Specifies the Eureka Server URL for registration and service discovery.

spring.cloud.gateway.routes:
routes: Section for defining routes in Spring Cloud Gateway.

Route for order-service:

id:
Unique identifier of the route. In this case, it is order-service.

type:
URI for redirecting requests. The lb:// prefix indicates the use of the built-in load balancer, and order-service is the name of the service registered in Eureka.

predicates:
The conditions that must be met for a request to be forwarded along this route.

Path=/orders/:
Path condition: All requests starting with /orders/** will match this route.

filters:
List of filters that will be applied to requests and responses.

RewritePath=/orders/(?.*), /${remaining}:
A filter that rewrites the request path. In this case, the /orders/ part of the path will be removed from the request before it is sent to order-service. The rest of the path will be preserved and passed on.

Route for inventory-service:

id:
Unique identifier of the route. In this case, it is inventory-service.

type:
URI to forward requests to. The lb:// prefix indicates the use of the built-in load balancer, and inventory-service is the name of the service registered in Eureka.

predicates:
The conditions that must be met for a request to be forwarded along this route.

Path=/inventory/:
Path condition: All requests starting with /inventory/** will match this route.

filters:
List of filters that will be applied to requests and responses.

RewritePath=/inventory/(?.*), /${remaining}:
A filter that rewrites the path of the request. In this case, the /inventory/ part of the path will be removed from the request before it is sent to inventory-service. The rest of the path will be preserved and passed on.

➤ What is Service Discovery

Service Discovery is a mechanism that allows microservices to find and interact with each other in a distributed system without having to explicitly specify network addresses. In modern microservice architectures, where the number of services can be very large, manually managing their addresses becomes complex and inconvenient. Service Discovery automates the process of finding services and simplifies scaling and management of the system.

The main components of Service Discovery are:

Service Registry:
A service registrar is a central repository where all available services are registered with their metadata such as IP addresses, ports, and states. Examples of service registrars include Consul, Eureka, Zookeeper, and Etcd.

Service Clients:
Service clients are components that interact with the service registrar to register, update, and find services.

Health Checks:
Health check mechanisms are used to monitor the availability and health of services. Services regularly send information about their health to the service registrar to keep the data up to date.

Types of Service Discovery:

Client-Side Discovery:
In this approach, client applications themselves query the registrar service to obtain addresses of available services. Client discovery is usually implemented using libraries integrated into client applications.
Example:
Netflix Eureka
Consul

Server-Side Discovery:
In this approach, client applications send requests to a server proxy or gateway (such as API Gateway), which interacts with a service registrar and forwards requests to the desired services.
Example:
AWS Elastic Load Balancer (ELB)
Kubernetes Service

➤ What tools are there for monitoring microservices

Prometheus:
Monitoring and alerting system with a powerful query language.

Grafana:
A platform for visualizing metrics and creating dashboards.

Elasticsearch, Logstash, Kibana (ELK Stack):
Solution for centralized logging and data analysis.

Jaeger, Zipkin:
Distributed tracing tools.

➤ How to use Elasticsearch in Spring

Elasticsearch is an open-source, distributed search and analytics system designed to store, search, and analyze large amounts of data in real time. Elasticsearch is often used for full-text search, logging, and analytics.

dependencies {
    implementation("org.springframework.boot:spring-boot-starter-data-elasticsearch")
}

spring.elasticsearch.uris=http://localhost:9200
spring.elasticsearch.username=elastic
spring.elasticsearch.password=your_password

import org.springframework.data.annotation.Id
import org.springframework.data.elasticsearch.annotations.Document
import org.springframework.data.elasticsearch.annotations.Field


@Document(indexName = "products")
data class Product(
    @Id
    val id: String? = null,
    val name: String,
    val description: String,
    val price: Double,
    @Field(type = Date) 
    val date: String;
)

import org.springframework.data.elasticsearch.repository.ElasticsearchRepository
import org.springframework.stereotype.Repository


@Repository
interface ProductRepository : ElasticsearchRepository<Product, String> {
  
    fun findByName(name: String): List<Product>
    
}

import org.springframework.stereotype.Service


@Service
class ProductService(private val productRepository: ProductRepository) {
  
    fun findAllProducts(): List<Product> {
        return productRepository.findAll().toList()
    }
    
    fun saveProduct(product: Product): Product {
        return productRepository.save(product)
    }
    
    fun findProductByName(name: String): List<Product> {
        return productRepository.findByName(name)
    }
    
}

import org.springframework.web.bind.annotation.*

  
@RestController
@RequestMapping("/products")
class ProductController(private val productService: ProductService) {
  
    @GetMapping
    fun getAllProducts(): List<Product> {
        return productService.findAllProducts()
    }
    
    @PostMapping
    fun createProduct(@RequestBody product: Product): Product {
        return productService.saveProduct(product)
    }
    
    @GetMapping("/search")
    fun searchProductsByName(@RequestParam name: String): List<Product> {
        return productService.findProductByName(name)
    }
    
}

➤ What is Trace ID and Span ID

Trace ID and Span ID are key components of distributed tracing, which are used to monitor and debug distributed systems. They help track and link requests across multiple services, allowing you to understand and diagnose system behavior.

Trace ID:
This is a unique identifier that is assigned to each request that passes through the system. This identifier remains unchanged throughout the life cycle of the request and is used to link all operations related to that request.

Target:
Trace ID allows you to combine all spans (parts of a request) into one common trace, so you can see the full path of a request through all microservices and components of the system.

Span ID:
is a unique identifier for a specific operation or part of a request. Each span represents a separate step or segment of a request, such as calling a method or executing a database query.

Target:
Span ID allows you to identify and track individual operations within a single request, providing detailed information about each step.

Example of using Trace ID and Span ID:
Let's look at how Trace ID and Span ID can be used in a Spring Boot application using the Spring Cloud Sleuth library, which integrates with Zipkin for distributed tracing.

dependencies {
    implementation("org.springframework.boot:spring-boot-starter-web")
    implementation("org.springframework.cloud:spring-cloud-starter-sleuth")
    implementation("org.springframework.cloud:spring-cloud-starter-zipkin")
    implementation("com.fasterxml.jackson.module:jackson-module-kotlin")
    implementation("org.jetbrains.kotlin:kotlin-reflect")
    implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
    testImplementation("org.springframework.boot:spring-boot-starter-test")
}
dependencyManagement {
    imports {
        mavenBom("org.springframework.cloud:spring-cloud-dependencies:2020.0.4")
    }
}

spring.zipkin.base-url=http://localhost:9411
spring.sleuth.sampler.probability=1.0

import org.springframework.web.bind.annotation.GetMapping
import org.springframework.web.bind.annotation.RestController
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.web.client.RestTemplate


@RestController
class TraceController(
  @Autowired val restTemplate: RestTemplate
) {

    @GetMapping("/trace")
    fun trace(): String {
        val response = restTemplate.getForObject(
          "http://localhost:8081/external",
          String::class.java
        )
        return "Trace ID and Span ID example: $response"
    }

    @GetMapping("/external")
    fun external(): String {
        return "External service response"
    }

}

Run Zipkin using Docker:

docker run -d -p 9411:9411 openzipkin/zipkin

Run your Spring Boot application and it will automatically generate Trace ID and Span ID for all requests.

View trace:
Go to http://localhost:9411 to open the Zipkin interface.
Send a GET request to http://localhost:8080/trace.
Go back to the Zipkin interface and find the trace associated with your request. You will see a Trace ID and Span ID showing the path of the request through your services.

Java Spring – Microservices