www.javatodev.com Open in urlscan Pro
172.67.168.34  Public Scan

Form analysis
0 forms found in the DOM

Text Content

JavaToDev
 * Home
 * Start Here
 * Spring Boot
 * Microservices
 * DevOps
 * AWS

Sign in Subscribe
Spring Boot Featured


HOW TO USE AMAZON SQS WITH SPRING BOOT

 * 

CHINTHAKA DINADASA

Nov 29, 2022 • 5 min read

Amazon SQS (Simple Queue Service) allows us to configure software components to
send, receive and store messages. Here in this tutorial we will look into a
spring boot solution which configured with Amazon SQS with practical scenarios.


REQUIREMENT

Personal budget application has a requirement on generating invoices on expenses
when an user create a expense entry.

Here application should not wait until processing invoice generation when user
save expenses and user should get generated invoice emailed after completion.


BRIEF SOLUTION

We can design our API to accept expense creation and push those data to a AWS
SQS Queue. Then from a processor service the same published message can be
pulled and start processing.

Then we can proceed with sending email for the user with generated invoice on
completion of the expense processing and invoice generation.




CREATING QUEUE ON AWS SQS

In this tutorial we need a single message queue where we are going to store the
messages. There are 2 ways in creating this queue,

 1. Using AWS dashboard
 2. Using AWS SDK

💡
You can choose any preferred way to achieve this target, and here we are using
AWS SDK to create the queue programmatically.

--------------------------------------------------------------------------------




DEVELOPMENT

This sample project will have 2 spring boot components which will act as
publisher and consumer.

Tech stack that we are going to use,

 * Spring Boot 3.0.0
 * Java 17
 * Gradle
 * Lombok
 * AWS Java SDK SQS

You can create applications using spring initializr or follow our tutorial on
How to Create a Spring Boot Project.

Creating spring boot publisher application using spring initializr

Add following dependency to the application build.gradle, this will add
necessary dependencies to access amazon SQS through the SDK.

implementation 'com.amazonaws:aws-java-sdk-sqs:1.12.349'



PROJECT PREREQUISITES

Here we need to have AWS access key id and secret which has AWS SQS permission
granted, to manage SQS programmatically.

💡
Here I've created an user from AWS IAM console with programatical access and
AmazonSQSFullAccess permission.

Then Add following into the application properties in both publisher and
consumer applications.

app.config.aws.access_key_id=<YOUR KEY ID>
app.config.aws.secret_key_id=<YOUR SECRET>


Also we are using a name to uniquely identify the message queue and it will be
loaded from application properties. Feel free to directly use it as a string as
well.

Add this to application.properties in both publisher and consumer applications.

app.config.message.queue.topic=JAVATODEV_SQS_QUEUE_EXPENSES



MESSAGE PUBLISHER SPRING BOOT APPLICATION DEVELOPMENT

First, We should have to build the communication from our application to AWS
SQS. To do that we can use Amazon SQS interface which allows us to access amazon
SQS.

Additionally this can be created as a bean to the application since the we have
to use it everywhere when we need to access AWS SQS as follows,

package com.javatodev.app.configuration;

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.sqs.AmazonSQS;
import com.amazonaws.services.sqs.AmazonSQSClientBuilder;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class SQSClientConfiguration {

    @Value("${app.config.aws.access_key_id}")
    private String awsAccessKeyId;

    @Value("${app.config.aws.secret_key_id}")
    private String awsSecretKeyId;

    @Bean
    public AmazonSQS amazonSQSClient() {
        BasicAWSCredentials awsCredentials = new BasicAWSCredentials(awsAccessKeyId, awsSecretKeyId);
        return AmazonSQSClientBuilder.standard()
            .withCredentials(new AWSStaticCredentialsProvider(awsCredentials))
            .withRegion("us-west-1")
            .build();
    }
}


Then create the MessageQueueService class where we are mainly doing everything
related to message queue handling in publisher side.

package com.javatodev.app.service;

import com.amazonaws.services.sqs.AmazonSQS;
import com.amazonaws.services.sqs.model.CreateQueueRequest;
import com.amazonaws.services.sqs.model.CreateQueueResult;
import com.amazonaws.services.sqs.model.QueueNameExistsException;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;

import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;

@Slf4j
@Service
@RequiredArgsConstructor
public class MessageQueueService {

    @Value("${app.config.message.queue.topic}")
    private String messageQueueTopic;

    private final AmazonSQS amazonSQSClient;

    public void createMessageQueue() {
    
        log.info("Creating message queue on AWS SQS");

        CreateQueueRequest request = new CreateQueueRequest();
        request.setQueueName(messageQueueTopic);

        try {
            CreateQueueResult queue = amazonSQSClient.createQueue(request);
            log.info("Create Queue Response {}", queue.getQueueUrl());
        } catch (QueueNameExistsException e) {
            log.error("Queue Name Exists {}", e.getErrorMessage());
        }

    }
}


Let's create another Configuration class where we use @PostConstruct annotation
to create this message queue on application startup.

package com.javatodev.app.configuration;

import com.javatodev.app.service.MessageQueueService;

import org.springframework.context.annotation.Configuration;

import jakarta.annotation.PostConstruct;
import lombok.RequiredArgsConstructor;

@Configuration
@RequiredArgsConstructor
public class AppConfiguration {

    private final MessageQueueService messageQueueService;

    @PostConstruct
    public void initializeMessageQueue() {
        messageQueueService.createMessageQueue();
    }

}


Then create an endpoint where we can submit messages using AWS SDK client we
have already configured.

Add following to the MessageQueueService we have created earlier.

public void publishExpense(CreateExpenseDto createExpenseDto) {
        try {
            GetQueueUrlResult queueUrl = amazonSQSClient.getQueueUrl(messageQueueTopic);
            log.info("Reading SQS Queue done: URL {}", queueUrl.getQueueUrl());
            amazonSQSClient.sendMessage(queueUrl.getQueueUrl(), createExpenseDto.getType() + ":" + createExpenseDto.getAmount());
        } catch (QueueDoesNotExistException | InvalidMessageContentsException e) {
            log.error("Queue does not exist {}", e.getMessage());
        }

    }





MESSAGE CONSUMER SPRING BOOT APPLICATION DEVELOPMENT

We have already discussed and developed the first part on building communication
with SQS client, Just add SQSClientConfiguration class and related app
configurations to the consumer application as well.

Here in this sample project we are using spring scheduler to pull things from
the message queue in AWS.

First enable scheduling in the consumer application using @EnableScheduling
annotation.

package com.javatodev.app;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.scheduling.annotation.EnableScheduling;

@EnableScheduling
@SpringBootApplication
public class SqsSpringbootConsumerApplication {

    public static void main(String[] args) {
        SpringApplication.run(SqsSpringbootConsumerApplication.class, args);
    }
}



Then create the MessageQueueService which involves in pulling messages from the
message queue and process the request.

Here we are going to process message one by one, hence only processing first
message from the messages list.

package com.javatodev.app.service;

import com.amazonaws.services.sqs.AmazonSQS;
import com.amazonaws.services.sqs.model.Message;
import com.amazonaws.services.sqs.model.QueueDoesNotExistException;
import com.amazonaws.services.sqs.model.ReceiveMessageResult;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;

import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;

@Slf4j
@Service
@RequiredArgsConstructor
public class MessageQueueService {

    @Value("${app.config.message.queue.topic}")
    private String messageQueueTopic;

    private final AmazonSQS amazonSQSClient;

    @Scheduled(fixedDelay = 5000) //executes on every 5 second gap.
    public void receiveMessages() {
        try {
            String queueUrl = amazonSQSClient.getQueueUrl(messageQueueTopic).getQueueUrl();
            log.info("Reading SQS Queue done: URL {}", queueUrl);

            ReceiveMessageResult receiveMessageResult = amazonSQSClient.receiveMessage(queueUrl);

            if (!receiveMessageResult.getMessages().isEmpty()) {
                Message message = receiveMessageResult.getMessages().get(0);
                log.info("Incoming Message From SQS {}", message.getMessageId());
                log.info("Message Body {}", message.getBody());
                processInvoice(message.getBody());
                amazonSQSClient.deleteMessage(queueUrl, message.getReceiptHandle());
            }

        } catch (QueueDoesNotExistException e) {
            log.error("Queue does not exist {}", e.getMessage());
        }
    }

    private void processInvoice(String body) {
        log.info("Processing invoice generation and sending invoice emails from here..");
    }

}




💡
AWS ECS temporarily hide a message when an consumer pulled it. hence same
message will not be delivered again and again for given period of time.
💡
We should handle message after we pulled it from AWS SQS with acknowleding it,
and sending delete message request with the message ID. Otherwise same message
will be available to pull again which will cause message duplications.



DEAD LETTER QUEUE

Dead letter queue allows us to handle any unprocessed messages which coming from
publishers. Additionally this allows us to debug our messaging queue setup in
order to identify why some messages are not getting processed.


CONCLUSIONS AND FURTHER READING

In this article we have discussed how we can setup Amazon SQS queue with Spring
boot application on a practical scenario.

Refer official documentation on Amazon SQS from here.

The implementation of all these examples and code snippets can be found in our
Github repository.

Happy coding.


Featured


HOW TO SETUP HASHICORP VAULT WITH SPRING BOOT APPLICATION

Assume we need to setup secure storage for our application tokens, passwords or
any other secret and sensitive data. HashiCorp vault allows us to setup secure
access to those sensitive data with reduce the risk of breaches and data
exposure with identity-based security automation and encryption-as-a-service. In
this article we
Dec 14, 2022 5 min read 2 comments
Featured


TASK SCHEDULING ON SPRING BOOT USING @SCHEDULER

In this article we are going to discuss how we can build task scheduling
requirements in spring boot using @scheduler annotation. Assume, you have a
requirement to automate repetitive tasks which run on specific time gap, or
specific time in a month, week or year, you can use scheduler job
Dec 3, 2022 4 min read
Featured


MICROSERVICES - CENTRALIZED CONFIGURATIONS WITH SPRING CLOUD CONFIG


Mar 13, 2022 5 min read
JavaToDev © 2022
 * Sign up
 * Contact Us
 * DMCA
 * Cookie Privacy Policy
 * Privacy Policy
 * Terms & Conditions
 * About Us
 * Resources

Powered by Ghost




AddThis Sharing
Facebook
, Number of shares
Twitter
, Number of shares
Print
, Number of shares
Email
, Number of shares
AddThis
, Number of shares
AddThis Sharing
FacebookTwitterPrintEmailAddThis