www.baeldung.com Open in urlscan Pro
172.66.43.8  Public Scan

Form analysis
1 forms found in the DOM

POST https://www.getdrip.com/forms/15866254/submissions

<form method="post" action="https://www.getdrip.com/forms/15866254/submissions"><input type="hidden" value="https://www.getdrip.com/forms/15866254/submissions">
  <div>
    <div style="padding-top: 20px;">
      <input class="input-email" name="fields[email]" id="drip-email" type="email" placeholder="Your Email" value="" label="Email Address">
    </div>
    <div><input name="website" id="website" type="text" placeholder="website" value="" label="Website" style="display:none"></div>
    <div style="padding-top: 20px;"><button type="submit">FOLLOW THE SPRING</button></div>
  </div>
</form>

Text Content

Baeldung Live - Do Not Process My Personal Information

If you wish to opt-out of the sale, sharing to third parties, or processing of
your personal or sensitive information for targeted advertising by us, please
use the below opt-out section to confirm your selection. Please note that after
your opt-out request is processed you may continue seeing interest-based ads
based on personal information utilized by us or personal information disclosed
to third parties prior to your opt-out. You may separately opt-out of the
further disclosure of your personal information by third parties on the IAB’s
list of downstream participants. This information may also be disclosed by us to
third parties on the IAB’s List of Downstream Participants that may further
disclose it to other third parties.



 * Personal Data Processing Opt Outs

CONFIRM
Data DeletionData AccessPrivacy Policy

 * 
 * 
 * Start Here
 * Courses ▼▲
   
   
    * REST WITH SPRING BOOT
      
      The canonical reference for building a production grade API with Spring
   
   
    * LEARN SPRING SECURITY ▼▲
      
      THE unique Spring Security education if you’re working with Java today
      
      
       * LEARN SPRING SECURITY CORE
         
         Focus on the Core of Spring Security 6
      
      
       * LEARN SPRING SECURITY OAUTH
         
         Focus on the new OAuth2 stack in Spring Security 6
   
   
    * LEARN SPRING
      
      From no experience to actually building stuff
   
   
    * LEARN SPRING DATA JPA
      
      The full guide to persistence with Spring Data JPA

 * Guides ▼▲
   
   
    * SPRING BOOT
      
      Get started and go deep into Spring Boot 3
   
   
    * PERSISTENCE
      
      The Persistence with Spring guides
   
   
    * REST
      
      The guides on building REST APIs with Spring
   
   
    * SECURITY
      
      The Spring Security guides

 * About ▼▲
   
   
    * FULL ARCHIVE
      
      The high level overview of all the articles on the site.
   
   
    * BAELDUNG EBOOKS
      
      Discover all of our eBooks
   
   
    * ABOUT BAELDUNG
      
      About Baeldung.

 * 
 * 
 * 




INTRODUCTION TO SPRING BATCH

Powered By

10 Sec


Spring Security - Remember Me with Persistence (LSS - Module 3 - Lesson 3)




Next
Stay

1





Last updated: June 15, 2023

Written by: Eugen Paraschiv
 * Spring+

 * reference
 * Spring Batch

Course – launch – Black Friday 2024 – NPI (cat=Baeldung)

Yes, we're now running the only sale of the year - our Black Friday launch. All
Courses are 33% off until Monday, December 2nd:

>> EXPLORE ACCESS NOW




1. OVERVIEW



In this tutorial, we’re going to look at a practical, code-focused intro to
Spring Batch. Spring Batch is a processing framework designed for robust
execution of jobs.

It’s current version 5.0.0 supports Spring 6 and Java 17.

Here are a few interesting and practical use cases of the framework.


2. WORKFLOW BASICS



Spring Batch follows the traditional batch architecture where a job repository
does the work of scheduling and interacting with the job.

A job can have more than one step. And every step typically follows the sequence
of reading data, processing it and writing it.



And of course the framework will do most of the heavy lifting for us here —
especially when it comes to the low-level persistence work of dealing with the
jobs — using h2 for the job repository.


2.1. EXAMPLE USE CASE



The simple use case we’re going to tackle here is migrating some financial
transaction data from CSV to XML.

The input file has a very simple structure.

It contains a transaction per line, made up of a username, the user id, the date
of the transaction and the amount:

username, userid, transaction_date, transaction_amount
devendra, 1234, 31/10/2015, 10000
john, 2134, 3/12/2015, 12321
robin, 2134, 2/02/2015, 23411Copy


3. THE MAVEN POM



Dependencies required for this project are Spring Core, Spring Batch and H2
database:



<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-oxm</artifactId>
    <version>6.1.4</version>
</dependency>
<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
    <version>2.1.214</version>
</dependency>
<dependency>
    <groupId>org.springframework.batch</groupId>
    <artifactId>spring-batch-core</artifactId>
    <version>5.1.1</version>
</dependency>Copy


4. SPRING BATCH AND JOB CONFIG



The basic Spring Batch configuration is displayed below along with our job
description for the CSV to XML functionality.



Java-based job configuration:

@Profile("spring")
public class SpringBatchConfig {

    @Value("input/record.csv")
    private Resource inputCsv;

    @Value("file:xml/output.xml")
    private Resource outputXml;

    @Bean
    public ItemReader<Transaction> itemReader()
      throws UnexpectedInputException, ParseException {
        FlatFileItemReader<Transaction> reader = new FlatFileItemReader<Transaction>();
        DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer();
        String[] tokens = { "username", "userid", "transactiondate", "amount" };
        tokenizer.setNames(tokens);
        reader.setResource(inputCsv);
        DefaultLineMapper<Transaction> lineMapper = 
          new DefaultLineMapper<Transaction>();
        lineMapper.setLineTokenizer(tokenizer);
        lineMapper.setFieldSetMapper(new RecordFieldSetMapper());
        reader.setLineMapper(lineMapper);
        return reader;
    }

    @Bean
    public ItemProcessor<Transaction, Transaction> itemProcessor() {
        return new CustomItemProcessor();
    }

    @Bean
    public ItemWriter<Transaction> itemWriter(Marshaller marshaller)
      throws MalformedURLException {
        StaxEventItemWriter<Transaction> itemWriter = 
          new StaxEventItemWriter<Transaction>();
        itemWriter.setMarshaller(marshaller);
        itemWriter.setRootTagName("transactionRecord");
        itemWriter.setResource(outputXml);
        return itemWriter;
    }

    @Bean
    public Marshaller marshaller() {
        Jaxb2Marshaller marshaller = new Jaxb2Marshaller();
        marshaller.setClassesToBeBound(new Class[] { Transaction.class });
        return marshaller;
    }

    @Bean
    protected Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager, ItemReader<Transaction> reader, 
                      ItemProcessor<Transaction, Transaction> processor, ItemWriter<Transaction> writer, ) {
        return new StepBuilder("step1", jobRepository).<Transaction, Transaction> chunk(10, transactionManager)
          .reader(reader).processor(processor).writer(writer).build();
    }

    @Bean(name = "firstBatchJob")
    public Job job(JobRepository jobRepository, @Qualifier("step1") Step step1) {
        return new JobBuilder("firstBatchJob", jobRepository).preventRestart().start(step1).build();
    }
    
    public DataSource dataSource() {
     EmbeddedDatabaseBuilder builder = new EmbeddedDatabaseBuilder();
     return builder.setType(EmbeddedDatabaseType.H2)
           .addScript("classpath:org/springframework/batch/core/schema-drop-h2.sql")
           .addScript("classpath:org/springframework/batch/core/schema-h2.sql")
           .build();
    }
    
    @Bean(name = "transactionManager")
    public PlatformTransactionManager getTransactionManager() {
        return new ResourcelessTransactionManager();
    }
    
    @Bean(name = "jobRepository")
    public JobRepository getJobRepository() throws Exception {
        JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
        factory.setDataSource(dataSource());
        factory.setTransactionManager(getTransactionManager());
        factory.afterPropertiesSet();
        return factory.getObject();
    }
    
    @Bean(name = "jobLauncher")
    public JobLauncher getJobLauncher() throws Exception {
       TaskExecutorJobLauncher jobLauncher = new TaskExecutorJobLauncher();
       jobLauncher.setJobRepository(getJobRepository());
       jobLauncher.afterPropertiesSet();
       return jobLauncher;
    }
}Copy



And the XML-based configuration:

<bean id="itemReader" class="org.springframework.batch.item.file.FlatFileItemReader">
    <property name="resource" value="input/record.csv" />
    <property name="lineMapper">
        <bean
            class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
            <property name="lineTokenizer">
                <bean
                    class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
                    <property name="names"
                        value="username,userid,transactiondate,amount" />
                </bean>
            </property>
            <property name="fieldSetMapper">
                <bean
                    class="com.baeldung.batch.service.RecordFieldSetMapper" />
            </property>
        </bean>
    </property>
    <property name="linesToSkip" value="1" />
</bean>

<bean id="itemProcessor" class="com.baeldung.batch.service.CustomItemProcessor" />

<bean id="itemWriter" class="org.springframework.batch.item.xml.StaxEventItemWriter">
    <property name="resource" value="file:xml/output.xml" />
    <property name="marshaller" ref="marshaller" />
    <property name="rootTagName" value="transactionRecord" />
</bean>

<bean id="marshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
    <property name="classesToBeBound">
        <list>
            <value>com.baeldung.batch.model.Transaction</value>
        </list>
    </property>
</bean>

<batch:job id="firstBatchJob">
    <batch:step id="step1">
        <batch:tasklet>
            <batch:chunk reader="itemReader" writer="itemWriter"
                processor="itemProcessor" commit-interval="10">
            </batch:chunk>
        </batch:tasklet>
    </batch:step>
</batch:job>


<!-- connect to H2 database -->
<bean id="dataSource"
      class="org.springframework.jdbc.datasource.DriverManagerDataSource">
    <property name="driverClassName" value="org.h2.Driver" />
    <property name="url" value="jdbc:h2:file:~/repository" />
    <property name="username" value="" />
    <property name="password" value="" />
</bean>

<!-- create job-meta tables automatically -->
<jdbc:initialize-database data-source="dataSource">
    <jdbc:script
            location="org/springframework/batch/core/schema-drop-h2.sql" />
    <jdbc:script location="org/springframework/batch/core/schema-h2.sql" />
</jdbc:initialize-database>

<!-- stored job-meta in database -->
<bean id="jobRepository"
      class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
    <property name="dataSource" ref="dataSource" />
    <property name="transactionManager" ref="transactionManager" />
    <property name="databaseType" value="h2" />
</bean>

<bean id="transactionManager"
      class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />

<bean id="jobLauncher"
      class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
     <property name="jobRepository" ref="jobRepository" />
</bean>Copy

Now that we have the whole config, let’s break it down and start discussing it.


4.1. READ DATA AND CREATE OBJECTS WITH ITEMREADER



First, we configured the cvsFileItemReader that will read the data from the
record.csv and convert it into the Transaction object:



@SuppressWarnings("restriction")
@XmlRootElement(name = "transactionRecord")
public class Transaction {
    private String username;
    private int userId;
    private LocalDateTime transactionDate;
    private double amount;

    /* getters and setters for the attributes */

    @Override
    public String toString() {
        return "Transaction [username=" + username + ", userId=" + userId
          + ", transactionDate=" + transactionDate + ", amount=" + amount
          + "]";
    }
}Copy

To do so, it uses a custom mapper:

public class RecordFieldSetMapper implements FieldSetMapper<Transaction> {
 
    public Transaction mapFieldSet(FieldSet fieldSet) throws BindException {
        DateTimeFormatter formatter = DateTimeFormatter.ofPattern("d/M/yyy");
        Transaction transaction = new Transaction();
 
        transaction.setUsername(fieldSet.readString("username"));
        transaction.setUserId(fieldSet.readInt(1));
        transaction.setAmount(fieldSet.readDouble(3));
        String dateString = fieldSet.readString(2);
        transaction.setTransactionDate(LocalDate.parse(dateString, formatter).atStartOfDay());
        return transaction;
    }
}Copy




4.2. PROCESSING DATA WITH ITEMPROCESSOR



We have created our own item processor, CustomItemProcessor. This doesn’t
process anything related to the transaction object.

All it does is pass the original object coming from reader to the writer:

public class CustomItemProcessor implements ItemProcessor<Transaction, Transaction> {

    public Transaction process(Transaction item) {
        return item;
    }
}Copy


4.3. WRITING OBJECTS TO THE FS WITH ITEMWRITER



Finally, we are going to store this transaction into an XML file located at
xml/output.xml:

<bean id="itemWriter"
  class="org.springframework.batch.item.xml.StaxEventItemWriter">
    <property name="resource" value="file:xml/output.xml" />
    <property name="marshaller" ref="marshaller" />
    <property name="rootTagName" value="transactionRecord" />
</bean>Copy


4.4. CONFIGURING THE BATCH JOB



So, all we have to do is connect the dots with a job using the batch:job syntax.

Note the commit-interval. That’s the number of transactions to be kept in memory
before committing the batch to the itemWriter.

It will hold the transactions in memory until that point (or until the end of
the input data is encountered).

The Java bean and correspondent XML configuration are displayed:



@Bean
protected Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager, @Qualifier("itemProcessor") ItemProcessor<Transaction,
        Transaction> processor, ItemWriter<Transaction> writer) {
    return new StepBuilder("step1", jobRepository)
            .<Transaction, Transaction> chunk(10, transactionManager)
            .reader(itemReader(inputCsv))
            .processor(processor)
            .writer(writer)
            .build();
}Copy

<batch:job id="firstBatchJob">
    <batch:step id="step1">
        <batch:tasklet>
            <batch:chunk reader="itemReader" writer="itemWriter"
              processor="itemProcessor" commit-interval="10">
            </batch:chunk>
        </batch:tasklet>
    </batch:step>
</batch:job>Copy


4.5. RUNNING THE BATCH JOB



Now let’s set up and run everything:

@Profile("spring")
public class App {
    public static void main(String[] args) {
        // Spring Java config
        AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
        context.register(SpringConfig.class);
        context.register(SpringBatchConfig.class);
        context.refresh();
        
        JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
        Job job = (Job) context.getBean("firstBatchJob");
        System.out.println("Starting the batch job");
        try {
            JobExecution execution = jobLauncher.run(job, new JobParameters());
            System.out.println("Job Status : " + execution.getStatus());
            System.out.println("Job completed");
        } catch (Exception e) {
            e.printStackTrace();
            System.out.println("Job failed");
        }
    }
}Copy

We run our Spring application using -Dspring.profiles.active=spring profile.



In the next section, we configure our example in a Spring Boot application.


5. SPRING BOOT CONFIGURATION



In this section, we’ll create a Spring Boot application and convert the previous
Spring Batch Config to run in the Spring Boot environment. In fact, this is
roughly the equivalent of the previous Spring Batch example.


5.1. MAVEN DEPENDENCIES



Let’s start by declaring spring-boot-starter-batch dependency in a Spring Boot
application in the pom.xml:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-batch</artifactId>
</dependency>Copy

We need a database to store the Spring Batch job information. In this tutorial,
we use an in-memory H2 database. Therefore, we need to use h2 with Spring Boot:

<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
    <version>2.1.214</version>
    <scope>runtime</scope>
</dependency>Copy


5.2. SPRING BOOT CONFIG



We use the @Profile annotation to distinguish between the Spring and Spring Boot
configurations. We set the spring-boot profile in our application:

@SpringBootApplication
public class SpringBatchApplication {

    public static void main(String[] args) {
        SpringApplication springApp = new SpringApplication(SpringBatchApplication.class);
        springApp.setAdditionalProfiles("spring-boot");
        springApp.run(args);
    }

}Copy


5.3. SPRING BATCH JOB CONFIG



We use the batch job configuration the same as the SpringBatchConfig class from
earlier:



@Configuration
public class SpringBootBatchConfig {

    @Value("input/record.csv")
    private Resource inputCsv;

    @Value("input/recordWithInvalidData.csv")
    private Resource invalidInputCsv;

    @Value("file:xml/output.xml")
    private Resource outputXml;

    // ...
}Copy

Starting with spring-boot 3.0, @EnableBatchProcessing annotation is
discouraged. We declare manually JobRepository, JobLauncher and
TransactionManager beans. In addition, JobBuilderFactory and StepBuilderFactory
are deprecated, and it’s recommended to use JobBuilder and StepBuilder class
with the name of the job/step builder.


6. CONCLUSION



In this article, we learned how to work with Spring Batch and how to use it in a
simple use case.

We saw how we can easily develop our batch processing pipeline and how we can
customize different stages in reading, processing, and writing.

As always, the full implementation of this article can be found in over on
GitHub.

Course – launch – Black Friday 2024 – NPI (All)



Yes, we're now running the only sale of the year - our Black Friday launch. All
Courses are 33% off until Monday, December, 2nd:

>> EXPLORE ACCESS NOW




COURSES

 * All Courses
 * Baeldung All Access
 * Baeldung All Team Access
 * The Courses Platform

SERIES

 * Java “Back to Basics” Tutorial
 * Jackson JSON Series
 * Apache HttpClient Series
 * REST with Spring Series
 * Spring Persistence Series
 * Security with Spring
 * Spring Reactive Series

ABOUT

 * About Baeldung
 * The Full Archive
 * Editors
 * Our Partners
 * Partner with Baeldung
 * eBooks
 * FAQ
 * Baeldung Pro

 * Terms of Service
 * Privacy Policy
 * Company Info
 * Contact



Follow the Spring Category


Follow the Spring category to get regular info about the new articles and
tutorials we publish here.
FOLLOW THE SPRING


Freestar.com