I will explain the sample code of Spring Batch.

architecture

The developer creates the "Job" part of Figure 1.Job is a unit of batch processing, and the inside consists of multiple "Steps".Write the execution control rule of each step in an XML file.You can also specify various attributes such as restartable.

Figure 1 Architecture

"Job Launcher" in Figure 1 is provided by Spring Batch.The developer gets the job you want to execute from JobLauncher (provided) and executes Job with the job parameter as an argument.

Job execution status (status) is stored in "Job Repository" in a timely manner.Internally, RDB is used, and its management table is provided by Spring Batch.Wagby has a mechanism for viewing this management table.

Reader-Processor-Writer pattern

The "Step" created by the developer is further provided with the framework of Reader, Processor, Writer.Reader reads a certain amount of data from database or file.We treat the read data as "Item" and pass it to Processor.Processor does something, but when it reaches the commit number specified in advance, it passes Item to Writer.The number read by the Reader does not necessarily match the number written to Writer.Writer writes the received Item to the database or file.This processing operation called Reader, Processor, Writer is collectively called "Chunk".

Figure 2 Reader-Processor-Writer pattern
Developers can use model classes and service classes automatically generated by Wagby (in a batch program).Without using SQL, you can develop high quality batch programs in a short time.

Here is a sample of code conforming to the Reader-Processor-Writer pattern in Figure 2.

Figure 3 Structure of sample code

MyItemReader

Create a class that implements the ItemReader interface provided by Spring Batch.Read the data.

We will target the customer model.We will collect data one by one by using the service class automatically generated by Wagby.

package jp.jasminesoft.wagby.batch.chunk;

import java.util.List;
import jp.jasminesoft.jfc.dao.CriteriaConverter;
import jp.jasminesoft.jfc.dao.FinderContext;
import jp.jasminesoft.jfc.service.JFCEntityService;
import jp.jasminesoft.wagby.model.customer.Customer;
import jp.jasminesoft.wagby.model.customer_c.CustomerC;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.NonTransientResourceException;
import org.springframework.batch.item.ParseException;
import org.springframework.batch.item.UnexpectedInputException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.stereotype.Component;

@Component("MyItemReader")
public class MyItemReader implements ItemReader<Customer> {
    @Autowired
    @Qualifier("CustomerEntityService")
    protected JFCEntityService<Customer, Integer> customerEntityService;

    @Autowired
    @Qualifier("CustomerCriteriaConverter")
    protected CriteriaConverter<CustomerC> converter;
  
    private FinderContext<CustomerC> finderContext;
    private List<Customer> results;
    private int index;

    public void init() {
        finderContext = new FinderContext<CustomerC>();
        finderContext.setCondition(new CustomerC());
        finderContext.setPageSize(10);
        finderContext.setCriteriaConverter(converter);
        results = customerEntityService.find(finderContext);
    }

    public Customer read() throws Exception, UnexpectedInputException,
    ParseException, NonTransientResourceException {
        if (finderContext == null) {
       	    init();
        }
        if (index >= results.size()) {
       	    if (finderContext.isNextPage()) {
       	        finderContext.next();
       	        results = customerEntityService.find(finderContext);
       	        index = 0;
       	    } else {
       	        return null;
       	    }
        }
        Customer customer = results.get(index++);
        System.out.println("index="+index+",customer="+customer);
        return customer;
    }
}

MyItemProcessor

Create a class implementing the ItemProcessor interface provided by Spring Batch.We will process the data.

In this case, the input value is returned as it is.In general, we perform calculation processing and processing processing here.

package jp.jasminesoft.wagby.batch.chunk;

import jp.jasminesoft.wagby.model.customer.Customer;
import org.springframework.batch.item.ItemProcessor;

public class MyItemProcessor implements ItemProcessor<Customer,Customer> {
    @Override
    public Customer process(Customer customer) throws Exception {
        return customer; // Do nothing
    }
}

MyItemWriter

Create a class that implements the ItemWriter interface provided by Spring Batch.Write processing of processed data is performed.

Here, the value is output to the console.In general, it describes processing to write to a database or file.

package jp.jasminesoft.wagby.batch.chunk;

import java.util.List;
import jp.jasminesoft.wagby.model.customer.Customer;
import org.springframework.batch.item.ItemWriter;

public class MyItemWriter implements ItemWriter<Customer> {
    @Override
    public void write(List<? extends Customer> data) throws Exception {
        System.out.println("MyItemWriter,data="+data);
    }
}

MyItemListener

This class is not mandatory.

Developers can create classes that inherit ItemListenerSupport.The following example confirms the behavior of the method beforeRead called immediately before the read operation.

package jp.jasminesoft.wagby.batch.chunk;

import jp.jasminesoft.wagby.model.customer.Customer;
import org.springframework.batch.core.listener.ItemListenerSupport;

public class MyItemListener extends ItemListenerSupport<Customer,Customer> {

    @Override
    public void beforeRead() {
        System.out.println("beforeRead !");
    }
}

I will introduce the code to execute the job using "Job Launcher" in Figure 1.

MyShowCustomerController

It describes the process of calling the SpringBatch program prepared by the developer at the timing when the original button is pressed.

package jp.jasminesoft.wagby.controller.customer;

import static jp.jasminesoft.util.ExcelFunction.*;

import java.security.Permission;
import java.io.File;
import java.io.IOException;
import java.util.*;
import java.sql.*;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import jp.jasminesoft.util.*;
import jp.jasminesoft.jfc.*;
import jp.jasminesoft.jfc.controller.*;
import jp.jasminesoft.jfc.service.*;
import jp.jasminesoft.jfc.menu.*;

import jp.jasminesoft.wagby.app.*;

import org.apache.log4j.Logger;
import org.apache.commons.lang3.*;
import org.apache.commons.lang3.math.NumberUtils;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameter;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersInvalidException;
import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException;
import org.springframework.batch.core.repository.JobRestartException;
import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException;
import org.springframework.batch.core.launch.support.SimpleJobLauncher;

/**
 * SpringBatch Sample
 *
 * @author JasmineSoft
 */
@Controller
public class MyShowCustomerController
    extends ShowCustomerController
{
    @Override
    public String do_original(DbActionParameter p)
    throws IOException, ServletException, SecurityException
    {
        SimpleJobLauncher launcher = (SimpleJobLauncher)p.appctx.getBean("jobLauncher");
        Job job = (Job)p.appctx.getBean("Batch1");
        try {
            System.out.println("job="+job);
	    java.util.Map<String,JobParameter> params = new HashMap<String,JobParameter>();
	    params.put("date1", new JobParameter(new java.util.Date()));
            JobExecution jobExecution = launcher.run(job, new JobParameters(params));
            System.out.println("jobExecution="+jobExecution);
        } catch (JobExecutionAlreadyRunningException e) {
        } catch (JobRestartException e) {
        } catch (JobInstanceAlreadyCompleteException e) {
        } catch (JobParametersInvalidException e) {
        }
        return do_show(p);
    }
}
  • SimpleJobLauncher is a class provided by SpringBatch.It provides basic functions for calling jobs prepared by developers.
  • The job name "Batch 1" is explained in the next chapter.
  • The job parameter date1 is dummy information.In fact it is not used.It is described as an example of how to pass parameters here.
  • For details on how to extend the controller class (prefix My with the override only required methods)Customization using JavaPlease read.

Finally, we summarize the classes developed by the developer so far and give them names. In Wagby and Spring Batch cooperation, the file name is fixed as batch - context.xml.Please describe the job flow in this file.

Job, Step, Chunk, Listener can be specified in the job flow.Chunk deals with data collectively in commit units.

Figure 4 Job Flow
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:batch="http://www.springframework.org/schema/batch" 
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/batch
		http://www.springframework.org/schema/batch/spring-batch.xsd
		http://www.springframework.org/schema/beans 
		http://www.springframework.org/schema/beans/spring-beans.xsd">

  <bean id="MyItemReader" class="jp.jasminesoft.wagby.batch.chunk.MyItemReader"/>
  <bean id="MyItemProcessor" class="jp.jasminesoft.wagby.batch.chunk.MyItemProcessor"/>
  <bean id="MyItemWriter" class="jp.jasminesoft.wagby.batch.chunk.MyItemWriter"/>
  <bean id="MyItemListener" class="jp.jasminesoft.wagby.batch.chunk.MyItemListener"/>

  <!-- Batch1 start -->
  <batch:job id="Batch1">
    <batch:step id="step1">
      <batch:tasklet>
	<batch:chunk reader="MyItemReader" processor="MyItemProcessor"
		     writer="MyItemWriter" commit-interval="1">
	  <batch:listeners>
	    <batch:listener ref="MyItemListener" />
	  </batch:listeners>
	</batch:chunk>
      </batch:tasklet>
    </batch:step>
  </batch:job>
  <!-- Batch1 end -->
</beans>
This XML file works with Wagby R7.10 or later version.If you use Wagby before that, please change "spring - batch.xsd" part to "spring - batch - 2.2.xsd".
  • The job requires an identifier (ID).I decided here as "Batch 1".
  • For simplicity, we have one job step.Multiple steps can be prepared.

Commit interval

For chunk you can specify an attribute called commit - interval.By doing this, it is possible to run the ItemWriter once, for example, after executing N ItemReaders.

For multiple reads, one style of writing is a good idea.

Restart position

The internal management table remembers the number committed by ItemReader. When restarting, reading is resumed from that position.

Skip processing at exception occurrence

An exception can be thrown when an abnormality is detected by processing in ItemProcessor.

In the skippable-execution-classes element, specify an exception to skip on exception detection and continue processing.This process is rolled back.

You can also specify skip-limit (upper limit value of skip count).If it exceeds this, the job will be treated as failed.

<batch:tasklet>
  <batch:chunk reader="MyItemReader" processor="MyItemProcessor"
                writer="MyItemWriter" skip-limit="10">
    <batch:skippable-exception-classes>
      <batch:include class="jp.jasminesoft.wagby.job.MySkipException" />
    </batch:skippable-exception-classes>
  </batch:chunk>
</batch:tasklet>

Conditional branch

You can specify conditions with the on attribute of the next element.

<batch:job id="job1">
  
   <batch:step id="step1">
      <next on="*" to="next3"/>
      <next on="FAILED" to="next2"/>
   </batch:step>
 
   <batch:step id="step2" next="step3"/>
  
   <batch:step id="step3"/>
</batch:job>

A job instance is a concept for distinguishing started jobs.

Figure 5 Job Instance

If the combination of job ID and job parameter is the same, it is regarded as the same job instance. A job that ended normally can not be re-executed.(Prevention of double start)

The number of job execution results is one at normal termination.If there is a rerun (re-execution), it has N job execution results.

When starting one job many times

In SpringBatch, it is assumed that target data to be processed is decided by job parameters for jobs to be executed.

for that reason,If it is the same job parameter, re-execution is not possible. If you want to start the same job multiple times, please change job parameters intentionally.

In general, specify the date for job parameters.(For example, if target = 2016 - 12 - 01 is specified, only data for that day will be covered and data on other days will not be used.)

The points of job design are as follows.

  • How far is the process to put in one "step"?
    • Failure to make it finer, it is easy to shorten the recovery time at re-execution.
    • If it is too fine, it becomes difficult to control.
  • Does it sometimes execute updating batch processing in parallel while accepting online processing?
    • What should be the behavior of batch processing when locking on the online side?
    • When locking with batch processing, can you let me fail on the online side?

Database session

Apart from online processing (database processing by screen operation), Wagby prepares a database session dedicated to Spring Batch.

The connection destination database information is the same as online processing.This is the information described in "Environment> Database" of Designer.

Therefore, the transaction manager is also separate from the online system.It can not be a transaction that spans online and batch (Spring Batch).

The added files are as follows.

customize/java/jp/jasminesoft/wagby/batch/chunk/MyItemListener.java
customize/java/jp/jasminesoft/wagby/batch/chunk/MyItemProcessor.java
customize/java/jp/jasminesoft/wagby/batch/chunk/MyItemReader.java
customize/java/jp/jasminesoft/wagby/batch/chunk/MyItemWriter.java
customize/java/jp/jasminesoft/wagby/controller/customer/MyShowCustomerController.java
customize/webapp/WEB-INF/applicationContext/batch-context.xml

You can download this file.Please expand it under the customize folder.

Separately, prepare the "customer" model before trying.Customer The definition of the model is irrelevant.
Enable "Set Java source code> Output class for customization".
Please prepare your own button on the detail screen.(The button name is optional.All event names, action names, and additional parameters should be left blank.)
Fig. 6 Output class for customization & Setting original button