9 min read

In this article by P. Raja Malleswara Rao, author of the book Spring Batch Essentials, we will see how the Spring Batch provides the configuration to write the read and processed data to a different output (destination). The writer can integrate easily with different relational frameworks. It can also be customized for the different formats.

(For more resources related to this topic, see here.)

ItemWriter

Spring Batch provides an interface in the form of ItemWriter to write bulk data.The following is the definition of the ItemWriter interface:

public interface ItemWriter<T> { void write(List<? extends T> items) throws Exception; }


Based on the destination platform onto which we have to write the data, we have the following item writers:

  • Flat file item writers
    : These write the content onto a flat file (fixed width and delimited)
  • XML item writers: These write the data onto an XML file
  • Database item writers : These write the data onto a database

Flat file item writers

The data read from any of the existing formats can be processed to the desired format and then be written onto multiple formats, including flat files. The following are the APIs that help in flat file item writing.

LineAggregator

The LineAggregatorAPI concatenates multiple fields into a String to write onto the flat file. This works exactly the opposite way of LineTokenizer in the read operation.

public interface LineAggregator<T> { public String aggregate(T item); }


PassThroughLineAggregator

PassThroughLineAggregator is an implementation of
LineAggreagator that considers the object in use is already aggregated and simply returns the String from the object using the toString() method.

public class PassThroughLineAggregator<T> implements LineAggregator<T> { public String aggregate(T item) { return item.toString(); } }


The FlatFileItemWriter can be configured with the
PassThroughLineAggregator, as follows:

<bean id=”itemWriter” class=” org.springframework.batch.item.file.FlatFileItemWriter”> <property name=”resource” value=”file:target/outputfiles/employee_output.txt”/> <property name=”lineAggregator”> <bean class=” org.springframework.batch.item.file.transform.PassThroughLineAggregator”/> </property> </bean>


FieldExtractor

If the object writing is more than just writing its String form onto the file, FieldExtractor needs to be used, wherein each object gets converted to the array of fields, aggregated together to form a String to write onto the file.

public interface FieldExtractor<T> { Object[] extract(T item); }


Field extractors are primarily of two types:

  • PassThroughFieldExtractor: For the scenario where the object collection has to just be converted to the array and passed to write
  • BeanWrapperFieldExtractor: With a field-level configuration of how each field of the object gets placed in the String to write onto the file, this works exactly the opposite way of BeanWrapperFieldSetMapper

The BeanWrapperFieldSetExtractor works as follows:

BeanWrapperFieldExtractor<Employee> extractor = new BeanWrapperFieldExtractor<Employee>(); extractor.setEmployees(new String[] { “id”, “lastname”, “firstname”,”designation”,”department”,”yearofjoining”}); int id = 11; String lastname = “Alden”; String firstname = “Richie”; String desination = “associate”; String department = “sales”; int yearofjoining = 1996; Employee employee = new Employee(id, lastname, firstname,designation, department, yearofjoining); Object[] values = extractor.extract(n); assertEquals(id, values[0]); assertEquals(lastname, values[1]); assertEquals(firstname, values[2]); assertEquals(designation, values[3]); assertEquals(department, values[4]); assertEquals(yearofjoining, values[5]);/p>

Writing delimited files

If the Java object can be written onto the flat files in delimited file format, we can perform it as shown in the following example. Let’s consider the Employee object defined already.

This object can be configured with the FlatFileItemWriter, the DelimitedLineAggregator, and the BeanWrapperFieldExtractor to perform the delimited flat file, as follows:

<bean id=”itemWriter” class=”org.springframework.batch.item.file.FlatFileItemWriter”> <property name=”resource” ref=”outputResource”/> <property name=”lineAggregator”> <bean class=” org.springframework.batch.item.file.transform.DelimitedLineAggregator”> <property name=”delimiter” value=”,”/> <property name=”fieldExtractor”> <bean class=” org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor”> <property name=”employees” value=”id,lastname,firstname,designation,department,yearofjoining”/> </bean> </property> </bean> </property> </bean>


Writing a fixed width file

Spring Batch supports fixed width file writing with the help of
FormatterLineAggregator. Considering the same example data as delimited flat file writing, we can perform the fixed width file writing using the following configuration:

<bean id=”itemWriter” class=”org.springframework.batch.item.file.FlatFileItemWriter”> <property name=”resource” ref=”outputResource”/> <property name=”lineAggregator”> <bean class=” org.springframework.batch.item.file.transform.FormatterLineAggregator”> <property name=”fieldExtractor”> <bean class=” org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor”> <property name=”employees” value=” id,lastname,firstname,designation,department,yearofjoining”/> </bean> </property> <property name=”format” value=”%-2d%-10s%-10s%-10s%-15s%-4d”/> </bean> </property> </bean>


The format value is formed based on the following listed formatter conversions, where argrepresents the argument for conversion:

Conversion Category Description
b, B general This converts Boolean to the String format. The value is falsefor null
h, H general This is the Integer.toHexString(arg.hashCode())
s, S general If argimplements Formattable, then arg.formatTo()Otherwise, arg.toString()
c, C character This is a Unicode character
d integral This is a decimal integer
o integral This is an octal integer
x, X integral This is a hexadecimal integer
e, E floating point This is a decimal number in computerized scientific notation
f floating point This is a decimal number
g, G floating point This is a computerized scientific notation or decimal format, depending on the precision and value after rounding
a, A floating point This is a hexadecimal floating point number with a significand and an exponent
t, T date/time This is the prefix for date and time conversion characters
% percent This is a literal %(u0025)
n line separator This is the platform-specific line separator

FlatFileItemWriter can be configured with the shouldDeleteIfExists option, to delete a file if it already exists in the specified location. The header and footer can be added to the flat file by implementing FlatFileHeaderCallBack and FlatFileFooterCallBack and including these beans with the headerCallback and footerCallback properties respectively.

XML item writers

The data can be written to the Extensible Markup Language
(XML)
format using StaxEventItemWriter. The Spring Batch configuration for this activity, for the employee example can be the following:

<bean id=”itemWriter” class=”org.springframework.batch.item.xml.StaxEventItemWriter”> <property name=”resource” ref=”outputResource”/> <property name=”marshaller” ref=”employeeMarshaller”/> <property name=”rootTagName” value=”employees”/> <property name=”overwriteOutput” value=”true”/> </bean>


Using the XStream to do the marshalling activity, the following is the configuration:

<bean id=”employeeMarshaller” class=”org.springframework.oxm.xstream.XStreamMarshaller”> <property name=”aliases”> <util:map id=”aliases”> <entry key=”employee” value=”batch.Employee”/> <entry key=”ID” value=”java.lang.Integer”/> </util:map> </property> </bean>


The Java code for the preceding configuration can be realized as follows:

StaxEventItemWriter staxItemWriter = newStaxEventItemWriter(); FileSystemResource resource = new FileSystemResource(“export/employee_output.xml”) Map aliases = newHashMap(); aliases.put(“employee”,”batch.Employee”); aliases.put(“ID”,”java.lang.Integer”); Marshaller marshaller = newXStreamMarshaller(); marshaller.setAliases(aliases); staxItemWriter.setResource(resource); staxItemWriter.setMarshaller(marshaller); staxItemWriter.setRootTagName(“employees”); staxItemWriter.setOverwriteOutput(true); ExecutionContext executionContext = newExecutionContext(); staxItemWriter.open(executionContext); Employee employee = new Employee(); employee.setID(11); employee.setLastName(“Alden”); employee.setFirstName(“Richie”); employee.setDesignation(“associate”); employee.setDepartment(“sales”); employee.setYearOfJoining(“1996”); staxItemWriter.write(employee);


Database item writers

Spring Batch supports database item writing with two possible access types: JDBC and ORM.

JDBC-based database writing

Spring Batch supports JDBC-based database writing with the help of
JdbcBatchItemWriter, which is an implementation of ItemWriter, which executes multiple SQL statements in the batch mode. The following is the sample configuration for the employee example with the JDBC-based database writing:

<bean id=”employeeWriter” class=”org.springframework.batch.item.database.JdbcBatchItemWriter”> <property name=”assertUpdates” value=”true” /> <property name=”itemPreparedStatementSetter”> <bean class=”batch.EmployeePreparedStatementSetter” /> </property> <property name=”sql” value=”INSERT INTO EMPLOYEE (ID, LASTNAME, FIRSTNAME, DESIGNATION, DEPARTMENT, YEAROFJOINING) VALUES(?, ?, ?, ?, ?, ?)” /> <property name=”dataSource” ref=”dataSource” /> </bean>


The ItemPreparedStatementSetter can be implemented for our example of Employee data as follows:

public class EmployeePreparedStatementSetter implements ItemPreparedStatementSetter<Employee> { @Override public void setValues(Employee item, PreparedStatement ps) throws SQLException { ps.setInt(1, item.getId()); ps.setString(2, item.getLastName()); ps.setString(3, item.getFirstName()); ps.setString(4, item.getDesignation()); ps.setString(5, item.getDepartment()); ps.setInt(6, item.getYearOfJoining()); } }


ORM-based database writing

Object relational mapping(ORM) is defined as a programming technique to convert data between incompatible type systems in object-oriented programming languages. ORM takes care of the data persistence from the object oriented program to the database. Spring Batch supports multiple ORMs including Hibernate, JPA, and iBatis.

In our example, the Employee class should be annotated to be used with ORM (Hibernate/JPA) for persistence as follows:

@Entity(“employee”) public class Employee { @Id(“id”) private int id; @Column(“lastName”) private String lastName; @Column(“firstName”) private String firstName; @Column(“designation”) private String designation; @Column(“department”) private String department; @Column(“yearOfJoining”) private int yearOfJoining; public int getID() { return id; } public void setID(int id) { this.id = id; } public String getLastName() { return lastName; } public void setLastName(String lastName) { this.lastName = lastName; } public String getFirstName() { return firstName; } public void setFirstName(String firstName) { this.firstName = firstName; } public String getDesignation() { return designation; } public void setDesignation(String designation) { this.designation = designation; } public String getDepartment() { return department; } public void setDepartment(String department) { this.department = department; } public int getYearOfJoining() { return yearOfJoining; } public void setYearOfJoining(int yearOfJoining) { this.yearOfJoining = yearOfJoining; } }


The annotations specify that the class Employee is representing a corresponding table in the database with a name as shown with @Entity, and each field corresponds to a column in the database as shown with the @ID and @Column annotations.

The following is the configuration to be made with Hibernate for the employee example:

<bean id=”employeeWriter” class=”org.springframework.batch.item.database.HibernateItemWriter”> <property name=”hibernateTemplate” ref=”hibernateTemplate” /> </bean>


Similarly, for JPA and iBatis, the configurations can be made with JpaItemWriter and IbatisBatchItemWriter respectively.

Custom item readers and writers

Spring Batch supports custom item readers’ and writers’ configurations. This can be done easily by implementing the ItemReader and ItemWriter interfaces for the respective read and write operations with the business logic we want, and configuring the ItemReader and ItemWriter in the XML batch configuration.

Summary

Through this article we learned the essential data handling mechanism – writing the data to different destinations including flat files, XML, and databases. Now we have an understanding of the Spring Batch support that is used to custom formats by implementing the interface to match the business needs that are different from the default formats.

Resources for Article:

 


Further resources on this subject:

LEAVE A REPLY

Please enter your comment!
Please enter your name here