Transforming CSV to Java Objects using Camel

by

In this tutorial, we will explore how to use Apache Camel in combination with Bindy to produce and consume CSV files. Camel is an open-source integration framework that provides a powerful routing and mediation engine. Bindy is a Camel component that allows for the transformation of Java objects to/from various data formats, including CSV.

Prerequisites

To follow along with this tutorial, you will need:

  • Java Development Kit (JDK) installed on your machine
  • Apache Maven installed on your machine
  • Basic knowledge of Java and Apache Camel

Step 1: Set Up your project dependencies

Include in your pom.xml the following dependencies:

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-core</artifactId>
</dependency>
 
<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-bindy</artifactId>
</dependency>

Step 2: Create CSV Data Model

  1. Create a new package com.sample.csv in the src/main/java directory.
  2. Inside the package, create a new Java class User.
  3. Define properties and corresponding getter and setter methods for the User class. For example:
package com.sample.csv;

import java.io.Serializable;
import org.apache.camel.dataformat.bindy.annotation.CsvRecord;
import org.apache.camel.dataformat.bindy.annotation.DataField;

@CsvRecord(separator = ",", generateHeaderColumns = true)
public class User implements Serializable {
  @Override
  public String toString() {
    return "User [name=" + name + ", surname=" + surname + ", id=" + id + "]";
  }

  @DataField(pos = 1)
  private String name;

  @DataField(pos = 2)
  private String surname;

  private String id;

  // Getter/Setters omitted for brevity

}

Here are some details about the CSV annotations:

@CsvRecord: This annotation marks the class as a CSV record and allows customization of CSV-specific properties. In this case, separator = "," specifies that the CSV fields are separated by commas, and generateHeaderColumns = true indicates that the header line should be generated in the CSV file.

@DataField: This annotation marks the fields that need to be mapped to CSV columns. The pos attribute specifies the position of the field in the CSV record. In this case, name corresponds to the first field (position 1), and surname corresponds to the second field (position 2).

In the next section we will learn how to convert a CSV file to Java and viceversa using the above model.

From CSV to Java with Camel

In the first example, we will consume a CSV file and unmarshal it using BindyCsvDataFormat. To do that, we need to create an instance of it specifying the Model to use:

final DataFormat bindy = new BindyCsvDataFormat(User.class);

Next, let’s code a simple Route:

from("file:src/main/resources/data?noop=true&include=.*\\.csv")
  .split(body().tokenize("\n"))
    .log("Body: ${body}")
      .unmarshal(bindy).process(new SimpleProcessor()).log("Body: ${body}");

from("file:src/main/resources/data?noop=true&include=.*\\.csv"): This line sets up the route’s starting point. It consumes files from the directory src/main/resources/data with the file extension .csv. The noop=true option prevents the files from being deleted after processing.

.split(body().tokenize("\n")): This line splits the content of the file into individual lines. It uses the split EIP (Enterprise Integration Pattern) to iterate over each line.

.log("Body: ${body}"): This line logs the content of each line to the console. The ${body} placeholder represents the current message body.

.unmarshal(bindy): This line performs unmarshalling, converting the CSV data into Java objects using the Bindy library. The bindy component is responsible for data format conversion.

.process(new SimpleProcessor()): This line invokes a custom processor, SimpleProcessor, to perform additional processing on the unmarshalled data. The processor can implement custom logic for manipulating or transforming the data.

The Processor performs a simple Task: it adds a random UID to the User Class:

public class SimpleProcessor implements Processor {
  public void process(Exchange exchange) throws Exception {
      User user = (User) exchange.getIn().getBody();
      user.setId(UUID.randomUUID().toString());
      exchange.getIn().setBody(user);
    
  }
}

To learn more about Processors, check this article: Using a Camel Processor to modify the message

Finally, add a sample CSV file in the resources/data folder of your application:

John,Doe
Jane,Smith
Michael,Johnson
Emily,Williams

Here is the project view:

camel csv tutorial

Running the example

By running our route, you will see that the CSV entries are unmarshalled and the Processor does its job to add an userid:

[file://src/main/resources/data] route1                         INFO  Body: User [name=surname, surname=name, id=4950fcde-1635-40b4-bbd6-c0797ff44c20]
[file://src/main/resources/data] route1                         INFO  Body: John,Doe
[file://src/main/resources/data] route1                         INFO  Body: User [name=Doe, surname=John, id=c7e3985a-a71f-4524-be5a-d6b12be10f63]
[file://src/main/resources/data] route1                         INFO  Body: Jane,Smith
[file://src/main/resources/data] route1                         INFO  Body: User [name=Smith, surname=Jane, id=51c4cea1-99ff-4a43-86e5-5f3932cbb48d]
[file://src/main/resources/data] route1                         INFO  Body: Michael,Johnson
[file://src/main/resources/data] route1                         INFO  Body: User [name=Johnson, surname=Michael, id=6cc698a9-1dfd-469e-a63f-3933e6850fe3]
[file://src/main/resources/data] route1                         INFO  Body: Emily,Williams
[file://src/main/resources/data] route1                         INFO  Body: User [name=Williams, surname=Emily, id=05069b1b-91c5-453d-a0cf-8b61568b1a16]

From Java to CSV with Camel

In the send part of this article, we will do the reverse step: from a Java User class, we will produce an entry in a CSV file. Since we will not use a Processor in this step, the Camel Route is pretty lean:

from("direct:start")
.marshal(bindy).log("Body: ${body}");

To trigger the direct:start component, we will use a ProducerTemplate and add an User object in it:

ProducerTemplate template = context.createProducerTemplate();
User user = new User();
user.setName("John");
user.setSurname("Smith");

template.sendBody("direct:start", user);

Since our CSVRecord contains generateHeaderColumns = true , the CSV file that we will marshall will contain the Headers for the User Class:

INFO  Body: surname,name
Smith,John

Advanced Transformation using @BindyConverter

Sometimes you need to perform transformation of data when transforming from Java to CSV and viceversa. You can use the @BindyConverter for this purpose and specify in it a Class that will act as a Converter. The provided class must implement the Format interface.

For example:

 @DataField(pos =  1)
 @BindyConverter(CustomConverter.class)
 public String name;

Here is a sample Converter class that simply reverses the field value:

public static class CustomConverter implements Format<String> {
    @Override
    public String format(String object) throws Exception {
        return (new StringBuilder(object)).reverse().toString();
    }

    @Override
    public String parse(String string) throws Exception {
        return (new StringBuilder(string)).reverse().toString();
    }
}

Conclusion

In this tutorial, we explored how to leverage the power of Apache Camel and the Bindy component to handle CSV files effectively. Apache Camel provides a flexible integration framework, while Bindy simplifies the transformation of Java objects to and from CSV format. By combining these technologies, you can streamline the process of producing and consuming CSV files in your applications.

Source code for this article: https://github.com/fmarchioni/masterspringboot/tree/master/camel/csv

Found the article helpful? if so please follow us on Socials
Twitter Icon       Facebook Icon       LinkedIn Icon       Mastodon Icon