Blog Archives

How to Add Header and Footer to a File

I will demonstrate you, how to add custom header, footer to file. This is the often required functionality for file processing and validation.

Requirement: Create delimited file with below structure.

Header Part

File Name: Name of the actual File.

PID : process ID.

Header Columns.

Body Part

Text Data. 

Footer Part

File Created Date: Date & Time.

Number of Records: record count.

This is our final job design.

Final Job Design

Final Job Design

Step 1: We`ll create Header part first.

Step 2: Add tFixedFlowInput component and configure as below.

  • Add one column “value” with string type.
  • Select “Use inline table” option and add three rows.
  • Add each line of below code to the each line we created previously.

"File Name: CustomerDetails.csv"

"PID :"+pid

"Id,Name,City,State,Street"

  • See the image for more details.
tFixedFlowInput Setting

tFixedFlowInput Setting

Step 3: Add tFileOutputDelimited and connect with tFixedFlowInput component using Main flow, then configured as follows.

  • Add file path name “C:/home/CustomerDetails.csv”.(you can change this)
  • Row separator=”n”
  • Field Separator=”,”

Step 4: Add tRowGenerator and configured as explained in previous blog post.

Step 5: Copy Paste tFileOutputDelimited_1 component which we have created in Step 3, then connect  tFileOutputDelimited_2 with tRowGenerator using main flow. Configure as follows.

  • Select Option “Append”.
  • Click on  “Main” flow and go to the “Advance setting” tab and select option “Monitor this connection”.
  • Unchecked “Use Input Connection as Label”. it will show you a “Label” and text box to write, write  “Number Of Records:” in same text box.

Step 6: Create context variable named as “NumberOfRows”, then copy and paste tFixedFlowInput_1.

Step 7: Configure newly pasted  tFixedFlowInput_2 component.

  • Delete second last line from list.
  • One second line add this code "File Created Date: "+TalendDate.getDate("dd-MMM-yyyy HH:mm:ss").
  • on third line add the context variable. context.NumberOfRows. 

Step 8: Copy paste tFileOutputDelimited_2 and connect with tFixedFlowInput_2 using main flow. no need to do any additional configuration.

Step 9: Add tFlowMeterCatcher component and add tJavaRow and connect each other using main flow, don`t click on “Synch” button on tJavaRow. Write below code in tJavaRow component.

context.NumberOfRows=input_row.label+input_row.count;

Step 10: Run the job it will create file with Header part, Data part and footer part. below is our final output.

Header and Footer Output

Header and Footer Output

 

 

 

Advertisements

Generate sample data in Talend

This post we will generate sample data for further utilisation.  In ETL world, if you want to test some components you must need data, but getting a desire sample data is very difficult.

To generate sample data we will use tRowGenerator component which has built in editor where you can select functions or write your won expressions to get expected sample data.

Spte 1: Start writing on Talend designer canvas “trow…” it will show you list of component, from the list select tRowGenerator component.

Note: This is new feature from Talend wherein you don`t need to search component in pallet and then drag and drop.

See in picture.

Add Component writing  on Talend Designer pan

Add Component writing on Talend Designer pan

Step 2: Double click on component and do the setting in tRowGenerator using it`s editor.

  • Click on [+] sign to add new column, add new column with name =”name”,
  • Select function form “Function” tab on same columns grid.
  • Select “TalendDataGenerator.getFirstName” function from function list.
  • Add following columns and select the relavant function as we did previously.
    • City = TalendDataGenerator.getUsCity
    • State=TalendDataGenerator.getUsState
    • Street=TalendDataGenerator.getUsStreet
  • Now we have 4 columns but we need another one column for Identity number, so add column “ID” with “integer” data type.
  • In function Tab select “” (three dots). You will see function parameters window with single row, down to the columns grid.
  • There are three tabs first one “Parameter” is fixed, with no edit option, second on for “value” and last one for “comment”
  • Click on value tab it will show you “…” dots then click on it, it opens expression builder for edit. you can add your custom logic here.
  • Select “Numeric” routine then select “sequence” keep the default values.
  • In “Number of Rows for RowGenerator” text box write value=10. ( we required only ten rows to be generated)
  • Click on preview button on below window it will show you generated sample data as a result, it will look like below Image.
tRowGenerator setting

tRowGenerator setting

For demonstration we have generated only ten rows, but you are free to generate as many as rows you require.

Step 3: Add tLogRow, and connect with tRowGenerator using main flow.

Step 4: Run the job it will show you below result.

tRowGenerator output sample data

tRowGenerator output sample data

If you want to insert this data to the file or database then use the specific compoent e.g. tFileOutputDelimited to store in delimited file.