Monthly Archives: March 2013
To get user`s detail we will have to use users/show API so make sure you are able to access this API. Once you click on mention API you will gate a XML file which has many details, so making it simple and understandable we will take below details from XML, if you want then you can take all the details.
Create a Job in Talend Open Studio and follow the step for creating mapping/Schema of users/show XML.
Click on Metadata Node, and right Click on File XML node, then click on Create File XML option from Pop up.
Provide mapping name and users/show XML File.
Once done go to the next Tab and configure all the properties like below screen. and select below listed fields for display.
Our Xpath Loop expression is: /user/status
select below list of field using Ctrl+click and drag and drop to “Fileds to extract” and click on “Refresh Preview” button to make sure you have parsed XML properly.
Now we have ready our sample file with twitter user details. we have to store this information into CSV file, hence you need to drag and drop tFileOutPutDelimited, Drag and Drop schema mapping we created recently for XML on designer and select tFileInputXML. Connect tFileInputXML to tFileOutPutDelimited using Main connector , and synch source schema to tFileOutPutDelimited component,
Give the output file path and name other configuration. once done execute the job to sure every thing is working fine. your final job look like below with output.
Here we have completed two part of Twitter API, one to get the user ID`s and other one is to get user details.
In next part of this post we will integrate both jobs in single one to retrieve each user id and their details in CSV file.
I splitted this post into 4 parts, so readers can go to the specific topic.
Requirement for Demo.
Talend Open Studio.
Twitter API Access.
Access to below API.
Above 4 API will used to get followers, Friends, and user details, our sample Twitter user name is “pubscode”.
first we will call API to get all the followers Id`s then because we don`t have any other information associated with each Id so we have used these Id`s to get the detail information about each Id, and each detail information will be stored in .CSV file.
Id`s API will return all the followers Id`s in below XML format so before move ahead we will create Mapping/Schema.
There are various ways to get API call done in Talend so I am explaining you a simplest way which i use.
Create a job with name “Twitter_API” and drop tFileFetch. Select tFileFetch and click on “Component Tab” you will see all the properties of tFileFetch so below screen will help you to configure properties.
To make sure we have configured all the things properly just run the job and check whether you got the file “pubscode.xml” in your “Destination directory” text box you specified.
Now we have our sample file ready to process using tFileInputXML
Drop tFileInputXML component from Palette and click on “Component Tab” to read XML file. select tFileInputXML component to configure all the required properties as shown in below screen.
once you configured tFileInputXML Component connect tFileFetch to tFileInputXML using “OnComponentOK” trigger. final job will look like below screen.
We have downloaded users ID`s in XML format, Next Blog will see, how can we get details of each user and stored their details to .CSV file.
Required Things For Demo.
Sample XML (Download)
Talend Open Studio Installed
Here is our Sample XMl, you can Download it from above link.
<?xml version=”1.0″ encoding=”UTF-8″?>
<item id=”111″ clientName=”SB”>
<item id=”112″ clientName=”GJ”>
- Create a new job named as “XML_From_GDrive_2_CSV“.
- Create Metadata for above sample XML file using Metadata repository wizard see below screen.
- Once you Created Schema/ Mapping then Drop the tFileInputXML from metadata you created in the Job and make it as Built in.
- Drop tFileFetch, tFileOutPutDelimited.
- Configure tFileFetch property as below screen.
- Set above XML download path in URI property.
- Now configure tFileInputXML with our Download Directory path with file name as below.
- Your final job will be look like this run the application and check for result it will be shown you result we can see here.