CloverETL and Facebook Graph API

Share this article:

Today we'll tackle how to utilize CloverETL with Facebook. Facebook Graph API allows applications to access social graphs. This blog post describes how to use Graph API within CloverETL.

Querying Facebook via Graph API consist of two steps:

  1. Generate an access token using App ID and App Secret
  2. Use the access token to invoke API calls

We’ll use App Access Token, as it doesn’t require user interaction during the generation process and does not expire. Check out the documentation to see the differences between available tokens and their limitations. For example App Access Token does not allow to use "me" as userId or search queries are limited to Page and Place type only.

Register the Facebook App

Let’s get to it. First, you need to register the Facebook app. Note that you have to be a verified Facebook user to be able to register the application. Go to https://developers.facebook.com/apps and register a new application there.

CloverETL and Facebook Graph API

After registering your application, you’ll see App ID and App Secret in the application dashboard. You’ll need these two values in the CloverETL graph.

CloverETL and Facebook Graph API

CloverETL and Facebook

Now we can proceed to CloverETL. This example graph retrieves Facebook friends and stores them in an XML file in alphabetical order.

Facebook Graph API and CloverETL data integration

It uses the HTTPConnector component to retrieve the access token. The query URL is https://graph.facebook.com/oauth/access_token. App ID and App Secret are used as HTTP request parameters.

The access token query in the example is:

https://graph.facebook.com/oauth/access_token?client_id=${APP_ID}&client_secret=${APP_SECRET}&grant_type=client_credentials

The response content is:

access_token=702578739774856|B0HwJl3tIFM4ShwBXHEuyB3rB20

The access token is parsed using the Reformat component and its value is passed to the next HTTPConnector where it’s used as a query parameter. The query URL is https://graph.facebook.com/{user_id}/friends – this time with the access_token request parameter. This is a regular Graph API call. See the Graph API documentation for other available methods.

An example query API call:

https://graph.facebook.com/${USER_ID}/friends?access_token=702578739774856%7CB0HwJl3tIFM4ShwBXHEuyB3rB20

Response JSON is parsed using the JSONExtract component and one more HTTP request (https://graph.facebook.com/{user_id}) is performed for each result (=each friend). This query retrieves detailed information about the friend.

The JSONExtract mapping is very simple because it uses automapping by field names. Mapping used in JSONExtract:

<Mappings>
<Mapping element="json_object" outPort="0">
</Mapping>
</Mappings>

The final part of the graph just sorts detailed info about friends using the ExtSort component and writes them into an XML file using XMLWriter. The XML writer uses this mapping:

<?xml version="1.0" encoding="UTF-8"?>
<root xmlns:clover="http://www.cloveretl.com/ns/xmlmapping">
<clover:element clover:name="facebook_item" clover:inPort="0">
<clover:element clover:name="first_name">$0.first_name</clover:element>
<clover:element clover:name="last_name">$0.last_name</clover:element>
<link>$0.link</link>
<username>$0.username</username>
</clover:element>
</root>

to produce XML like this (real values are replaced by “...” - I do not want to show my friend’s names here):

<?xml version="1.0" encoding="ISO-8859-1"?>
<root>
<facebook_item>
<first_name>...</first_name>
<last_name>...</last_name>
<link>http://www.facebook.com/...</link>
<username>...</username>
</facebook_item>
<facebook_item>
<first_name>...</first_name>
<last_name>...</last_name>
<link>http://www.facebook.com/...</link>
<username>...</username>
</facebook_item>

</root>

Download the example

Connecting to Facebook using CloverETL - Example

Share this article:
Contact Us

Further questions? Contact us.

Forum

Talk to peers on our forum.

Want to keep in touch?

Follow our social media.

Topics

see all