APIs, Webhooks and Enriching Integration with FME Server

When shopping for integration solutions you’ll come across two terms that may at times feel interchangeable: ‘API’ and ‘webhook’. But while they share similarities, APIs and webhooks have some important differences, and understanding these distinctions is key if you are to get full value from FME Server.

Application programming interfaces (APIs) and webhooks play a crucial role in today’s software development landscape, providing a means for different systems to communicate and exchange data with one another. In this blog, we’ll explore integrating APIs using FME Server, a powerful and adaptable platform designed to build, manage and deploy APIs with ease.

What are APIs and webhooks? Why are they important? And how do they work? Here’s our comprehensive guide to APIs, webhooks and their implementation using FME Server.

Webhooks vs APIs

First things first: what is a webhook and what is an API?

Application programming interfaces (APIs)

APIs allow software applications to communicate through requests – e.g. App A requests information from App B and App B decides whether or not to return the information.

An analogy: Imagine you’re seated in a restaurant. You (App A) look at the menu and find what you want to eat. You call over your waiter (the API) and give them your order, which the waiter passes to the kitchen (App B). If all’s well, the waiter returns with your food.

The benefits of APIs are many and varied, but include:

• Integration and accessibility: Customers, partners and other third parties can access your systems in a secure and stable way.
• Competitiveness: APIs allow you to do more, and do it better. At their best – usable, intuitive and well-implemented – they can form a true competitive advantage for any organisation.
• Flexibility and adaptability: APIs allow you to use and leverage your desired services, leading to more innovation, faster development and lower risk.
• Cloud computing: APIs mean it’s never been easier to move processing and storage to the cloud, facilitating both initial migration and continuous integration.
• Mobile devices: Sensor-rich devices like smartphones are built for the service-based structure of an API.
• A proven technology: Think of any modern disruptor – Ebay, Uber, Amazon, Salesforce – and their success has been predicated on the use of APIs.

It should be noted that the process of getting the dish from the kitchen – pulling (AKA polling) the information from App B – can be resource intensive with APIs, as you have to ask the waiter every time you want to order food.

Which leads us to webhooks.

Webhooks

Webhooks are a specific type of API that is initiated not by an app’s request for information, but another form of trigger, e.g. a new work order being created, a timesheet being approved or a vehicle moving into a geofence. In this case, the transfer of data is initiated by the application that sends it. App A can sit dormant until an event occurs and a message is triggered by App B, which is why webhooks are sometimes referred to as ‘reverse APIs’.

An analogy: Imagine you’re in a shoe store, and have found a pair that you want. The store doesn’t have your size, but they are happy to order the correct size in. If you applied an API to this situation, it would be up to you to continually call the store to check whether your shoes have arrived. But if you apply a webhook, the arrival of the shoes into stock would trigger the store to call you – a far more efficient use of time for both you and the store.

Webhooks remove the need to poll an API for information. Action is taken whenever predefined criteria are met or information changes. This leads to a number of benefits, including:

• Instantaneousness: Webhooks are ideal for use in applications that utilise real-time data, as they can ensure all information is relevant and up-to-date.
• Efficiency: Rather than having to request information whenever you need it, webhooks deliver it to your door at the perfect moment and without you needing to lift a finger.
• Clarity: The above means an organisation can gain a complete picture of its situation at any given time.
• Integration: APIs and webhooks allow systems to talk to each other seamlessly.

To summarise the above, APIs and webhooks allow applications to share data and communicate without the helping hand of a human.

They allow organisations to automate and integrate systems, which has become ever more  critical since the explosion of cloud-based products and apps. It’s now imperative that tech ecosystems are interconnected, and that data flows easily within them. That’s all thanks to API and webhook integration.

The successful utilisation of APIs and webhooks is far easier with high performance middleware. Take FME Server, which seamlessly brokers information between applications using both APIs and webhooks.

FME can poll an API (known as a ‘GET’ request). It can change information through an API (‘POST’, ‘PUT’ and ‘DELETE’ requests). And it can receive any message broadcast from a webhook.

Now that we understand what APIs and webhooks are, and a little about how they work, let’s now take a look at how to use them with FME.

Creating a web connection

The methods used to connect to an API or register a webhook subscription depends on how the app that you’re dealing with is authenticated. Some of the most common include:

• Basic authentication: A simple username and password (this method is not very secure).
• Token-based authentication: Encrypted text that includes information about the user that expires after a predefined period of time.
• OAuth 2.0: A more complex protocol that involves a series of request and response exchanges to verify you are who you say you are. This is the most secure option, and is widely used in web applications.

FME web connections make using OAuth 2.0 simple, by including ready-to-use web connections for popular platforms such as Google, Microsoft 365, Salesforce and ArcGIS Online. You can also create your own custom web connection using any three of these methods if you so choose.

Pulling data from an API

The process of pulling data from an API is called polling. Polling an API is best when:

• Source information changes infrequently.
• The destination system doesn’t require the most up-to-date information.
• No webhook is available.

The importance of middleware cannot be understated. FME Server offers out-of-the-box functionality that allows it to read data from hundreds of popular platforms, including:

What’s more, FME can connect to virtually any API using HTTPCaller. HTTPCaller authenticates using a web connection to make HTTP GET requests – though be warned that you may have to dig into the documentation of the API you intend to interact with.

Pushing data to an API

A key benefit of APIs is their ability to securely update information, though it’s important to note that the information you can update will depend on the platform you’re connecting to and the permissions you have been granted in your web connection.

As with pulling data, FME can push or write data to hundreds of popular platforms, including:

And again, HTTPCaller can offer FME Server the same extended functionality, allowing it to modify data in virtually any API, authenticating using a web connection and making HTTP POST, PUT and DELETE requests.

Receiving data from a webhook

The process of receiving data from a webhook subscription is made simple with the help of FME Server.

FME Server creates URLs that can be registered with a third-party app as a webhook subscription. This third-party app then sends messages to FME Server when an event occurs, and this event can be further processed by FME data processes.

While FME can be extended to process virtually any type of webhook payload, the most common are JSON messages. Because webhooks can send information at any time, your middleware must always be on, ready to receive them. This makes FME Server a great choice as it can be hosted on your own infrastructure or FME Cloud. The image below outlines the basic process.

 

 

Building APIs using FME Server

The ability of FME Server to receive data in response to web events means that it can be extended to build out full low-code/no-code APIs. Data flows (called workspaces) can be published to FME Server and exposed as a webhook which listens for web requests and responds accordingly.

FME offers an effective way to build APIs if:

• Agility is key and you’re looking for a short-term solution, such as an API that only needs to last the lifetime of a project.
• You’re prototyping a new service.
• The API is for a service that will only be used within your organisation.

On the flipside, FME may not be the right tool to create APIs if:

• The API is large, complex or involves third parties.
• You’re expecting a large user base and/or millions of requests.
• You wish to create a slick and aesthetically pleasing developer experience using a tool like Swagger (although you can easily use AWS API Gateway as a storefront if necessary).

In our experience, FME Server is more than adequate for a wealth of API-based solutions, even those that may at first seem beyond its capabilities. We’ve used FME Server on projects for organisations as big as New Zealand-scale local governments and utility companies.

Conclusion

The FME Platform provides an ideal solution for organisations to integrate data between systems and automate workflows, saving time, reducing risk and increasing data quality.