Webhooks

The World Wide Web has evolved from its early beginning as a simple network of computers to an entire ecosystem of devices, servers, and more. Among this chaos of endpoints, a consistent and reliable communication model is essential.

Every time your computer goes online, it requests web pages from relevant servers. Similarly, applications rely on other services to keep running. Every time an application goes online, it connects to other servers that provide various services through an application programming interface (API).

APIs themselves are quite simple. A client requests resources from a server, then the server processes the request and sends back data. As long as the client keeps sending requests, the server will keep sending back data, and the client will keep consuming the information.

But how does one application know that another application has new or updated data that it needs to consume? What if the data requested by the client is not ready yet? And what if the application needs real-time data? Should the client keep sending requests?

What is a Webhook?

Going back to the question above, should the client continuously send requests to the server until the desired data is available?

Sending requests to the server requires resources on the client-side. Imagine your banking app or social media app on your phone keeps running in order to have the most recent information. This would drain your device of resources for other functions.

Webhooks resolve this issue by alerting the client whenever data is ready for consumption, a change has been made, or an event has occurred that warrants an exchange of data. Instead of the client application constantly polling the server for new or updated data, imagine a system of push notifications that keeps the client updated.

Webhook vs API

Before taking a look at the nitty-gritty of webhooks, let’s clarify the difference between a webhook and API.

Traditionally, the client application takes an active role in polling the server when it needs resources. The client sends a request to the server through its API, which it then processes. The server returns the request with the relevant data or with a notification that the requested resources are not available. In summary, the client pulls data from the server.

On the other hand, webhooks allow the server to broadcast events to the client. Along with the broadcast, the server can include the data required by the client for faster consumption. Through this system, the client is alerted right away without wasting resources for constant polling or requests. In simpler words, the server pushes data to the client.

In effect, a webhook is an API that is “pushes” instead of “pulling” data. Think of APIs and webhooks as siblings, which is why webhooks are sometimes referred to as “reverse API.”

How Does a Webhook Work?

To understand how webhooks work, let us walk through the steps of setting up a webhook between a client and a server. The technical details are left out for now for clarity.

• A URL is created on the client-side (application). It should point to any typical web page that is ready to receive and process an HTTP request (often a POST request). The page should be able to handle an HTTP POST request in JSON, XML, form-data or any other format from the server.
• Provide the URL to the server that will be sending the POST request. The server is also known as a webhook provider.
• A relevant event or action happens on the server.
• The server makes an HTTP POST request to the URL provided by the client. The POST request usually includes the notification of the event on the server and related data known as the payload.
• The client-side will send an HTTP POST back to the webhook provider notifying receipt of the payload (or details of any issues). The client then processes and consumes the data from the payload.

Much like any API, webhook providers have documentation that will guide you through the set-up of webhooks. The server’s documentation should outline the events and actions broadcasted by their service including the POST request format, payload data, and more.

Why Use a Webhook?

As a consumer, you have probably used webhooks without realizing it. But, if you decide to develop your own application, you should know when to use webhooks. After all, API’s can do what webhooks can, albeit in a brute force fashion. Why learn another communication model between your application and various servers?

As already mentioned, sending API requests need resources from both the client-side and server-side. Continuously polling a service (another name for a server) is a significant waste of resources.

Additionally, webhooks also provide an efficient solution for long-running actions such as data conversion or media transcoding. Now, there is no need to maintain an open connection between the client and the server while waiting for the job to finish. The server can broadcast a notification to the client once it is done processing, and the data required by the client is ready.

Lastly, API’s are not inherently built for real-time information. Let’s say your application polls a server every minute for the latest information. What if a new data becomes available between your last API request and your next one? Even a 30-second delay is too long. Through webhooks, the client is alerted without losing a second, which prevents information lag.

Knowing when to use webhooks and API’s ensures that your application is as efficient as it can be.

Quick Example of Setting Up a Webhook

Let’s now look at a simple demonstration of a webhook in action. Client-side processes are not relevant for now as we will focus on the set-up and communication between the client and the server.

We will use https://webhook.site/ as our client, which will catch the HTTP POST requests made by the server on the webhook. Also, we will use a Github repository as a webhook provider in this demo.

Visit https://webhook.site/ which will automatically create an instance with a unique URL that will be provided to the webhook provider.

Visit Github’s Webhook documentation to determine the events broadcasted and the payloads. Webhooks are associated with each repository and are set up under its Settings.

Under the repository’s Settings, go to Webhooks. Then, click “Add webhook”. Replace the entry under Payload URL with the unique URL from https://webhook.site/. Choose “Send me everything” under the event trigger. Note that the Secret is a security mechanism to ensure that the payload from the POST request is from Github. For this demo, we will not be using Secrets (read more here).

Adding the webhook will trigger an event which you can examine at the client site. Tip: Check Format JSON to easily examine the payload.

Actions such as committing changes to the README of the repository should trigger a broadcast across the webhook.

The client receives the POST request including the payload.

In this specific demo, the payload includes the commit made and the relevant data.

Real-life Webhook Examples

Many applications are already using webhooks. At least one of the services that you are using every day is running on webhooks. Here is a real-life service in the wild with webhooks under their hoods.

Stripe is a popular online payment processing service used by most e-commerce businesses. If you own an online store, you are probably aware of how Stripe updates payments and other processes from their servers to your e-commerce portal. As a consumer, paying for anything online using your credit card through Stripe triggers several webhooks.

In a typical online store, the customer is redirected to the Stripe portal for payment. Stripe processes the payment using the customer’s credit card information. Then, Stripe sends a secure HTTP request through a webhook to the online store’s server. The request contains a payload with the customer’s details along with the payment information. Then, the online store processes the broadcast from Stripe to update the customer’s information and transactions in the store database.

Moving Forward with Webhooks

Webhooks are part of an application’s integration to various servers and services. Hence, the introductory topics described above are just the tip of the iceberg. Since using webhooks involve communication over the Internet, you should be familiar with HTTP request methods and how they work.

Also, security should be your top priority when dealing with API’s and webhooks. Considering exploring HTTPS, passing secrets or tokens, or processing signed requests. Lastly, you should be able to test and debug webhooks during development to ensure that your application is receiving the correct data.