If you've been building websites sites for a while you will realise that no site lives in isolation. Almost every site you build integrates with some form of API, and this is especially the case for the more enterprise sites where data is often synchronised back to a CRM system or similar. Drupal's hook and service architecture means that you can easily build integration points to that API to pull in data.
Pulling in data from an API into a Drupal site means installing an off the shelf module or creating a custom module to provide the integration. What route you go for depends on the integration, but for enterprise sites the API is quite often very custom to the business. I have even seen APIs being built at the same time as the site that it needs to integrate with, which is especially the case for startups and new businesses.
One of the biggest issues in getting things working with a site that relies heavily on an API is testing. Both in terms of behavioural testing and user testing you need to be able to have a repeatable set of items that you can use and having an active changing API makes this a little bit tricky. If your API is still being built then you literally have no API to integrate against and so you can't build your site yet.
This is where stubbing comes in. Instead of directly asking the API for data we can swap out the API integration for a different data source. In other words we redirect the API calls to a local file or database table so that the functionality still exists but the data source is different. This is possible to do in Drupal thanks to the modular and extensible way in which services are used.
By stubbing we also get a repeatable set of data that we can use to run tests. If special situations are found that cause issues on the site they can be added to the stub dataset and tests can be written to check those situations. This ultimately makes for a more solid website that can handle edge cases in the API data.
Making An API
In order for this process to work correctly your API must be integrated into your site through a Drupal service class. This means that your API is controlled through one or more classes that are the gateway to the data in the API. This is best practice as you would otherwise tightly couple your API into your system, meaning that it would be difficult to create a stub. The API doesn't need to be directly built into the Drupal classes, it can easily be abstracted out into a composer package, but you still need a point of entry into your system that will allow Drupal to use it.
To demonstrate how to do stubbing I decided to create a simple example module that shows an API in action. I didn't think it was a good idea to create a large module with a complex API integration for a single article so I went looking for a small scale API. To this end I decided to create a simple integration with the free to use Joke API. This API is a RESTful service that returns a random joke from some given parameters and is perfect to show this technique in action.
I didn't want to add lots of code to this article so the integration with the Joke API is about as simple as possible. We first need to define a service class called JokeAPI that will contain the integration with the API. As the API is a RESTful service we need to use the GuzzleHttp\Client package that is available in Drupal, so this needs to be injected into the class as a dependency.
Assuming the module is called joke_api, then we create a file called joke_api.services.yml to register the JokeAPI as a service. This contains the following.
services:
joke_api.joke:
class: Drupal\joke_api\JokeApi
arguments: ['@http_client']
The class JokeApi has the necessary dependency injection code and a single method called getJoke() that will pull a joke from the API. This method takes a couple of parameters that govern what sort of joke is returned from the API.
<?php
namespace Drupal\joke_api;
use Drupal\Component\Utility\UrlHelper;
use GuzzleHttp\Client;
use Symfony\Component\HttpFoundation\Response;
class JokeApi implements JokeApiInterface {
/**
* The URL of the API.
*
* @var string
*/
protected $url = 'https://v2.jokeapi.dev/joke/';
/**
* Guzzle\Client instance.
*
* @var \GuzzleHttp\Client
*/
protected $httpClient;
/**
* JokeApi constructor.
*
* @param Client $http_client
* The http client.
*/
public function __construct(Client $http_client) {
$this->httpClient = $http_client;
}
/**
* {@inheritdoc}
*/
public function getJoke($options = [], $category = 'any') {
$url = $this->url . $category;
array_filter($options);
if (!empty($options)) {
$url .= '?' . UrlHelper::buildQuery($options);
}
$request = $this->httpClient->request('GET', $url);
if ($request->getStatusCode() != Response::HTTP_OK) {
return FALSE;
}
$data = json_decode($request->getBody()->getContents());
return $data;
}
}
The JokeApi class itself extends an interface called JokeApiInterface, which just contains the getJoke() method. Creating an interface here follows SOLID principles and is very important for the stubbing process later on as we will be creating a new service that will mimic the behaviour of this class.
The parameters to the getJoke() method are separated like this as the URL for the RESTful API always consists of the category, followed by an optional list of query parameters. We just build up the URL and use the GuzzleHttp\Client object to make the request to the API. If the request is successful then the method decodes the JSON we receive back and returns this. If something went wrong the method returns false. By the way, returning "false" from methods that would otherwise return an object isn't best practice as it changes the return type of the method. It would be better to throw an exception here, but it would mean adding more code so I opted for this simpler version here.
There is a second API endpoint that allows us to submit a joke to the API, but I won't be integrating with that in this code.
A service class that interacts with the API isn't that interesting if we can't see the output. So let's create a form that will allow us to pull jokes from the joke API service using this service class. The first thing that we need is to add a route for the form so we create a file called joke_api.routing.yml that contains a rule to point the path '/get-joke' to a form class called GetJokeForm.
get_joke:
path: '/get-joke'
defaults:
_form: '\Drupal\joke_api\Form\GetJokeForm'
_title: 'Get Joke'
requirements:
_access: 'TRUE'
In Drupal, forms are part of the ContainerInjectionInterface interface, and so we use that to inject the JokeApi service into the form itself. Note that the form constructor accepts an object of the type JokeApiInterface and as such allows us to pass in any object that implements this interface. This is part of following SOLID principles and is critical in later allowing us to swap out the JokeApi service for our own stubbed service.
The form consists of a submit button and a single text field that allows us to search for a joke. The joke returned from the API will be printed out above the form. I have tried to keep the code as short as possible, but there is always a little bit of boiler place code in Drupal forms.
<?php
namespace Drupal\joke_api\Form;
use Drupal\Core\Form\FormBase;
use Drupal\Core\Form\FormStateInterface;
use Drupal\joke_api\JokeApiInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;
class GetJokeForm extends FormBase {
/**
* {@inheritdoc}
*/
public function getFormId() {
return 'get_joke';
}
/**
* JokeApi service.
*
* @var \Drupal\joke_api\JokeApiInterface
*/
protected $jokeApi;
/**
* GetJokeForm constructor.
*
* @param \Drupal\joke_api\JokeApiInterface $joke_api
* The get joke API service.
*/
public function __construct(JokeApiInterface $joke_api) {
$this->jokeApi = $joke_api;
}
/**
* {@inheritdoc}
*/
public static function create(ContainerInterface $container) {
return new static(
$container->get('joke_api.joke')
);
}
/**
* {@inheritdoc}
*/
public function buildForm(array $form, FormStateInterface $form_state) {
$output = $form_state->getValue('joke');
if ($output) {
$form['joke'] = [
'#markup' => '<p>' . $output . '</p>',
];
}
$form['contains'] = [
'#type' => 'textfield',
'#title' => 'Contains',
];
$form['submit'] = [
'#type' => 'submit',
'#value' => 'Get Joke',
];
return $form;
}
/**
* {@inheritdoc}
*/
public function submitForm(array &$form, FormStateInterface $form_state) {
$options = [
'contains' => $form_state->getValue('contains'),
];
$joke = $this->jokeApi->getJoke($options);
if ($joke === FALSE || $joke->error == 'true') {
$jokeString = 'Could not get joke.';
}
elseif ($joke->type == 'single') {
$jokeString = $joke->joke;
}
elseif ($joke->type == 'twopart') {
$jokeString = $joke->setup . '<br>' . $joke->delivery;
}
$form_state->setValue('joke', $jokeString);
$form_state->setRebuild(TRUE);
}
}
The return from the API can either be a 'single' joke, or a 'twopart' joke, each of which have slightly different data structures. Depending on what it is we build up a variable called $jokeString and pass that back into the Drupal form storage. It could also be an error returned from the API service, so we account for that as well.
There are a number of other options available to filter the Joke API in different ways, but as a simple integration this works and allows some filtering to happen to show the API in action.
When we visit the page at /get-joke we see the form printed out to the screen.
If we click the "Get Joke" button the page is refreshed and we see a joke printed out above the form elements.
This is clearly a joke. If we click the button again we get more jokes returned from the API. Be warned that some of the jokes from this API are NSFW, but you can add filters to remove that stuff from the results if you need to.
Stubbing The API
Now that we have this API integration with the Joke API we can set about creating a stub module that will mimic the API without actually pulling data from it.
In order to make the stub a plug and play solution a second module is created so that we can activate the stub by just enabling a module. Best practice is to keep the stub module within the parent module it is overriding and simply add the suffix "stub" to the module name. This will create a directory structure like this.
/joke_api
- /modules
- - /joke_api_stub
- - - joke_api_stub.info.yml
- /src
joke_api.info.yml
The stub module will override the API using a built in feature of Drupal that allows us to override service providers. This feature gives us the ability to alter things about any service defined in Drupal, but in this case we will be overriding the PHP class that gets used. To do this we need to create a PHP class with a special name by converting the module name to camel case and adding "ServiceProvider" to the end.
joke_api_stub -> JokeApiStubServiceProvider
This JokeApiStubServiceProvider class needs to extend the ServiceProviderBase class. This is an abstract class that we can either use to alter or register a service using methods that are automatically detected and called by Drupal. To alter the service we just need to implement the alter() method in our new class and use the parameter sent to the method to find and alter the class used in the joke_api.joke service.
<?php
namespace Drupal\joke_api_stub;
use Drupal\Core\DependencyInjection\ContainerBuilder;
use Drupal\Core\DependencyInjection\ServiceProviderBase;
/**
* Class JokeApiStubServiceProvider.
*
* @package Drupal\joke_api_stub
*/
class JokeApiStubServiceProvider extends ServiceProviderBase {
/**
* {@inheritdoc}
*/
public function alter(ContainerBuilder $container) {
$definition = $container->getDefinition('joke_api.joke');
$definition->setClass('Drupal\joke_api_stub\JokeApiStub');
}
}
By placing this specially named class in the modules src directory Drupal will automatically find it and run any methods we have created. There is no need to add additional service definitions for this.
We have now overridden the joke_api.joke service with a different class. The next step is to extend the JokeApi class so that we get all of the same injected service as the original class. We can then override methods in the class for our own needs. Also, as we told the GetJokeForm to accept an interface instead of a class name it means that we don't need to change anything to get this to work. The new JokeApiStub class will still implement the JokeApiInterface and as such can be passed without any problems to anything we built in the original module.
The simplest implementation of the stub class in this situation is to return a single joke, so that's what I have done here. The important part is that the return of the getJoke() method must return the same kind of data that our original method returned so this takes a JSON string from the API and runs it through json_decode() just like the original method did.
<?php
namespace Drupal\joke_api_stub;
use GuzzleHttp\Client;
use Drupal\joke_api\JokeApi;
class JokeApiStub extends JokeApi {
/**
* {@inheritdoc}
*/
public function getJoke($options = [], $category = 'any') {
$data = '{
"error": false,
"category": "Programming",
"type": "twopart",
"setup": "A web developer walks into a restaurant.",
"delivery": "He immediately leaves in disgust as the restaurant was laid out in tables.",
"flags": {
"nsfw": false,
"religious": false,
"political": false,
"racist": false,
"sexist": false,
"explicit": false
},
"id": 6,
"safe": true,
"lang": "en"
}';
return json_decode($data);
}
}
All you need to do to activate this stub is turn the module on. With the module active, when we use the form again we get the same joke over and over again. This is because we are now pulling data from the stubbed class instead of from the actual API itself.
If you want to give this code a go then I have created a Github repo that contains the entire JokeApi and stub modules. Feel free to give this a go and see how it works.
Conclusion
The code here demonstrates the very simplest version of stubbing data. We could store this data in a database table or CSV files and write some code to return that data. Using more complex data storage like database tables or CSV is good because it allows us to create situations that can react to input, which is important for any API. For example, we could return different jokes depending on what sort of search query or filter was performed.
Taking this a step further we can create full data sets where we return things like user account details or event information from this stubbed service. It's possible to create fleshed out interactions by just using stubbed data. It's probably a good idea to use CSV as this will allow the stub data to live with the code and allow different code to be tested by just checking out that branch.
The usefulness of this technique speaks for itself. It can be easily activated and adds lots of possibilities to your development workflow. By also combining this with configuration split you can automatically activate the stub module locally so that it always provides that consistent interface.
In addition to testing there is also the benefit of convenience. By making the API essentially self contained with the site it means we don't need to access the API when building things. If the API being integrated with is behind a VPN or whitelisted IP address setup then there's a good chance that not all of your team will have access. By creating a stub module you instantly give the entire team the ability to work on the site without having access to the API. This really helps for building out front end components that integrate with the API as your theme builder will not need access to the API at all.
You can also turn the stub module around and prevent your Drupal site leaking testing data to a third party API. For example, if you have an analytics module that relies on user interaction you can swap out that module for something that records the user actions to a log, rather than sending data to the API. This allows you to run tests and ensure that your event setup works, without sending a bunch of test data to the analytics provider. This is especially useful if you are integrating with an analytics provider that has a limited stage environment.
The only downside is that it does take a little bit of time to get this setup and working and you have to keep it up to date. You must keep the stub module up to date with any changes to the API otherwise it's less than useless.
I have used this technique on a variety of different Drupal sites and it has always improved the live of the developers working on these projects.
Add new comment