An essential part of any successful Drupal site is an automated testing process. As you apply updates or changes to the site then making sure that existing functionality works in the same way prevents errors from creeping in. Having the testing suite run in an automated fashion at some point before the code is deployed is a really good idea as it stops those errors reaching any servers.
Drupal has it's own internal testing framework, which is built upon PHPUnit and can be used to test Drupal from the ground up. Unit tests in Drupal start with a blank site, so to test a page you need to first install the correct modules and create the content types, fields, taxonomy terms, blocks, and users that are required for the page. Only once you have everything in place you can add the content and create the page.
Whilst it is possible to test individual parts Drupal very thoroughly using this system, it is often best to also test Drupal as a complete website after it has been installed and fully configured. This is where Cypress comes in.
Cypress is a JavaScript based testing framework that can run tests against an installed Drupal site using a web browser. The key part of this is that you use the Drupal configuration to install a pre-configured site, add a page of content, and test that the correct components are on the page. Often, the step of creating the content is a test in itself and can be a useful part of the test process.
Any Drupal site should therefore have unit tests to test any custom code inside the site, and Cypress tests to test the site as a whole. The complexity of the Drupal site doesn't matter all that much here as any site will benefit from a few Cypress tests to check important functionality.
With the tests in place we can then get developers to run the unit tests before merging code.
If we have two testing frameworks then we don't want to rely on our developers to run the full testing suite every time they want to change anything. We should instead have an automated way of running those tests, which is possible using a continuous integration system like the Bitbucket Pipelines.
Bitbucket Pipelines
Pipelines in Bitbucket are a way of automating certain tasks. These can be simple tasks like synchronising your code with another repo, or more complex tasks like running a full unit testing suite.
Pipelines are defined using yml, in a file called bitbucket-pipelines.yml. A very basic pipeline structure might be the following:
pipelines:
pull-requests:
'*':
- step:
# perform a number of actions when a pull request is created
branches:
main:
- step:
# perform a number of actions when code is merged to main Pipelines make use of Docker containers as well, so you can either make use of a number of containers provided by Bitbucket, or create your own and supply those as part of the pipeline run.
Run Cypress On Bitbucket
In order to run Cypress tests against a Drupal site we first need to install (and run) Drupal. This means that we need to have PHP and Node installed, as well as a database image that we can use to run the site on. There are quite a few steps involved in getting to the point where we can run the Cypress tests.
To get Cypress tests running for your pull requests place a bitbucket-pipelines.yml file in the root of your repo and Bitbucket will automatically pick it up and start using it. The best approach I have found is to add this testing step to all pull requests, and then configuring Bitbucket to require a passing build before allowing the pull request to be merged.
To set up the pipeline so that it triggers on a pull request we need to add this section to the bitbucket-pipelines.yml file.
pipelines:
pull-requests:
'*':
- step:
image: php:8.3
name: Cypress Tests
fail-fast: true
script:
# build steps go hereThis will trigger if any pull request is made and run the steps in the "script" section and include the Bitbucket PHP docker container with PHP 8.3 installed.
Whilst we can create our own docker images and supply them to the pipeline build process it is simpler and easier to use the off the shelf containers from Bitbucket. That way, when we need to update the version number we can do so without having to re-build the container. This also means that I can share the code here without pointing you towards docker images that you need to battle with.
Of course, rolling your own docker images (if you are happy with that) will speed up the setup processes of your testing pipelines. You can remove the first 3-4 minutes of time taken for pipelines to run by removing all of this setup stuff at the start.
The fail-fast clause just means that if something throws an error the pipeline should fail as soon as it can. This basically means that if the setup of the environment wasn't successful then the Cypress tests aren't run against a broken Drupal site that we know isn't working.
The process of installing and running Drupal so that we can test it using Cypress isn't that straightforward. Whilst there exists a PHP docker container from Bitbucket, it doesn't have composer or some of the required third party extensions installed. We therefore need to install and configure the relevant software requirements for our Drupal system to run.
The entire first section of the script section is therefore devoted to installing composer, installing the requirements for our Drupal install, and also installing Node (and additional dependencies) so that we can run Cypress.
# Install the needed dependencies for composer and the requirements of the composer file.
- export COMPOSER_ALLOW_SUPERUSER=1
- apt-get update
- apt-get install -y curl unzip libfreetype6-dev libjpeg62-turbo-dev libpng-dev libsodium-dev default-mysql-client
- curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
- docker-php-ext-configure gd
- docker-php-ext-install -j$(nproc) gd
- docker-php-ext-install -j$(nproc) sodium
- docker-php-ext-install -j$(nproc) pdo
- docker-php-ext-install -j$(nproc) mysqli
- docker-php-ext-install -j$(nproc) pdo_mysql
- docker-php-ext-install -j$(nproc) sockets
- echo "memory_limit = 512M" > $PHP_INI_DIR/conf.d/php-memory-limits.ini
# Install the source code.
- composer install --no-interaction --dev
# Copy the default bitbucket settings file into place.
- cp web/sites/default/bitbucket.settings.php web/sites/default/settings.php
# Install Cypress dependencies
- apt-get install -y libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libnss3 libxss1 libasound2 libxtst6 xauth xvfb
# Install node and npm.
- curl -sL https://deb.nodesource.com/setup_18.x -o nodesource_setup.sh | bash -
- apt-get install -y nodejs
- apt-get install -y npm
- node --version
- npm --versionOnce that's done we can install Drupal. This is done using Drush, which is part of the composer packages we have installed in the codebase.
# Install the Drupal site.
- ./vendor/bin/drush si myprofile --existing-config --yes --account-name=admin --account-pass=admin --db-url=mysql://drupal:drupal@127.0.0.1:3306/drupalAs this Drupal site contains no content we then use a module called Default Content Deploy to inject content into the site. This is just a case of running a Drush command after the site has been installed.
# Import default content.
- ./vendor/bin/drush default-content-deploy:import --force-override --yesHaving Default Content Deploy install some content gives us a known site that we can use to test things with.
For example, we might have a site with a content type that has a number of relationships. Rather than getting the Cypress tests to run through all of the steps to create those relationships we instead inject them into the site so that we can test other interactions with those content items. Taxonomy terms are one of those types of content that need to be in place on a lot of Drupal sites for them to function correctly and these are an excellent candidate for Default Content Deploy.
The next step is to use the built in PHP web server to serve the site. Thankfully, we can do this using Drush since it has a built in command that runs the server.
# Run the PHP web server.
- ./vendor/bin/drush -vvv --debug runserver 8080 &We can now get Cypress to go to the page at http://127.0.0.1:8080, where the site will be available. Using the built in PHP web server like this means that we don't have to install and configure Apache or Nginx to get the site served. The PHP web server isn't the best and most reliable web server in the world, but it does the job for the duration of the tests.
The final step in all this is to move into the Cypress directory in the repo (which is kept in tets/cypress in this example) and install Cypress before running the tests. This isn't done in a previous step here as we needed to change directories to do this and separating this step to the end means we don't have multiple "cd" calls dotted throughout the process.
# Install Cypress.
- cd tests/cypress
- npm install
- npx cypress install
# Run Cypress tests.
- npx cypress run --config "baseUrl=http://127.0.0.1:8080"By passing in the --config flag with the baseUrl configuration setting on the command line we are overwriting any other instance of baseUrl being set within the configuration of Cypress. This means that we also don't need to worry about ensuring the hostnames of the site are configured.
In addition to the Drush command setting up the webserver for the Drupal site we also need to have a database for the site. I haven't mentioned this yet, but at the bottom of this step we also inject another docker container that we label as "mysql".
services:
- mysqlThe mysql service is defined at the bottom of the bitbucket-pipelines.yml file and just pulls in the Bitbucket mysql docker container and sets some configuration options.
definitions:
services:
mysql:
image: mysql:latest
variables:
MYSQL_DATABASE: 'drupal'
MYSQL_ROOT_PASSWORD: 'drupal'
MYSQL_USER: 'drupal'
MYSQL_PASSWORD: 'drupal'The database details are held in a settings file called bitbucket.settings.php, that is then copied into place for the site to pickup.
The final thing to add to the build step (under the services section) is the artefacts that should be saved when the build finishes. Just create a section called "artifacts" (note the American spelling) and add the directories in your repo that should be downloadable.
artifacts:
- tests/cypress/cypress/screenshots/**
- tests/cypress/cypress/videos/**Having a test report that you can download is really important as it can be handed out, or attached to the production release ticket to prove that everything is in order prior to the production release.
Putting It All Together
Here is the bitbucket-pipelines.yml file for running Cypress tests on Bitbucket in full. Comments have been added here to show what the differect sections are doing.
pipelines:
pull-requests:
'*':
- step:
image: php:8.3
name: Cypress Tests
fail-fast: true
script:
# Install the needed dependencies for composer and the requirements of the composer file.
- export COMPOSER_ALLOW_SUPERUSER=1
- apt-get update
- apt-get install -y curl unzip libfreetype6-dev libjpeg62-turbo-dev libpng-dev libsodium-dev default-mysql-client
- curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
- docker-php-ext-configure gd
- docker-php-ext-install -j$(nproc) gd
- docker-php-ext-install -j$(nproc) sodium
- docker-php-ext-install -j$(nproc) pdo
- docker-php-ext-install -j$(nproc) mysqli
- docker-php-ext-install -j$(nproc) pdo_mysql
- docker-php-ext-install -j$(nproc) sockets
- echo "memory_limit = 512M" > $PHP_INI_DIR/conf.d/php-memory-limits.ini
# Install the source code.
- composer install --no-interaction --dev
# Copy the default bitbucket settings file into place.
- cp web/sites/default/bitbucket.settings.php web/sites/default/settings.php
# Install Cypress dependencies
- apt-get install -y libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libnss3 libxss1 libasound2 libxtst6 xauth xvfb
# Install node and npm.
- curl -sL https://deb.nodesource.com/setup_18.x -o nodesource_setup.sh | bash -
- apt-get install -y nodejs
- apt-get install -y npm
- node --version
- npm --version
# Install the Drupal site.
- ./vendor/bin/drush si myprofile --existing-config --yes --account-name=admin --account-pass=admin --db-url=mysql://drupal:drupal@127.0.0.1:3306/drupal
# Import default content.
- ./vendor/bin/drush default-content-deploy:import --force-override --yes
# Run the PHP web server.
- ./vendor/bin/drush -vvv --debug runserver 8080 &
# Install Cypress.
- cd tests/cypress
- npm install
- npx cypress install
# Run Cypress tests.
- npx cypress run --config "baseUrl=http://127.0.0.1:8080"
caches:
- composer
- npm
- node
- cypress
services:
- mysql
artifacts:
- tests/cypress/cypress/screenshots/**
- tests/cypress/cypress/videos/**
definitions:
services:
mysql:
image: mysql:latest
variables:
MYSQL_DATABASE: 'drupal'
MYSQL_ROOT_PASSWORD: 'drupal'
MYSQL_USER: 'drupal'
MYSQL_PASSWORD: 'drupal'
caches:
npm: $HOME/.npm
cypress: $HOME/.cache/CypressPlease note that to get this working you need to correctly set up your repository with Cypress and Drupal in the appropriate places.
Once you have everything running here and have your Cypress tests running then you might want to think about splitting up your Cypress tests a little. Having a few tests running on your pull requests is great, but there's an upper limit of about 2 hours for the pipelines to run. Plus, you don't want to have developers waiting for 2 hours for the tests to run for every small change.
Thankfully, the package cypress/grep exists to allow you to configure a subset of the tests to run for every pull request. Picking the most important tests to run on the site is makes for an interesting discussion, but you should pick a few tests that will quickly show if something is broken. On the site that we use this pipeline the content is editor driven and so our focus is on the main pages that editors will use to create content.
It goes without saying that you should be running the entire test suite somewhere in your build process. At the very least this should be once before you release your code to production. In addition to this pull request test we also have a scheduled pipeline that runs the full test suite against the dev branch every night, but we can also opt to run this pipeline whenever we need to. This means that any errors can be spotted quickly.