Cypress Testing and Github Actions

Test, Rinse, Repeat

This was not my first dive into testing, but typically I've inherited the test environment and have been told to “make them green.” For this project I have multiple codebases with similar setups. I'd like to maintain quality across all codebases while working on them more efficiently, so I decided to write my own test suite using Cypress.

The Requirements

1. Environment variables for use across different repos
2. Functional testing in the browser (multilple browsers)
3. Automation of tests
4. Ability to add visual testing

The Setup

# .env

SHOPIFY_URL=********.myshopify.com
THEME_PASSWORD=********
THEME_ID=********
CYPRESS_PROJECT_ID=*******

// cypress.config.js

const { defineConfig } = require("cypress");

require('dotenv').config()

module.exports = defineConfig({
  projectId: process.env.CYPRESS_PROJECT_ID,
  e2e: {
    setupNodeEvents(on, config) {
      // implement node event listeners here
      // we can grab some process environment variables
      // and stick it into config.env before returning the updated config
      config.env = config.env || {}

      // you could extract only specific variables
      // and rename them if necessary
      config.env.SHOPIFY_URL = process.env.SHOPIFY_URL
      config.env.THEME_PASSWORD = process.env.THEME_PASSWORD
      config.env.THEME_ID = process.env.THEME_ID
      console.log('extended config.env with process.env.{SHOPIFY_URL, THEME_PASSWORD, THEME_ID}')

      return config
    },
  },
});

Once you have environment variables set up and passed to Cypress you can access them through Cypress.env('SHOPIFY_URL'). This makes it easier to use the theme password in your beforeEach loops inside your test specs.

// homepage.cy.js

describe('Home Page', () => {
  beforeEach(() => {
    cy.visit(`${Cypress.env('SHOPIFY_URL')}?preview_theme_id=${Cypress.env('THEME_ID')}`);
    cy.get('form[action="/password"]')
      .then(() => {
        cy.get('input[type="password"]').type(`${Cypress.env('THEME_PASSWORD')}{enter}`)
      })
  });

  it('loads the homepage', () => {
    cy.url().should('include', 'myshopify.com')
  });
});

A sample spec file to test the homepage

Automating Tests with Github Actions

I had tests running locally without any issues but I wanted to automate running them when I push to the repository. I had used CircleCI before, but I didn't want to go through the full setup. Since my code is already hosted on Github, I thought Github Actions would be a great candidate.

The documentation on Cypress has some great examples, but it was going to take some frankensteining to get what I needed. Locally, I had environment variables for the Shopify URL, Theme ID, and Theme Password (optional: development stores are password protected.)

Under Settings => Secrets I found a way to pass environment variables to my action.

Adding secrets here allows me to access them using the following yml:

# main.yml

env:
  SHOPIFY_URL: ${{ secrets.SHOPIFY_URL }}
  THEME_PASSWORD: ${{ secrets.THEME_PASSWORD }}
  THEME_ID: ${{ secrets.THEME_ID }}
  CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}

I'll be adding more browsers, functional tests, and visual tests in the future, but the possibilities of this are really exciting. Below is the final main.yml file to automatically run the tests on pushes to the main branch and save the tests to my Cypress dashboard.

# main.yml

name: E2E
on:
  push:
    branches:
      - main
jobs:
  cypress-run:
    runs-on: ubuntu-latest
    steps:
      - run: echo "🎉 The job was automatically triggered by a ${{ github.event_name }} event."

      - name: Check out repository code
        uses: actions/checkout@v2

      - name: run cypress tests
        uses: cypress-io/github-action@v4
        timeout-minutes: 10
        with:
          record: true
          parallel: true
        env:
          SHOPIFY_URL: ${{ secrets.SHOPIFY_URL }}
          THEME_PASSWORD: ${{ secrets.THEME_PASSWORD }}
          THEME_ID: ${{ secrets.THEME_ID }}
          CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}

Debugging

I ran into an odd error with Github actions that took me longer than I'd like to admit to debug. While I assumed that a package-lock.json file should never be edited by a human, in this case it was a requirement. Changing the lockfile to let Github Actions know this code was safe to run on an ubuntu system. It once and for all defeated the error and allowed the tests to run.

# action log file

npm ERR! code EBADPLATFORM
npm ERR! notsup Unsupported platform for fsevents@2.3.2: wanted {"os":"darwin"} (current: {"os":"linux","arch":"x64"})
npm ERR! notsup Valid OS:    darwin
npm ERR! notsup Valid Arch:  undefined
npm ERR! notsup Actual OS:   linux
npm ERR! notsup Actual Arch: x64
Error: The process '/usr/local/bin/npm' failed with exit code 1

// package-lock.json

"node_modules/fsevents": {
  ...
  "os": [
    "darwin",
    "win32",
    "linux"
  ],
  ...
}