End-to-End (E2E) Testing

Last Updated: January 8, 2026

This document outlines the setup, approach, and best practices for our end-to-end testing suite.

Overview

We use Playwright to run our E2E tests. These tests simulate real user interactions in a browser environment to verify that our application's critical flows work as expected from start to finish. The goal is to catch regressions and ensure a high-quality user experience.

All E2E test-related files are located in the app/__e2es__/ directory.

Core Concepts & Approach

Our E2E testing strategy is built on a few key principles: speed, reliability, and isolation.

1. Test Seeding via API Endpoints

Instead of creating test data through the UI (which can be slow and brittle), we use dedicated API endpoints for test setup.

Location: These endpoints are located in app/pages/api/dev/.

Function: Each endpoint is responsible for creating a specific, isolated state for a test scenario. This includes creating users, workspaces, workflows, sheets, and any other necessary database records.

Execution: At the beginning of a test, Playwright navigates to one of these endpoints (e.g., await page.goto('http://localhost:3000/api/dev/simple-workflow')). The endpoint script runs, seeds the database, and then redirects the browser to the starting page for the test.

Safety: These endpoints are guarded and will only run in development, test, or ci environments to prevent them from being used in production.

This approach makes our tests:

• Faster: Bypasses time-consuming UI interactions for setup
• More Reliable: Less prone to failures from unrelated UI changes
• Easier to Debug: The starting state for each test is clearly defined in a single file

Most endpoints use the createTestScenario service (see below) to declaratively build test scenarios. Some endpoints still use manual database seeding for specific edge cases.

Examples:

• pages/api/dev/simple-workflow.ts sets up a basic workflow with various input block types for testing
• pages/api/dev/session-with-rules.ts sets up a session with conditional logic (rules) for rules.spec.ts
• pages/api/dev/session-sheet-section.ts sets up a workflow with sheet sections for testing sheet interactions
• pages/api/dev/workflow-reorder.ts uses manual seeding for a workflow with multiple sections to test reordering functionality

2. Authentication Flow

We handle authentication efficiently to avoid logging in for every single test file.

1. Setup (auth.setup.ts): A global setup test runs first. It signs up a new, unique user for the test run and saves the authenticated session state (cookies, local storage) to a file (playwright/.auth/user.json).

2. Test Execution: All subsequent test suites are configured in playwright.config.ts to depend on the setup project. They load the saved authentication state, so they start each test as an already logged-in user.

3. Teardown (global.teardown.ts): After all tests have run, a teardown script executes. It navigates to the user's profile page and deletes the account, ensuring a clean state for the next run.

3. Test Structure

• Tests are located in app/__e2es__/
• Each .spec.ts file corresponds to a major feature or user flow (e.g., workspace.spec.ts, workflow.spec.ts)
• Helper functions and shared context (like generating unique user emails) are in spec.helpers.ts

Running Tests

Locally

The test runner is configured to start the development server automatically.

1. Ensure you have a local .env.test file with the necessary environment variables.

2. Run the tests from the app directory:

   npx playwright test

3. To run a specific test file:

   npx playwright test __e2es__/workspace.spec.ts

4. To open the UI mode for debugging:

   npx playwright test --ui

5. View the HTML report after a run:

   npx playwright show-report

On CI/CD

Tests are automatically executed on every push and pull request to the main and staging branches via GitHub Actions.

• Workflow File: .github/workflows/playwright.yml
• Process: The CI job installs dependencies, runs database migrations, builds the application, and then runs the full Playwright suite.
• Reports: If tests run, the HTML report is uploaded as a build artifact named playwright-report, which can be downloaded for inspection.

Configuration

The main configuration for the test suite is in playwright.config.ts. Key aspects include:

• testDir: Points to __e2es__
• webServer: Defines the commands to start the Next.js application and other services (like Inngest) before tests run
• projects: This is where we define our test suites and their dependencies. The configuration ensures the setup project runs first, followed by all feature tests, and finally the teardown project. This orchestrated flow is critical for our authentication strategy.

Writing a New Test

Follow these steps to add a new E2E test:

1. Create the Spec File

Create a new file in app/__e2es__/, for example my-new-feature.spec.ts.

2. Create a Seeding Endpoint (if needed)

If your test requires a specific database state, create a new endpoint in app/pages/api/dev/, for example my-new-feature-setup.ts.

Recommended: Use the createTestScenario service (see below) to declaratively build your test scenario. This ensures consistency and reuses production services.

Alternative: For edge cases, you can use manual database seeding with knex and our services directly.

End the script with a redirect to the page where your test should begin:

res.redirect(303, '/workspaces/...')
// or
res.redirect(303, '/sessions/workflows/...')

3. Write the Test

In my-new-feature.spec.ts, start your test by navigating to your new setup endpoint:

import { expect, test } from '@playwright/test'

test('should do something amazing', async ({ page }) => {
  // 1. Set up state and navigate to the start page
  await page.goto('http://localhost:3000/api/dev/my-new-feature-setup')

  // 2. Write your test logic and assertions
  await expect(page.getByText('My New Feature')).toBeVisible()
  await page.getByRole('button', { name: 'Click me' }).click()

  // ... more assertions
})

4. Update Playwright Config

Open playwright.config.ts and add a new project definition for your test suite. Make sure to add the dependencies: ['setup'] property.

// In playwright.config.ts projects array
{
  name: 'my new feature',
  testMatch: 'my-new-feature.spec.ts',
  dependencies: ['setup'],
  use: {
    ...devices['Desktop Chrome'],
    storageState: 'playwright/.auth/user.json'
  }
},

Current Test Coverage

Our E2E suite currently covers the following core features:

The createTestScenario Service

Most of our E2E test seeding endpoints use the createTestScenario service (app/services/scenarios/createTestScenario.ts) to declaratively build test scenarios. This service provides a functional factory pattern that orchestrates calls to our existing business logic services (e.g., createWorkflow, createGroup, createSheet) to build complex test scenarios.

How It Works

The createTestScenario function accepts a single declarative configuration object that describes the entire desired state for a test (workflows, pages, sections, blocks, roles, sheets, etc.). It then:

1. Creates entities in the correct order - Groups → Workflows → Sheets → Columns → Pages → Sections → Blocks → Roles → Assignments
2. Resolves relationships - Maps names (like page titles, role names, column names) to database IDs automatically
3. Reuses production services - Calls createGroup, createWorkflow, createWorkflowRun, createSheet, etc., ensuring test data is created using the same logic as production
4. Handles defaults intelligently - createWorkflow automatically creates a default role, first page, and first section; the factory updates these when the config specifies different values

Example Usage

Here's a real example from pages/api/dev/session-with-rules.ts:

import { createTestScenario } from '../../../services/scenarios/createTestScenario'

const scenario = await createTestScenario({
  user: { id: user.id },
  workflow: {
    name: 'Test workflow with rules',
    scaffold_sheet: true,
    pages: [
      {
        title: 'First stage',
        is_starting: true,
        is_end: false,
        sections: [
          {
            blocks: [
              {
                type: 'string',
                label: 'Name',
                column: 'Name' // Column name resolved to column_id automatically
              },
              {
                type: 'checkbox',
                label: 'I do declare',
                column: 'Declaration'
              }
            ]
          },
          {
            name: 'HIDDEN SECTION',
            rule: {
              combinator: 'and',
              rules: [
                {
                  field: 'Name', // Column name resolved to column ID
                  operator: '=',
                  value: 'greg'
                }
              ]
            },
            blocks: [{ type: 'content', content: '' }]
          }
        ]
      },
      {
        title: 'Second stage',
        is_end: true
      }
    ],
    stageRoleAssignments: [
      {
        role: 'Guest', // Role name resolved to role ID
        page: 'Second stage' // Page title resolved to page ID
      }
    ],
    transitions: [
      {
        from: 'First stage',
        to: 'Second stage',
        rule: {
          combinator: 'and',
          rules: [
            {
              field: 'Declaration', // Column name resolved to column ID
              operator: '=',
              value: true
            }
          ]
        }
      }
    ]
  },
  sheets: [
    {
      name: 'Scaffold Sheet',
      is_scaffold: true,
      columns: [
        { name: 'Name', type: 'text', order: 1 },
        { name: 'Declaration', type: 'check', order: 2 }
      ]
    }
  ],
  workflowRun: {
    name: 'Test session with rules',
    preview: false
  },
  sessionRoleAssignments: [
    {
      role: 'Guest',
      assignee: user.id
    }
  ]
})

// Redirect to the created session
res.redirect(303, `/sessions/workflows/${scenario.workflowRun.id}`)

Key Features

• Declarative Configuration: Describe what you want, not how to create it
• Automatic ID Resolution: Reference entities by name, not by ID
• Production Service Reuse: Test data is created using the same logic as production
• Intelligent Defaults: Automatically handles default entities created by services

Configuration Options

The createTestScenario function accepts a configuration object with the following structure:

{
  user: { id: string },
  workflow: {
    name: string,
    scaffold_sheet?: boolean,
    pages: [...],
    transitions: [...],
    stageRoleAssignments: [...]
  },
  sheets: [...],
  workflowRun?: {
    name: string,
    preview?: boolean
  },
  sessionRoleAssignments?: [...]
}

Benefits

1. Consistency: All tests use the same pattern for setup
2. Maintainability: Changes to services automatically propagate to tests
3. Readability: Test setup is declarative and easy to understand
4. Speed: Faster than UI-based setup
5. Reliability: Less brittle than UI-based setup

Leveraging Workflow Import for Complex Test Scenarios

For particularly complex test scenarios (e.g., testing a workflow with dozens of blocks, intricate conditional logic, or multiple sheets), you can leverage the Workflow Import feature to seed test data.

The Use Case

• You need to test a feature that requires a complex, multi-page workflow with many blocks, rules, and relationships
• Manually building this scenario with createTestScenario or database seeding would be time-consuming and error-prone
• You already have a reference workflow in a development or staging environment that represents the scenario you want to test

How It Works

1. Export the Workflow: Use the workflow export feature to download a JSON file of your reference workflow from a development environment
2. Store the Export: Place the JSON file in a known location (e.g., app/__e2es__/fixtures/complex-workflow.json)
3. Import in Test Setup: In your test seeding endpoint, use the workflow import service to create the workflow from the JSON file
4. Customize as Needed: After import, you can still modify the workflow or create additional test data as needed

Integration Options

Option 1: Import in Seeding Endpoint

// pages/api/dev/complex-workflow-setup.ts
import { importWorkflow } from '../../../services/workflowImport'
import complexWorkflowData from '../../../__e2es__/fixtures/complex-workflow.json'

const importedWorkflow = await importWorkflow({
  data: complexWorkflowData,
  groupId: group.id,
  userId: user.id
})

// Create a workflow run and redirect
const workflowRun = await createWorkflowRun({
  workflowId: importedWorkflow.id,
  userId: user.id
})

res.redirect(303, `/sessions/workflows/${workflowRun.id}`)

Option 2: Import via UI in Test

// __e2es__/complex-workflow.spec.ts
test('should handle complex workflow', async ({ page }) => {
  // Navigate to workflows page
  await page.goto('http://localhost:3000/workspaces/...')
  
  // Use the import UI
  await page.getByRole('button', { name: 'Import' }).click()
  await page.setInputFiles('input[type="file"]', 'fixtures/complex-workflow.json')
  
  // Continue with test...
})

Benefits

1. Rapid Setup: Quickly set up complex scenarios without manual configuration
2. Realistic Data: Test with workflows that mirror production complexity
3. Maintainability: Update the fixture file when the reference workflow changes
4. Reusability: Share fixture files across multiple tests

Considerations

• Fixture Maintenance: Keep fixture files up to date with schema changes
• Test Isolation: Ensure imported workflows don't create unintended dependencies between tests
• Readability: Document what the fixture represents and why it's needed

Best Practices

When Writing Tests

1. Use descriptive test names - Clearly explain what is being tested
2. Keep tests focused - Each test should verify one specific behavior
3. Use page object patterns - Extract common interactions into reusable functions
4. Wait for elements - Use Playwright's built-in waiting mechanisms
5. Avoid hard-coded waits - Use waitForSelector instead of wait(5000)
6. Clean up after yourself - Tests should not leave side effects

When Using createTestScenario

1. Use meaningful names - Page titles, role names, and column names should be descriptive
2. Keep scenarios minimal - Only create what's needed for the test
3. Document complex scenarios - Add comments explaining why certain configuration is needed
4. Reuse fixtures - Share common scenarios across tests when appropriate

Related Documentation