Guide

Sitemap

Spotlight is TestParty's production-level accessibility scanner that runs automated tests on your live website or staging environments. It generates a YAML file that integrates with your CI/CD pipeline (GitHub Actions, GitLab CI, BitBucket Pipelines, or Jenkins) to run nightly scans.

Prerequisites

Before setting up Spotlight, you'll need:

  • Admin access to TestParty dashboard

  • Access to your GitHub/GitLab/BitBucket repository

  • List of URLs to scan OR your sitemap.xml file

  • Test account credentials (if scanning pages behind authentication)

  • Repository permissions to add secrets and workflows

Step 1: Access the Continuous Integration Setup

  1. Log into the TestParty dashboard at https://testparty.ai

  2. Navigate to the Continuous Integration tab in the main menu

  3. Click on the Sitemap section

Step 2: Configure Your Root Domain

Basic Setup (Public Pages)

  1. In the "Root Domain" field, enter your base URL

    • Example: https://www.yoursite.com/

    • Important: Include the trailing slash /

  2. Click "Update Root" to save

Setup with Authentication

If you need to scan pages behind a login:

  1. Set your main domain as the root (e.g., https://app.yoursite.com/)

  2. In the "Setup Endpoint" section, add your login page

    • Example: login.aspx or /auth/login

  3. Click "Set Root" on the setup endpoint

Step 3: Add Endpoints to Scan

Method 1: Manual URL Entry

  1. In the "Endpoint Path" field, add each page path you want to scan

    • Example: /products, /checkout, /dashboard

    • Do NOT include the domain or leading slash if your root domain ends with /

  2. Click "Add Endpoint" for each URL

  3. Repeat for all pages you want to scan

Method 2: Using XML Sitemap

If you have a sitemap.xml file:

  1. TestParty can automatically import all URLs from your sitemap

  2. Provide the sitemap URL (e.g., https://yoursite.com/sitemap.xml)

  3. The system will parse and add all URLs automatically

Method 3: Complex User Flows

For single-page applications or complex navigation:

  1. Add your main URL as the endpoint

  2. Click "Define Interaction Steps" next to the endpoint

  3. Add navigation steps (detailed in Step 4)

Step 4: Configure Interaction Steps (For Dynamic Content)

See our section on Interaction Steps for more.

Step 5: Generate the YAML Configuration

  1. After configuring all URLs and steps, click "Generate YAML"

  2. The system creates a GitHub Actions workflow file

  3. The generated YAML structure will look like:

yaml

name: TestParty Reporter
on:
  schedule:
    - cron: '0 10 * * *'  # Runs at 10 AM UTC daily
  workflow_dispatch:      # Allows manual triggering

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: TestParty Reporter
        uses: test-party/reporter@v0.0.1-alpha.5
        with:
          testparty_token: ${{ secrets.TESTPARTY_TOKEN }}
          repository_name: ${{ github.repository }}
          repository_id: ${{ github.repository_id }}
          urls: |
            [
              {
                "url": "https://www.yoursite.com/",
                "steps": []
              },
              {
                "url": "https://www.yoursite.com/products",
                "steps": []
              },
              {
                "url": "https://www.yoursite.com/secure-area",
                "steps": [
                  {
                    "type": "input",
                    "credential": "email",
                    "scanAfterAction": true,
                    "value": "test@test.com",
                    "selector": {
                      "inputType": "LogIn"
                    }
                  },
                  {
                    "type": "input",
                    "scanAfterAction": true,
                    "value": "TestPassword",
                    "selector": {
                      "inputType": "Password"
                    }
                  }
                ]
              }
            ]
          setup: |
            []
          teardown: |
            []

Key YAML Components:

  • schedule: Cron expression for automated runs

  • workflow_dispatch: Enables manual triggering

  • testparty_token: References your secret API token

  • urls: Array of URL objects with optional steps

  • setup/teardown: Optional arrays for pre/post scan actions

Step 6: Set Up API Token

  1. In TestParty dashboard, navigate to Continuous Integration → Sitemap → API Token

  2. Copy your unique API token

  3. Add it as a repository secret:

For GitHub:

  1. Go to Repository → Settings → Secrets and variables → Actions

  2. Click "New repository secret"

  3. Name: TESTPARTY_TOKEN (exact capitalization required)

  4. Value: Paste your API token

  5. Click "Add secret"

For GitLab:

  1. Go to Settings → CI/CD → Variables

  2. Click "Add variable"

  3. Key: TESTPARTY_TOKEN

  4. Value: Your API token

  5. Check "Mask variable"

For BitBucket:

  1. Go to Repository settings → Pipelines → Repository variables

  2. Name: TESTPARTY_TOKEN

  3. Value: Your API token

  4. Check "Secured"

(REVIEW) Step 7: Deploy to Your CI/CD Pipeline

GitHub Actions

  1. In TestParty, click "Commit YAML to GitHub"

    • This attempts to automatically create the workflow file

  2. If automatic commit fails:

    • Copy the generated YAML

    • In your repository, create: .github/workflows/testparty-scan.yml

    • Paste the YAML content

    • Commit and push

Handling Permission Errors:

If you see "Cannot find test-party/reporter in marketplace":

  1. Your organization restricts GitHub Actions

  2. Contact your SCM/DevOps team

  3. Request allowlisting of: test-party/reporter@v0.0.1-alpha.5

  4. Provide context: "Required for automated accessibility compliance testing"

GitLab CI

Add to .gitlab-ci.yml:

yaml

testparty-scan:
  image: ubuntu:latest
  script:
    - curl -X POST https://api.testparty.ai/scan
      -H "Authorization: Bearer $TESTPARTY_TOKEN"
      -d @scan-config.json
  only:
    - schedules

Jenkins

Create a new pipeline with:

groovy

pipeline {
    agent any
    triggers {
        cron('0 10 * * *')
    }
    stages {
        stage('Accessibility Scan') {
            steps {
                sh '''
                    curl -X POST https://api.testparty.ai/scan \
                    -H "Authorization: Bearer ${TESTPARTY_TOKEN}" \
                    -d @scan-config.json
                '''
            }
        }
    }
}

(REVIEW) Step 8: Run Your First Scan

Manual Trigger (GitHub Actions)

  1. Navigate to repository → Actions tab

  2. Select "TestParty Reporter" workflow

  3. Click "Run workflow" dropdown

  4. Select branch (usually main)

  5. Click green "Run workflow" button

  6. Monitor progress in the workflow run

Verify Successful Execution:

  • Green checkmark = Success

  • Red X = Failed (check logs)

  • Yellow circle = Running

Common First-Run Issues:

  1. Token not found: Verify TESTPARTY_TOKEN secret is set

  2. Reporter not found: Need to allowlist the action

  3. URLs unreachable: Check network/firewall settings

Step 9: View Results

  1. Return to TestParty dashboard

  2. Navigate to Reports section

  3. Find your scan (named by timestamp)

  4. Each scan provides:

    • Executive Summary: High-level metrics

    • Detailed Issues: Categorized by severity

    • Screenshots: Visual evidence with bounding boxes

    • Code Suggestions: Fixes for each issue

    • Export Options: CSV, PDF, or create tickets

Report Sections:

  • Critical Issues: Must fix immediately (legal risk)

  • Serious Issues: Should fix soon (user impact)

  • Low Issues: Good to fix (best practices)

(REVIEW) Step 10: Schedule Ongoing Scans

Modify Scan Frequency:

Edit the cron expression in your YAML:

yaml

schedule:
  - cron: '0 2 * * *'      # Daily at 2 AM
  - cron: '0 */6 * * *'    # Every 6 hours
  - cron: '0 9 * * 1-5'    # Weekdays at 9 AM
  - cron: '0 0 * * 0'      # Weekly on Sunday
  - cron: '0 0 1 * *'      # Monthly on the 1st

Cron Format: minute hour day-of-month month day-of-week

Best Practices for Different Architectures

Single Page Applications (SPAs)

For apps where URL doesn't change (like UKG):

json

{
  "url": "https://app.yoursite.com/",
  "steps": [
    {"type": "click", "selector": {"id": "nav-products"}, "scanAfterAction": true},
    {"type": "click", "selector": {"id": "nav-settings"}, "scanAfterAction": true},
    {"type": "click", "selector": {"id": "nav-reports"}, "scanAfterAction": true}
  ]
}

Multi-Page Applications

For traditional websites with distinct URLs:

json

[
  {"url": "https://site.com/", "steps": []},
  {"url": "https://site.com/products", "steps": []},
  {"url": "https://site.com/about", "steps": []},
  {"url": "https://site.com/contact", "steps": []}
]

Authenticated Areas

For sections requiring login:

json

{
  "url": "https://app.site.com/dashboard",
  "steps": [
    {"type": "input", "credential": "email", "value": "${{ secrets.TEST_EMAIL }}", "selector": {"id": "email"}},
    {"type": "input", "value": "${{ secrets.TEST_PASS }}", "selector": {"id": "password"}},
    {"type": "click", "selector": {"id": "login"}, "scanAfterAction": true}
  ]
}

(REVIEW) Troubleshooting

Debug Mode

Add to your YAML for verbose logging:

yaml

with:
  debug: true
  testparty_token: ${{ secrets.TESTPARTY_TOKEN }}

Common Selector Issues

If elements aren't found:

  1. Use browser DevTools to verify selectors

  2. Try different selector types:

    • id: Most reliable

    • class: For multiple elements

    • inputType: For form fields

    • custom: CSS selectors

Handling Dynamic Content

For JavaScript-heavy sites:

json

{
  "steps": [
    {"type": "wait", "duration": 3000},
    {"type": "click", "selector": {"id": "dynamic-button"}}
  ]
}

Support Resources

Remember: This is a one-time setup. Once configured, Spotlight runs automaticaly on your schedule, continuously monitoring accessibility without manual intervention.