Smoke Testing Workflow

This page is under construction. More content coming soon!

What are Smoke Tests?

Smoke tests are a subset of critical tests that verify the most important functionality of your application. They ensure the system is stable enough for further testing.

Think of smoke tests as “build verification tests”—if smoke tests fail, the build is not ready for deeper testing.

When to Run Smoke Tests

After Deployment

Verify deployment was successful and core features work

After Build

Validate new build before handing off to QA

On Schedule

Automated nightly runs to catch environmental issues

Before Release

Final sanity check before releasing to production

Creating a Smoke Test Suite

Identify Critical Paths

What functionality is absolutely essential?Examples:

User can log in
User can view homepage
User can add item to cart
User can complete checkout
User can log out

Tag Tests as Smoke

Use OQL to find and tag your critical tests:

priority IN (p0, p1) AND component IN (login, checkout, core)

Then bulk-tag them with smoke

Keep It Small

Smoke tests should be:

Fast: Complete in <15 minutes
Focused: Only critical functionality
Stable: No flaky tests allowed
Independent: No complex dependencies

If your smoke suite takes >30 minutes, it’s too large. Split into smoke + sanity suites.

Automate Where Possible

Automate smoke tests to run:

After every deployment (CI/CD pipeline)
On a schedule (nightly builds)
Before release (release gate)

Running Smoke Tests

Manual Execution
Automated (CI/CD)
Scheduled

Run Smoke Suite Manually

Select Tests

Use OQL to filter smoke tests:

tags CONTAINS "smoke" AND status = active
ORDER BY priority ASC

Create Test Run

Click ”+ New Test Run”:

Name: Smoke Tests - Build 2.1.0
Environment: Staging or Production
Build: 2.1.0

Execute

Run tests in order of priority (P0 first, then P1)

Report

If any test fails, STOP and report immediately. Don’t proceed with further testing until smoke tests pass.

Integrate with CI/CD Pipeline

# Example: GitHub Actions workflow
name: Smoke Tests

on:
  deployment_status

jobs:
  smoke-tests:
    runs-on: ubuntu-latest
    if: github.event.deployment_status.state == 'success'

    steps:
      - name: Run Smoke Tests
        run: |
          # Use OneTest API to trigger smoke test run
          curl -X POST https://api.onetest.ai/v1/runs \
            -H "Authorization: Bearer ${{ secrets.ONETEST_API_KEY }}" \
            -d '{
              "name": "Smoke Tests - ${{ github.sha }}",
              "environment": "${{ github.event.deployment_status.environment }}",
              "filters": "tags CONTAINS \"smoke\" AND status = active"
            }'

      - name: Wait for Results
        run: |
          # Poll for results and fail if smoke tests fail
          # ... polling logic ...

      - name: Notify Team
        if: failure()
        uses: slack-notify-action

Schedule Regular Smoke Tests

Set up automated recurring runs:

Create Saved Query

Save your smoke test filter:

tags CONTAINS "smoke" AND status = active

Configure Schedule

Frequency: Nightly, or after business hours
Environment: Staging or Production
Notification: Alert on failure

Monitor

Check dashboard each morning for overnight results

Example Smoke Test Suite

E-Commerce Smoke Tests

Critical Paths:

Homepage loads successfully
User can search for products
User can view product details
User can add product to cart
User can view cart
User can proceed to checkout
User can complete payment
User receives order confirmation

Total Time: ~8 minutes Priority: All P0 tests

SaaS Application Smoke Tests

Critical Paths:

Landing page loads
User can sign up
User can log in
Dashboard displays data
User can create new item
User can edit item
User can delete item
User can log out

Total Time: ~10 minutes Priority: All P0 tests

API Smoke Tests

Critical Endpoints:

Health check returns 200
Authentication endpoint works
GET /users returns data
POST /users creates user
PUT /users/ updates user
DELETE /users/ removes user
Database connection is healthy
Cache is accessible

Total Time: ~2 minutes Priority: All P0 tests

Smoke Test Best Practices

Keep It Fast

Target <15 minutes total execution time

Zero Flaky Tests

Remove or fix any flaky tests immediately

Run on Real Environments

Test on staging or production-like environments

Block on Failure

Failing smoke tests should block deployment

Version Control

Track which build/version was tested

Clear Reporting

Make results visible to entire team

Troubleshooting

Smoke Tests Taking Too Long

Problem: Smoke suite takes >30 minutesSolutions:

Review tests—are they all truly critical?
Remove redundant test steps
Parallelize test execution
Consider splitting into smoke + sanity suites

Flaky Smoke Tests

Problem: Smoke tests sometimes pass, sometimes failSolutions:

Identify the flaky tests using pass rate analysis
Fix root cause (timing issues, environment problems, etc.)
Remove flaky tests from smoke suite temporarily
Add explicit waits/retries in test steps

Smoke Tests Always Fail

Problem: Smoke tests consistently fail after deploymentSolutions:

Check if deployment actually completed
Verify environment is accessible
Review recent code changes
Check for database migration issues
Validate configuration/environment variables

What’s Next?

Regression Testing

Comprehensive testing workflow

Organizing Tests

Best practices for test organization

OQL Examples

Learn to filter tests with OQL

Best Practices

General testing best practices

Common Workflows

​What are Smoke Tests?

​When to Run Smoke Tests

After Deployment

After Build

On Schedule

Before Release

​Creating a Smoke Test Suite

​Running Smoke Tests

​Run Smoke Suite Manually

​Integrate with CI/CD Pipeline

​Schedule Regular Smoke Tests

​Example Smoke Test Suite

​Smoke Test Best Practices

Keep It Fast

Zero Flaky Tests

Run on Real Environments

Block on Failure

Version Control

Clear Reporting

​Troubleshooting

​What’s Next?

Regression Testing

Organizing Tests

OQL Examples

Best Practices

What are Smoke Tests?

When to Run Smoke Tests

Creating a Smoke Test Suite

Running Smoke Tests

Run Smoke Suite Manually

Integrate with CI/CD Pipeline

Schedule Regular Smoke Tests

Example Smoke Test Suite

Smoke Test Best Practices

Troubleshooting

What’s Next?