# How OneTest Builds Source: https://onetest.ai/about The principles behind OneTest's Applied AI projects # Tools for shared work between people and agents OneTest builds Applied AI products around a simple idea: AI becomes useful when it works from durable context, produces inspectable artifacts, and participates in the same systems as the people around it. ## Our principles ### Keep the work visible Plans, test assets, presentations, decisions, and reports should remain inspectable after an agent session ends. Whenever practical, they belong in files and repositories that teams already understand. ### Give agents explicit contracts An instruction is not enough. Useful agent workflows need acceptance criteria, structured tools, observable outcomes, and completion gates. ### Fit existing workflows Our products integrate with Git, GitHub, VS Code, browsers, desktop applications, MCP, and standard command-line tools. Adoption should not require moving all work into a proprietary environment. ### Prefer focused tools OneTest is a collection of focused projects rather than a single monolithic platform. Each tool should solve a concrete problem well and compose with the others through open interfaces. ### Build in public We publish source code, design decisions, limitations, and unfinished experiments. Public development makes the work easier to inspect, challenge, reuse, and improve. ## Our roots OneTest started with AI-assisted testing. That experience produced a broader set of questions: * How should agents share plans with people? * How does work survive beyond a chat session? * How can outcomes be verified rather than merely claimed? * How should agent-created artifacts fit established engineering workflows? OneTest TMS continues the software-quality work. DeepSeek Harness Desktop, Octodeck, and Octobots explore the same questions across desktop AI, presentations, and project execution. See the current OneTest product family. Follow development, open an issue, or contribute code and ideas. # Ask Questions Source: https://onetest.ai/ai/ask-questions Get instant insights about your tests and quality metrics The AI Assistant has access to all your test data and can answer questions about coverage, quality trends, and specific tests. ## What You Can Ask ``` What's our pass rate this sprint? Are we seeing more failures than usual? ``` ``` Which features don't have smoke tests? Do we have enough regression coverage? ``` ``` How has quality changed over time? Which tests fail most often? ``` ``` Why is TC-0042 failing? When was this test last updated? ``` ## Example Questions ``` What's the pass rate for checkout tests? ``` ``` How many tests failed in the last production run? ``` ``` Are there any flaky tests? ``` ``` What's the average test execution time? ``` ``` Do we have tests for the new payment feature? ``` ``` Which features have the least test coverage? ``` ``` How many smoke tests do we have? ``` ``` Are all critical paths covered? ``` ``` Is our pass rate improving? ``` ``` Which tests started failing recently? ``` ``` How long do smoke tests usually take? ``` ``` What's the failure trend for login tests? ``` ``` What tests should I create next? ``` ``` Which tests should I prioritize? ``` ``` What's missing from our test suite? ``` ``` Should I add more smoke tests? ``` ## Getting Better Answers **Be specific:** Instead of "how are tests doing?", ask "what's the pass rate for authentication tests in staging?" **Provide timeframe:** Mention "this week", "last sprint", or "since last release" **Ask follow-ups:** The AI remembers context, so you can drill deeper ## What's Next? Create tests with AI Advanced AI tips # AI Best Practices Source: https://onetest.ai/ai/best-practices Get the most out of the OneTest AI Assistant Learn how to write better prompts, get more accurate results, and use the AI Assistant effectively. ## Writing Better Prompts ❌ "Create a test" ✅ "Create a test for login with OAuth via Google on mobile" ❌ "Generate tests" ✅ "Generate smoke tests for the checkout flow in our e-commerce app" ❌ "Test the API" ✅ "Test the POST /users API endpoint with valid and invalid email formats" ❌ "Create tests for signup" ✅ "Create 3 separate tests for signup: happy path, invalid email, and weak password" ## Iteration Techniques Begin with a general request: ``` Create tests for user authentication ``` Ask the AI to adjust: ``` Add steps for two-factor authentication Include social login scenarios Make the test data more specific ``` Modify scope as needed: ``` Split this into separate tests for each auth method ``` ``` Expand this to cover all error scenarios ``` Once you're happy with the format, use it as a reference: ``` Create more tests like TC-0042 ``` ## Do's and Don'ts Provide examples of what you want Mention your product type or industry Specify test priorities and tags Ask for explanations if unclear Request test data to be included Mention platform (web, mobile, API) Reference user stories or requirements Ask the AI to review existing tests Don't assume the AI knows your internal terminology Don't skip reviewing AI-generated content Don't use vague requests without context Don't forget to specify test type (manual/automated) Don't include sensitive production data in prompts Don't expect perfect results on first try ## Advanced Techniques Upload or paste requirements directly: ``` Generate tests from this user story: [paste entire user story with acceptance criteria] ``` Upload API documentation: ``` Create API tests from these Swagger docs: [attach or paste OpenAPI spec] ``` Use existing tests as templates: ``` Create tests similar to TC-0042 but for the signup flow ``` ``` Update TC-0156 to include these new requirements: [paste requirements] ``` Get suggestions: ``` What edge cases should I test for file upload? ``` ``` Review my checkout tests and suggest improvements ``` ``` What tests am I missing for this feature? ``` Work with multiple tests: ``` Generate a complete test suite for user management including CRUD operations ``` ``` Create tests for all error scenarios in the payment flow ``` ``` Convert these manual tests to automated test scripts ``` ## Common Patterns ### Pattern 1: Feature-Complete Testing ``` Generate a complete test suite for [feature] including: - Happy path scenarios - Error handling - Edge cases - Performance requirements - Accessibility checks ``` ### Pattern 2: From User Stories ``` Create tests for this user story: Title: As a [user type], I want to [action] so that [benefit] Acceptance Criteria: - [criterion 1] - [criterion 2] - [criterion 3] ``` ### Pattern 3: API Testing ``` Generate API tests for [endpoint] including: - Valid request - Invalid parameters - Authentication failures - Rate limiting - Response validation ``` ### Pattern 4: Regression Suite ``` Create regression tests for [feature] covering: - Previously reported bugs - Recent changes - Integration points - Critical workflows ``` ## Quality Checklist Before saving AI-generated tests, verify: Title clearly describes what is being tested Steps are specific and actionable Expected results are measurable Preconditions are listed Test data is appropriate Tags and priority are correct Test is atomic (tests one thing) No assumptions about system state ## Troubleshooting **Try:** * Rephrase your question * Provide more context * Give an example of what you want * Break complex requests into smaller parts **Try:** * Include specific test data * Mention your product/industry * Reference existing tests as templates * Provide acceptance criteria or requirements **Try:** * Show an example of your preferred format * Specify which fields to include * Ask for specific sections to be expanded * Request step-by-step refinements **Try:** ``` What edge cases am I missing for [feature]? ``` ``` Generate negative test scenarios for [feature] ``` ``` What error conditions should I test? ``` ## Pro Tips **Save Good Prompts:** When you craft a prompt that works well, save it for future use **Use Templates:** Once you have a test format you like, ask the AI to create more tests using the same structure **Iterate in Conversation:** Don't try to get perfect results in one prompt—refine through conversation **Combine with OQL:** Use the AI to help build complex OQL queries for advanced searches ## What's Next? Put these practices to use Master advanced searches # Generate Tests with AI Source: https://onetest.ai/ai/generate-tests Create complete test cases in seconds using natural language The AI Assistant can generate complete, detailed test cases in seconds. Just describe what you want to test, and the AI creates a fully-formed test case with steps, expected results, and tags. ## Quick Start Press `Cmd + K` (Mac) or `Ctrl + K` (Windows/Linux) Type your request in plain English: ``` Create a test for user login with valid credentials ``` The AI will create a complete test case with: * Clear title * Test steps with expected results * Preconditions * Tags * Priority * Click **"Save"** to add it to your test suite * Or ask the AI to modify it: "Add a step for two-factor authentication" ## Generation Patterns **Create one specific test:** ``` Create a test for password reset ``` ``` Generate a test case for adding items to cart ``` ``` Write a test for user profile update ``` The AI will generate a single, focused test case. **Create a suite of related tests:** ``` Generate smoke tests for the checkout flow ``` ``` Create regression tests for user authentication ``` ``` Generate API tests for the products endpoint ``` The AI will create multiple test cases covering different scenarios. **Generate from user stories or requirements:** ``` Create tests for this user story: As a user, I want to filter products by price range so that I can find items within my budget. ``` The AI will analyze the requirement and generate appropriate tests. **Generate from API docs or specifications:** ``` Generate API tests from these docs: POST /api/users Creates a new user account Request Body: - email (required): string - password (required): string (min 8 chars) - name (optional): string Response: 201 Created ``` The AI will create tests covering all scenarios and edge cases. ## What Gets Generated When you ask the AI to create a test, it automatically generates: * Unique identifier (TC-XXXX) * Clear, descriptive title * Appropriate priority (P0-P4) * Relevant tags * Test type (manual/automated) * Preconditions * Detailed test steps * Expected results for each step * Overall pass/fail criteria * Test data (when applicable) * Follows naming conventions * Includes edge cases * Uses clear language * Atomic test design * Matches your product type * Uses existing tags * Follows team patterns * Considers related tests ## Example Prompts ``` Generate smoke tests for e-commerce checkout ``` **AI generates:** * Add item to cart * Update cart quantities * Apply discount code * Select shipping method * Enter payment information * Complete purchase * Receive confirmation email ``` Create API tests for user CRUD operations ``` **AI generates:** * Create user (valid data) * Create user (invalid email) * Create user (duplicate email) * Get user by ID * Update user information * Delete user * Verify deletion ``` Generate tests for mobile app onboarding ``` **AI generates:** * First launch experience * Permission requests * Account creation * Profile setup * Feature introduction screens * Skip onboarding flow * Complete onboarding ``` Create security tests for login ``` **AI generates:** * SQL injection attempts * XSS attack prevention * Brute force protection * Session fixation prevention * CSRF token validation * Password complexity enforcement * Account lockout after failures ## Refining Generated Tests You can iterate with the AI to refine tests: ``` Add a step to verify email notification is sent ``` ``` Make this test cover mobile view as well ``` ``` Split this into separate positive and negative tests ``` ``` Make this more detailed with specific test data ``` ``` Simplify this test to just the happy path ``` ``` Convert this to an API test ``` ``` This is for a healthcare app, add HIPAA compliance checks ``` ``` We use Stripe for payments, update the payment steps ``` ``` Include accessibility testing requirements ``` ## Advanced Techniques Generate many tests at once: ``` Create a complete test suite for the shopping cart feature including: - Adding items - Updating quantities - Removing items - Cart persistence - Checkout initiation ``` The AI will create 10-15 related tests covering all aspects. Upload UI screenshots: ``` Generate tests for this login screen [attach screenshot] ``` The AI analyzes the UI and creates appropriate tests. Generate test variations: ``` Create the same login test but for: - Desktop web - Mobile web - iOS app - Android app ``` The AI creates platform-specific test cases. Focus on edge cases: ``` What edge cases should I test for file upload? ``` The AI suggests and can generate tests for: * Empty file * File too large * Invalid file type * Special characters in filename * Network interruption * Concurrent uploads ## Quality Assurance **Review AI Output:** Always review generated tests to ensure they match your specific requirements and context. **Provide Feedback:** Tell the AI what to adjust: "Make step 3 more specific" or "Add assertions for response time" **Test Data:** Be cautious with sensitive data. The AI generates sample data, but you should use appropriate test data for your environment. ## Best Practices Instead of "create a login test," try "create a test for login with OAuth via Google" Mention the feature, platform, or user story: "Create a test for the new multi-factor authentication feature on mobile" Refine the test through conversation: "Add error handling scenarios" or "Include performance assertions" Once you get a good test format, save it as a template for consistency ## What's Next? Find tests using natural language Get insights about your test suite Use folders and tags effectively Advanced AI tips and techniques # AI Assistant Overview Source: https://onetest.ai/ai/overview Your intelligent testing companion powered by Claude The OneTest AI Assistant is your intelligent testing companion that helps you create tests, search your test suite, analyze results, and get instant answers—all through natural conversation. **Prerequisites:** The AI Assistant requires an LLM provider to be configured before use. Go to [Settings > LLM Configuration](/settings/llm-configuration) to set up your AI provider. AI capabilities are built into the platform and available across test generation, search, and analysis once your provider is configured. ## What Can the AI Do? Create complete, detailed test cases in seconds by describing what you want to test Find tests using natural language instead of complex filters Get instant answers about your tests, runs, and quality metrics Learn how to get the most out of the AI Assistant ## How to Access the AI Press `Cmd + K` (Mac) or `Ctrl + K` (Windows/Linux) Describe what you want in plain English The AI will generate tests, search results, or answers Save, refine, or continue the conversation ## Common Use Cases **Create a single test:** ``` Create a test for login with valid credentials ``` **Create multiple tests:** ``` Generate smoke tests for the checkout flow ``` **From requirements:** ``` Create tests for this user story: [paste story] ``` **From documentation:** ``` Generate tests from these API docs: [paste or upload] ``` The AI generates complete tests with: * Clear titles * Step-by-step instructions * Expected results * Preconditions * Tags **By status:** ``` Show me all failed tests from yesterday ``` **By feature:** ``` Find all smoke tests for login ``` **By metadata:** ``` Which tests haven't been run in the last 30 days? ``` **Complex queries:** ``` Show P1 tests that failed in staging but passed in dev ``` The AI translates your natural language into precise OQL queries automatically. **About quality:** ``` What's the pass rate for checkout tests this sprint? ``` **About coverage:** ``` Which features don't have any smoke tests? ``` **About trends:** ``` Are we seeing more failures than usual? ``` **About specific tests:** ``` Why is TC-0042 failing? ``` The AI has access to all your test data and can provide instant insights. **Learn features:** ``` How do I organize tests with folders? ``` **Understand concepts:** ``` What's the difference between a test case and a test run? ``` **Troubleshoot:** ``` Why can't I delete this test? ``` **Get suggestions:** ``` What tests should I create for a new feature? ``` ## Example Conversations **You:** Create a test for login with valid credentials **AI:** I'll create a comprehensive login test for you. **Generated Test:** ``` TC-0042: Verify successful login with valid credentials Type: Manual Priority: P2 Tags: smoke, login, authentication Preconditions: - User has a valid account in the system - User is not currently logged in - Login page is accessible Test Steps: 1. Navigate to the login page Expected: Login form displays with username and password fields 2. Enter valid username: test@example.com Expected: Username field accepts input 3. Enter valid password Expected: Password is masked (•••) 4. Click "Login" button Expected: User redirected to dashboard within 2 seconds 5. Verify user session Expected: User name displayed in header: "Welcome, Test User" Pass Criteria: - Login succeeds without errors - Dashboard loads within 2 seconds - Session is created and persists ``` **AI:** Would you like me to create additional test cases for invalid credentials or edge cases? **You:** Show me tests that failed in the last production run **AI:** I found 3 tests that failed in the most recent production run: 1. **TC-0156: Checkout with saved payment method** * Failed at: Step 4 (Payment processing) * Error: Payment gateway timeout * Last passed: 2 days ago 2. **TC-0089: User profile update** * Failed at: Step 3 (Save changes) * Error: 500 Internal Server Error * Last passed: 1 week ago 3. **TC-0234: Search with special characters** * Failed at: Step 2 (Execute search) * Error: Invalid query format * First failure Would you like me to: * Create bug reports for these failures? * Show the full test execution details? * Compare with staging results? **You:** Do we have enough smoke tests for the authentication feature? **AI:** Let me analyze your authentication test coverage: **Current Coverage:** * ✅ Login with valid credentials (TC-0042) * ✅ Logout (TC-0043) * ✅ Session timeout (TC-0198) * ⚠️ Login with invalid credentials (missing) * ⚠️ Password reset flow (TC-0167 exists but not tagged 'smoke') * ⚠️ "Remember me" functionality (missing) * ⚠️ Account lockout after failed attempts (missing) **Recommendation:** You have 3 smoke tests for authentication, but I recommend adding 4 more to cover critical failure scenarios and security features. Would you like me to: 1. Generate the missing smoke tests? 2. Update TC-0167 to include 'smoke' tag? 3. Create a complete authentication test suite? ## Interactive Forms When creating structured content like test cases, the AI presents **interactive forms** instead of plain text responses. These forms appear as editable cards in the chat: * **Pre-filled fields** based on your request (title, steps, priority, tags) * **Edit before saving** — modify any field directly in the form * **Expand to full view** for complex forms with many fields * **Submit or cancel** — submit saves the data, cancel dismisses the form This means you can say "create a test for login" and get a structured, editable form that creates a real test case when you submit it — no copy-pasting needed. ## AI Capabilities The AI Assistant can: Knows what page you're on and what you're working with Can see all your tests, runs, and results Recognizes your team's testing patterns and conventions Creates tests, tags, and documentation Identifies quality trends and potential issues Offers recommendations based on best practices ## Tips for Better Results **Be specific:** Instead of "create a test," try "create a test for user registration with email verification" **Provide context:** Mention the feature, user story, or requirement you're testing **Ask follow-ups:** The AI remembers your conversation, so you can refine and iterate **Upload documents:** You can paste or upload requirements, user stories, or API docs **Review AI output:** Always review generated tests to ensure they match your specific requirements ## Keyboard Shortcuts | Shortcut | Action | | ------------------ | ----------------- | | `Cmd/Ctrl + K` | Open AI chat | | `Esc` | Close chat | | `↑` | Edit last message | | `Cmd/Ctrl + Enter` | Send message | ## What's Next? Learn how to create tests with AI Master natural language search Get more insights from the AI Advanced tips and techniques # Search Tests with AI Source: https://onetest.ai/ai/search-tests Find tests instantly using natural language queries Use natural language to search your test suite. The AI translates your questions into precise OQL queries automatically. ## Quick Examples ``` Show me all failed tests from yesterday ``` ``` Find all smoke tests for login ``` ``` List all P0 tests that haven't run yet ``` ``` Show P1 tests that failed in staging but passed in dev ``` ## How It Works Type your search as a question or statement: ``` Which tests haven't been run in the last 30 days? ``` The AI converts your query to OneTest Query Language: ```oql theme={null} last_run < -30d ORDER BY title ``` See matching tests with relevant details highlighted Ask follow-up questions to narrow down results ## Common Search Patterns ``` Tests created this week Tests that failed yesterday Tests not run in the last month Tests updated in the last 24 hours ``` ``` All failed tests Passing smoke tests Tests in draft status Approved tests only ``` ``` All login tests Checkout flow tests API tests for users endpoint Mobile-specific tests ``` ``` Tests created by John Tests assigned to QA team My tests Unassigned tests ``` ## Advanced Searches ``` Show P1 login tests that failed in the last run ``` ``` Find smoke tests created this sprint that haven't been run ``` ``` List all API tests with high priority that are not automated ``` ``` Tests without any tags ``` ``` Tests that have never been run ``` ``` Tests not in any folder ``` ``` Tests that failed in prod but passed in staging ``` ``` Tests that take longer than 5 minutes ``` ``` Tests with pass rate below 80% ``` ``` Tests with "auth" in the title ``` ``` Tests starting with "TC-10" ``` ``` Tests containing "checkout" in description ``` ## What's Next? Write queries manually for more control Get insights about your tests # Quick Start Guide Source: https://onetest.ai/getting-started/quickstart Get up and running with OneTest in under 5 minutes Get up and running with OneTest in under 5 minutes. This guide will walk you through creating your first test, running it, and viewing results. ## Prerequisites Before you begin, make sure you have: A OneTest account ([Sign up here](https://tms.onetest.ai)) Access to the OneTest web application Basic understanding of software testing concepts If you're new to testing, check out [What is OneTest?](/getting-started/what-is-onetest) first for a gentle introduction. ## Step 1: Create Your First Product A **product** is a workspace that contains all your tests, test runs, and settings. Click **"+ New Product"** in the top navigation Enter the following information: * **Name:** `My First App` * **Description:** `Testing my web application` * **Type:** `Web Application` Click **"Create Product"** Create Product Demo

You can create multiple products for different applications or teams. ## Step 2: Create Your First Test Case Now let's create a simple test case. We'll use the AI Assistant to make it quick! Press `Cmd/Ctrl + K` to open the chat Type: ``` Create a test for login with valid credentials ``` Watch the AI create your test automatically Click **"Save"** when you're happy with it **Example AI Output:** ```markdown theme={null} ✅ Created TC-0001: Verify successful login with valid credentials Title: Verify successful login with valid credentials Type: Manual Priority: P2 Tags: smoke, login, authentication Preconditions: - User has a valid account - User is not already logged in Test Steps: 1. Navigate to the login page Expected: Login form is displayed with username and password fields 2. Enter valid username: test@example.com Expected: Username field accepts the input 3. Enter valid password Expected: Password is masked with dots (••••) 4. Click the "Login" button Expected: User is redirected to the dashboard within 2 seconds 5. Verify user name is displayed in the header Expected: Header shows "Welcome, Test User" Pass Criteria: - User successfully logs in without errors - Dashboard loads within 2 seconds - User session is created ``` AI-generated tests are usually more detailed and follow best practices automatically! Go to the **Test Management** page Click **"+ New Test Case"** button * **Title:** `Verify successful login with valid credentials` * **Type:** `Manual` * **Priority:** `P2` * **Tags:** Add `smoke` and `login` For each step: * Click **"+ Add Step"** * Enter **Action:** (e.g., `Navigate to login page`) * Enter **Expected Result:** (e.g., `Login form is displayed`) Click **"Save Test Case"** Create Test Case Demo

Your test case is automatically assigned a unique identifier (e.g., TC-0001) and is now ready to be executed! ## Step 3: Run Your First Test Now let's execute the test case you just created. Navigate to the **Test Management** page Locate your test case (TC-0001) Click the **"▶ Run"** button In the dialog, enter: * **Run Name:** `First Test Run - Staging` * **Environment:** Select `Staging` (or create one) * **Build:** Select a build or leave empty Click **"Start Run"** Create Test Run

## Step 4: Execute and Record Results OneTest will take you to the **Test Execution** page where you can record your results as you test. ### Recording Results For each test step: 1. **Perform the action** (e.g., navigate to login page) 2. **Observe the result** 3. **Click the status:** * ✅ **Pass** - Everything works as expected * ❌ **Fail** - Something doesn't match the expected result * ⏭️ **Skip** - Step wasn't executed * 🚫 **Blocked** - Can't proceed due to an issue 4. Add notes if needed (especially for failures!) ### Example Execution ```markdown theme={null} Step 1: Navigate to login page Status: ✅ Pass Notes: Login form loaded correctly Step 2: Enter valid username Status: ✅ Pass Step 3: Enter valid password Status: ✅ Pass Notes: Password was properly masked Step 4: Click Login button Status: ❌ Fail Notes: Got error "Server timeout" - dashboard didn't load Step 5: Verify user name Status: 🚫 Blocked Notes: Could not complete due to step 4 failure ``` When done, click **"Complete Execution"**. ## Step 5: View Your Results After completing the test run, you'll see a summary: ### Run Summary | Metric | Value | | --------------- | --------- | | **Total Tests** | 1 | | **Passed** | 0 | | **Failed** | 1 | | **Blocked** | 0 | | **Pass Rate** | 0% | | **Duration** | 8 minutes | Complete Test Run

### What's Next? * Continue creating more test cases * Build a smoke test suite * Set up automated runs * Review failure notes * Report bugs to your team * Retest after fixes ## Next Steps Congratulations! You've created and run your first test. Now you can: Create tests faster with AI assistance Use folders and tags effectively Search tests with powerful queries Configure staging and production Manage releases and sprints Collaborate with your team ## Common Questions Unlimited! Create as many as you need to properly test your application. Manual tests record human execution. For automation, integrate with tools like Selenium or Playwright (coming soon). Yes! Use the AI to convert existing test documentation: ``` Create tests from this document: [paste your test doc] ``` Use **folders** for hierarchy and **tags** for cross-cutting concerns: * Folders: By feature (Login, Checkout, Profile) * Tags: By type (smoke, regression, api) [→ Learn more about organizing tests](/workflows/organizing-tests) ## Need Help? Press `Cmd/Ctrl + K` and ask anything Join the community discussions Found a bug? Let us know Get help from our team Try generating multiple tests with AI next! 🚀 # What is OneTest TMS? Source: https://onetest.ai/getting-started/what-is-onetest Learn how OneTest TMS simplifies test management OneTest TMS helps teams create, organize, execute, and maintain tests with AI-assisted workflows. ## Why OneTest TMS? Traditional test management can be complex and disconnected from development. OneTest TMS brings test design, organization, execution, reporting, and AI assistance into one focused workflow. Chat with an AI to create tests, find issues, and get insights—no scripting required. Create tests in seconds, organize with folders and tags, track runs and results. Find tests instantly with natural language queries. ### AI Assistant in Action Simply describe what you want to test: ``` You: "Create a login test for me" AI: I'll create a test case for login... ✓ Created TC-0001: Verify user login with valid credentials ``` Use natural language to find tests: ``` Show me failed tests from yesterday Find all smoke tests for the login feature ``` Ask questions about your test suite: ``` What's the pass rate for checkout tests this sprint? Which tests are failing most often? ``` ## Who is it for? Create and execute test cases efficiently with AI assistance Track coverage, quality metrics, and team progress Run smoke tests before deployment and catch issues early Understand test coverage and quality trends ## How it works **Using the UI:** 1. Click "+ New Test Case" 2. Fill in title and steps 3. Save **Using AI:** ``` "Create a test for password reset flow" ``` The AI generates a complete test case for you. Group tests with: * **Folders** - By feature (Login, Checkout, Settings) * **Tags** - By type (smoke, regression, api) * **Suites** - For specific runs (Nightly, Pre-deploy) 1. Select tests or a suite 2. Choose environment (Staging, Production) 3. Click "Run" 4. Track progress in real-time Monitor key metrics: * ✅ **Pass Rate** - What percentage passed? * ⏱️ **Duration** - How long did it take? * 📊 **Trends** - Is quality improving? * 🐛 **Failures** - What broke? ## Key Concepts A test case describes **what** to test and **how** to test it. **Example:** ``` Title: User can reset password via email Steps: 1. Click "Forgot Password" 2. Enter email address 3. Click "Send Reset Link" 4. Check email inbox 5. Click reset link 6. Enter new password 7. Log in with new password Expected: User successfully resets password and can log in ``` A test run is an **execution** of test cases against a specific build or environment. **Example:** ``` Run Name: Staging Smoke Tests - Build 245 Environment: Staging Tests: 25 smoke tests Status: 24 passed, 1 failed Duration: 15 minutes ``` A suite is a **collection** of related tests that run together. **Common Suites:** * **Smoke** - Critical paths (5-15 tests, 15 mins) * **Regression** - Full coverage (100+ tests, 2+ hours) * **Feature** - Specific feature tests Labels to categorize tests across folders. **Common Tags:** ``` smoke, regression, api, ui, mobile, critical, high-priority, automated ``` ## What makes OneTest different? ❌ Complex configuration
❌ Manual test creation
❌ Hard to search and filter
❌ Separate tools for different needs Zero configuration - works out of the box AI-powered test creation Natural language search (OQL) All-in-one platform ## Getting Started Ready to create your first test? Create and run your first test in 5 minutes Detailed walkthrough of creating a test ## Need Help? Ask the AI assistant anything Step-by-step guides Frequently asked questions # Your First Test Source: https://onetest.ai/getting-started/your-first-test Create your first test case in OneTest in about 2 minutes Let's create your first test case in OneTest. This should take about 2 minutes. ## What You'll Learn How to create a test case How to add test steps How to use tags How to save and track tests ## Create Your Test 1. Log in to OneTest 2. Click **Test Management** in the sidebar 3. You'll see an empty test list (that's normal!) Click the **"+ New Test Case"** button in the top right. You'll see a form with several fields. Let's fill in the basics: ### Title Give your test a clear name that describes what it tests. **Example:** ``` User can view dashboard after login ``` ### Description (Optional) Add more details if needed. **Example:** ``` Verify that after successful login, the user is redirected to the dashboard and can see their profile information. ``` ### Type Select **Manual** (we'll cover automated tests later). ### Priority Choose how important this test is: | Priority | When to Use | | -------- | ---------------------------------- | | **P0** | Critical (blocks release if fails) | | **P1** | High (major feature) | | **P2** | Medium (standard feature) | | **P3** | Low | | **P4** | Nice-to-have | For this example, choose **P2**. Click **"+ Add Step"** to add each step of your test. For each step, enter: * **Action** - What to do * **Expected Result** - What should happen **Example Steps:** * **Action:** `Navigate to login page` * **Expected:** `Login form is displayed` * **Action:** `Enter valid username and password` * **Expected:** `Credentials are accepted` * **Action:** `Click "Login" button` * **Expected:** `User is redirected to dashboard` * **Action:** `Verify dashboard page` * **Expected:** `Dashboard loads with user's name visible` Tags help you organize and find tests. Add these tags: ``` smoke, login, ui ``` **Tag Usage:** * **smoke** - Critical path tests * **login** - Feature grouping * **ui** - Test type Click the **"Create"** button at the bottom. 🎉 **Congratulations!** You've created your first test case! You should see: * Test identifier (e.g., `TC-0001`) * Status: `Draft` * Your test details ## What Just Happened? OneTest automatically handled several things for you: Every test gets a unique identifier like `TC-0001` Created version 1 of your test for full history Automatically set to "Draft" status Recorded who created it and when ## Next Steps Now that you've created your first test, here are some great next steps: Try creating tests for common workflows: * User registration * Password reset * Checkout process Let the AI create complete tests for you automatically Learn how to use folders and tags effectively Execute your test and track results ## Quick Tips **Keep tests atomic** - One test = One scenario **Use clear titles** - Describe what is being tested, not how **Be specific in steps** - Anyone should understand what to do **Tag consistently** - Use the same tags across your team ## Need Help? Having trouble? Try these: Open the chat and type "How do I create a test?" Video tutorial on creating tests Email our support team ## What You Learned How to create a test case What fields to fill in How to add test steps How to use tags Where your test is stored Ready for more? Pick your next guide above! 👆 # OneTest Source: https://onetest.ai/introduction Open-source Applied AI tools for software teams # Applied AI, built for real work OneTest is an open-source product studio building tools for people and AI agents. Our projects live where software work happens—in repositories, GitHub, VS Code, native desktop applications, and the browser. We build focused tools that keep plans, artifacts, and decisions visible to the people doing the work.

A bento layout showing OneTest tools for testing, desktop AI, presentations, and project execution

Follow the work, try a project, open an issue, or contribute on GitHub. Start with GitHub-native test management for humans and agents. ## Projects GitHub-native test management. Keep test assets in files, manage runs through GitHub, and let agents operate the workflow through MCP. A native macOS shell for DeepSeek Harness with managed installs, plugins, MCP servers, notifications, an editor, and a real browser. Deck-as-code for people and agents. Build typed, interactive presentations in TypeScript and export them as self-contained HTML. A repository-native project board for AI coding teams, built from durable YAML plans, acceptance criteria, and completion gates. ## How we build Important work stays in files and standard developer systems instead of disappearing into a separate database. People and AI agents work from the same plans, artifacts, acceptance criteria, and history. Each project solves a concrete problem and connects through Git, MCP, plugins, and familiar development tools. Our tools, formats, decisions, and implementation are public, portable, and open to contribution. ## From software quality to Applied AI OneTest began with AI-assisted software quality. That work continues in **OneTest TMS**, now being rebuilt around GitHub-native workflows. Along the way, we found the same problem across software work: agents need durable context, explicit acceptance criteria, inspectable artifacts, and tools that fit the way teams already work. OneTest now explores that problem across planning, development, presentations, and desktop AI. ## Built in public These projects are active experiments as well as useful tools. We share the work early, document the tradeoffs, and improve it in the open. Browse source code, releases, issues, and project history. Learn the principles connecting the OneTest projects. # OQL Examples Source: https://onetest.ai/oql/examples Real-world OneTest Query Language examples Real-world OQL query examples for common testing scenarios. ## Test Management ```oql theme={null} tags CONTAINS "smoke" AND status = active ORDER BY priority ASC ``` Returns all active smoke tests, sorted by priority. ```oql theme={null} status = failed AND last_run >= -7d ORDER BY last_run DESC ``` Tests that failed in the last week. ```oql theme={null} priority IN (p0, p1) AND status != archived ORDER BY created_at DESC ``` All P0 and P1 tests that aren't archived. ```oql theme={null} last_run IS NULL AND status = active ORDER BY created_at ASC ``` Active tests that have never been executed. ```oql theme={null} updated_at >= -14d ORDER BY updated_at DESC LIMIT 50 ``` Tests modified in the last 2 weeks. ```oql theme={null} tags CONTAINS "deprecated" AND last_run < -90d AND status != archived ``` Deprecated tests not run in 3 months. ## Feature-Specific ```oql theme={null} (title ~ "login" OR title ~ "auth" OR title ~ "signup") AND tags CONTAINS ANY (smoke, regression) AND status = active ORDER BY priority ASC ``` Important authentication tests. ```oql theme={null} tags CONTAINS "api" AND title ~ "/users" AND test_type = automated ``` Automated API tests for users endpoint. ```oql theme={null} tags CONTAINS ANY (ios, android, mobile) AND status = active ORDER BY platform, priority ``` All mobile tests grouped by platform. ```oql theme={null} (title ~ "payment" OR title ~ "checkout" OR tags CONTAINS "payment") AND priority IN (p0, p1, p2) ORDER BY priority ASC, title ASC ``` Critical payment-related tests. ## Quality Analysis ```oql theme={null} pass_rate < 0.8 AND pass_rate > 0 AND execution_count >= 5 AND status = active ORDER BY pass_rate ASC ``` Tests with inconsistent results (60-80% pass rate). ```oql theme={null} avg_duration > 300 AND status = active ORDER BY avg_duration DESC LIMIT 20 ``` Tests taking longer than 5 minutes on average. ```oql theme={null} environment = production AND status = failed AND last_run >= -24h ORDER BY priority ASC ``` Recent production failures. ```oql theme={null} status = failed AND defect_links IS NULL AND last_run >= -7d ``` Failed tests without linked bugs. ## Sprint & Release Planning ```oql theme={null} tags CONTAINS "sprint-24" AND status IN (active, approved) ORDER BY priority ASC ``` Tests tagged for sprint 24. ```oql theme={null} priority = p0 AND (status = failed OR status = blocked) AND tags CONTAINS "release" ``` Critical tests blocking release. ```oql theme={null} tags CONTAINS "regression" AND status = active AND automated = true ORDER BY folder_path, priority ``` Automated regression tests by folder. ```oql theme={null} created_at >= startOfMonth() AND created_by = "current_user" ORDER BY created_at DESC ``` Your tests from this month. ## Team & Ownership ```oql theme={null} created_by = "current_user" ORDER BY updated_at DESC ``` All tests you created. ```oql theme={null} assigned_to = "current_user" AND status IN (active, in_review) ``` Tests assigned to you. ```oql theme={null} status = in_review AND reviewer = "current_user" ORDER BY created_at ASC ``` Tests waiting for your review. ```oql theme={null} team = "qa" AND created_at >= startOfWeek() ORDER BY created_at DESC ``` QA team's tests from this week. ## Organization & Cleanup ```oql theme={null} tags IS NULL OR tags = [] ORDER BY created_at DESC LIMIT 100 ``` Untagged tests needing organization. ```oql theme={null} folder_id IS NULL AND status = active ``` Tests at root level. ```oql theme={null} title ~ "copy" OR title ~ "duplicate" ORDER BY title ASC ``` Potentially duplicate tests by name. ```oql theme={null} status = draft AND created_at < -30d ORDER BY created_at ASC ``` Draft tests older than 30 days. ## Advanced Combinations ```oql theme={null} (tags CONTAINS "smoke" OR priority = p0) AND status = active AND (last_run IS NULL OR last_run < -7d) AND automated = false ORDER BY priority ASC, created_at DESC ``` Critical manual tests needing execution. ```oql theme={null} status = failed AND environment = staging AND (environment != production OR last_run_production = passed) AND last_run >= -24h ``` Tests failing in staging but passing (or not run) in production. ```oql theme={null} automated = false AND execution_count >= 10 AND pass_rate > 0.9 AND avg_duration < 600 AND tags CONTAINS ANY (smoke, regression) ORDER BY execution_count DESC ``` Stable manual tests run frequently (good automation candidates). ```oql theme={null} status = active AND last_run >= -30d AND ( (pass_rate < 0.7 AND pass_rate_previous_month > 0.9) OR (avg_duration > avg_duration_previous_month * 1.5) ) ORDER BY pass_rate ASC ``` Tests with declining quality or performance. ## Tips for Writing Queries Begin with one condition and add more: ```oql theme={null} status = active → status = active AND priority = p1 → status = active AND priority = p1 AND tags CONTAINS "smoke" ``` Make complex logic clear: ```oql theme={null} (A OR B) AND (C OR D) ``` Put most selective conditions first for better performance: ```oql theme={null} priority = p0 AND status = active # Better status = active AND priority = p0 # Still works, but less optimal ``` Save complex queries for reuse instead of rewriting them ## What's Next? Complete OQL syntax documentation All searchable fields # OQL Field Reference Source: https://onetest.ai/oql/field-reference All searchable fields in OneTest Query Language Complete reference of all fields you can query in OQL. ## Test Case Fields ### Identifiers & Metadata | Field | Type | Description | Example | | ------------------- | ------ | ------------------------------------ | -------------------------------- | | `identifier` | string | Test case ID (TC-NNN) | `identifier ^ "TC-10"` | | `title` | string | Test case title | `title ~ "login"` | | `description` | string | Test description | `description ~ "authentication"` | | `test_type` | enum | manual, automated, exploratory | `test_type = manual` | | `status` | enum | draft, ready, deprecated, archived | `status = ready` | | `priority` | enum | p1, p2, p3, p4 | `priority IN (p1, p2)` | | `component` | string | Component name | `component = "checkout"` | | `automation_status` | enum | not\_automated, candidate, automated | `automation_status = automated` | ### Dates & Times | Field | Type | Description | Example | | ------------- | -------- | ------------- | ------------------------------ | | `created_at` | datetime | When created | `created_at >= -7d` | | `updated_at` | datetime | Last modified | `updated_at >= startOfMonth()` | | `archived_at` | datetime | When archived | `archived_at IS NOT NULL` | ### Organization | Field | Type | Description | Example | | ----------- | ----- | ------------- | ----------------------- | | `folder_id` | uuid | Parent folder | `folder_id IS NOT NULL` | | `tags` | array | Test tags | `tags CONTAINS "smoke"` | ### Ownership | Field | Type | Description | Example | | ------------ | ------ | --------------- | ------------------------- | | `created_by` | string | Creator user ID | `created_by = "user_123"` | | `owner` | string | Test case owner | `owner IS NOT NULL` | ## Release Fields | Field | Type | Description | Example | | -------------- | -------- | ------------------------- | ----------------------------- | | `name` | string | Release name | `name ~ "v2"` | | `description` | string | Release description | `description ~ "hotfix"` | | `scope` | string | Release scope | `scope = "backend"` | | `status` | enum | planned, active, released | `status = active` | | `start_date` | date | Planned start date | `start_date >= 2024-01-01` | | `end_date` | date | Planned end date | `end_date <= +30d` | | `created_at` | datetime | When created | `created_at >= -30d` | | `updated_at` | datetime | Last modified | `updated_at >= startOfWeek()` | | `created_by` | string | Creator user ID | `created_by = "user_123"` | | `sprint_count` | number | Number of sprints | `sprint_count > 0` | | `build_count` | number | Number of builds | `build_count >= 5` | ## Environment Fields | Field | Type | Description | Example | | ------------------ | -------- | -------------------------------- | ---------------------------------- | | `name` | string | Environment name | `name ~ "staging"` | | `description` | string | Description | `description ~ "production"` | | `endpoint_url` | string | Environment URL | `endpoint_url ~ "api.example.com"` | | `type` | enum | development, staging, production | `type = production` | | `created_at` | datetime | When created | `created_at >= -90d` | | `updated_at` | datetime | Last modified | `updated_at >= -7d` | | `deployment_count` | number | Number of deployments | `deployment_count > 0` | ## Build Fields | Field | Type | Description | Example | | ------------- | -------- | ------------------------- | -------------------------------- | | `version` | string | Build version | `version ^ "2.1"` | | `git_sha` | string | Git commit SHA | `git_sha ^ "abc123"` | | `scope` | string | Build scope | `scope = "frontend"` | | `status` | enum | pending, active, archived | `status = active` | | `created_at` | datetime | When created | `created_at >= -7d` | | `updated_at` | datetime | Last modified | `updated_at >= -1d` | | `release` | string | Parent release name | `release ~ "v2"` | | `sprint` | string | Parent sprint name | `sprint ~ "Sprint 5"` | | `deployed_to` | array | Deployed environments | `deployed_to CONTAINS "staging"` | ## Sprint Fields | Field | Type | Description | Example | | ------------ | -------- | -------------------------- | -------------------- | | `number` | number | Sprint number | `number >= 5` | | `name` | string | Sprint name | `name ~ "Sprint 10"` | | `scope` | string | Sprint scope | `scope = "backend"` | | `status` | enum | planned, active, completed | `status = active` | | `start_date` | date | Sprint start | `start_date >= -14d` | | `end_date` | date | Sprint end | `end_date <= +7d` | | `created_at` | datetime | When created | `created_at >= -30d` | | `release` | string | Parent release name | `release ~ "v3"` | ## Test Run Fields ### Run Metadata | Field | Type | Description | Example | | ------------- | ------ | ----------------------------------------- | ---------------------------- | | `name` | string | Run name | `name ~ "nightly"` | | `description` | string | Run description | `description ~ "regression"` | | `run_type` | enum | manual, automated | `run_type = automated` | | `status` | enum | planned, in\_progress, completed, aborted | `status = completed` | | `assigned_to` | string | Assigned user | `assigned_to = "user_123"` | | `created_by` | string | Creator | `created_by = "user_123"` | ### Run Timing | Field | Type | Description | Example | | --------------- | -------- | ------------- | ----------------------------- | | `planned_start` | datetime | Planned start | `planned_start >= -7d` | | `planned_end` | datetime | Planned end | `planned_end IS NOT NULL` | | `actual_start` | datetime | Actual start | `actual_start >= -24h` | | `actual_end` | datetime | Actual end | `actual_end IS NOT NULL` | | `created_at` | datetime | When created | `created_at >= startOfWeek()` | | `updated_at` | datetime | Last modified | `updated_at >= -1d` | ## Field Type Details ### String Fields Case-insensitive by default. Use pattern operators: ```oql theme={null} title ~ "login" # Contains title ^ "TC-" # Starts with title $ "test" # Ends with ``` ### Enum Fields Use exact values or IN for multiple: ```oql theme={null} status = ready priority IN (p1, p2) test_type = automated ``` ### Number Fields Support comparison operators: ```oql theme={null} sprint_count > 0 build_count >= 5 number BETWEEN 1 AND 10 ``` ### Boolean Fields ```oql theme={null} # Use IS NULL / IS NOT NULL for optional fields owner IS NOT NULL archived_at IS NULL ``` ### Datetime Fields Support ISO dates and relative dates: ```oql theme={null} created_at = "2024-01-15" created_at >= -7d created_at >= startOfMonth() updated_at >= startOfWeek() ``` ### Array Fields Support contains operations: ```oql theme={null} tags CONTAINS "smoke" tags CONTAINS ANY (smoke, regression) tags CONTAINS ALL (smoke, critical) deployed_to CONTAINS "staging" ``` Custom fields are also searchable using dot notation: ```oql theme={null} custom_fields.jira_ticket = "PROJ-123" ``` ## Tips Not sure what field to use? Ask the AI: ``` What fields can I search for automated tests? ``` The AI will suggest appropriate fields and show examples. ## What's Next? Real-world query examples Complete OQL syntax # OQL Overview Source: https://onetest.ai/oql/overview OneTest Query Language for powerful test searches **OQL (OneTest Query Language)** is a simple, powerful way to search and filter tests. It's inspired by JQL (Jira Query Language) and designed to feel natural while giving you precise control. ## Why Use OQL? Filter by exact criteria, not fuzzy matches Save complex searches for reuse Use AND, OR, NOT for complex filters Optimized queries execute instantly ## Quick Examples ```oql Simple Filter theme={null} status = active ``` ```oql Multiple Conditions theme={null} status = active AND priority IN (p1, p2) ``` ```oql Pattern Matching theme={null} title ~ "login" AND scope ^ "auth" ``` ```oql Date Queries theme={null} created_at >= -7d ORDER BY created_at DESC ``` ```oql Arrays theme={null} tags CONTAINS "smoke" AND tags NOT CONTAINS "deprecated" ``` ## Basic Syntax Compare fields to values: | Operator | Meaning | Example | | -------- | ---------------- | ------------------------- | | `=` | Equals | `status = active` | | `!=` | Not equals | `priority != p4` | | `>` | Greater than | `created_at > 2024-01-01` | | `>=` | Greater or equal | `version >= 2.0` | | `<` | Less than | `priority < p2` | | `<=` | Less or equal | `updated_at <= -7d` | Match text patterns: | Operator | Meaning | Example | | -------- | ------------ | ----------------------------- | | `~` | Contains | `title ~ "login"` | | `^ ` | Starts with | `identifier ^ "TC-10"` | | `$` | Ends with | `title $ "test"` | | `!~` | Not contains | `description !~ "deprecated"` | Work with lists: | Operator | Meaning | Example | | -------------- | ------------------------- | --------------------------------------- | | `IN` | Value in list | `priority IN (p1, p2)` | | `NOT IN` | Value not in list | `status NOT IN (archived, deleted)` | | `CONTAINS` | Array contains value | `tags CONTAINS "smoke"` | | `CONTAINS ANY` | Array contains any value | `tags CONTAINS ANY (smoke, regression)` | | `CONTAINS ALL` | Array contains all values | `tags CONTAINS ALL (smoke, critical)` | Combine conditions: | Operator | Meaning | Example | | -------- | ---------------- | ---------------------------------------------------------- | | `AND` | Both conditions | `status = active AND priority = p1` | | `OR` | Either condition | `priority = p1 OR priority = p2` | | `NOT` | Negate condition | `NOT status = archived` | | `( )` | Group conditions | `(status = active OR status = approved) AND priority = p1` | ## Common Queries ```oql theme={null} status = failed AND last_run >= -7d ``` Find tests that failed in the last 7 days. ```oql theme={null} tags CONTAINS "smoke" AND status = active ORDER BY priority ``` All active smoke tests, ordered by priority. ```oql theme={null} priority IN (p0, p1) AND status != archived ORDER BY created_at DESC ``` All P0 and P1 tests that aren't archived. ```oql theme={null} updated_at >= -14d ORDER BY updated_at DESC LIMIT 50 ``` Tests updated in the last 2 weeks. ```oql theme={null} last_run IS NULL AND status = active ``` Active tests that have never been executed. ```oql theme={null} folder_path ~ "/authentication/" AND (tags CONTAINS "smoke" OR priority = p1) ``` Important authentication tests. ## Date & Time Queries OQL supports flexible date queries: ```oql theme={null} created_at >= -7d # Last 7 days created_at >= -2w # Last 2 weeks created_at >= -3M # Last 3 months created_at >= -1y # Last year ``` ```oql theme={null} created_at >= 2024-01-01 created_at BETWEEN 2024-01-01 AND 2024-12-31 updated_at = 2024-06-15 ``` ```oql theme={null} created_at >= startOfWeek() created_at >= startOfMonth() created_at >= startOfYear() created_at <= endOfMonth() ``` ```oql theme={null} updated_at > created_at last_run >= updated_at ``` ## Sorting & Pagination Control result order and size: ```oql theme={null} # Sort by single field ORDER BY created_at DESC # Sort by multiple fields ORDER BY priority ASC, created_at DESC # Limit results LIMIT 50 # Pagination LIMIT 50 OFFSET 100 ``` ## Saved Queries Save frequently-used queries: ```oql theme={null} priority IN (p0, p1) AND status = active AND tags CONTAINS "smoke" ``` Click "Save Query" and give it a name: ``` Critical Smoke Tests ``` Access from the "Saved Queries" dropdown ## Using with AI You don't need to write OQL manually! Just ask the AI: ``` Show me high priority tests that failed recently ``` The AI translates to: ```oql theme={null} priority IN (p0, p1) AND status = failed AND last_run >= -7d ``` ## What's Next? Complete OQL syntax documentation More real-world query examples All searchable fields Let AI write queries for you # OQL Syntax Reference Source: https://onetest.ai/oql/syntax Complete OneTest Query Language syntax documentation Complete syntax reference for OneTest Query Language (OQL). ## Query Structure ```oql theme={null} [ORDER BY [ASC|DESC]] [LIMIT ] [OFFSET ] ``` All parts except the filter expression are optional. ## Operators ### Comparison Operators | Operator | Description | Example | | ------------- | ----------------- | ---------------------------------------------- | | `=` | Equals | `status = active` | | `!=` | Not equals | `priority != p4` | | `>` | Greater than | `version > 2.0` | | `>=` | Greater or equal | `created_at >= 2024-01-01` | | `<` | Less than | `priority < p2` | | `<=` | Less or equal | `updated_at <= -7d` | | `IS NULL` | Field is null | `last_run IS NULL` | | `IS NOT NULL` | Field is not null | `folder_id IS NOT NULL` | | `BETWEEN` | Value in range | `created_at BETWEEN 2024-01-01 AND 2024-12-31` | ### String Pattern Operators | Operator | Description | Example | | -------- | --------------------------- | ----------------------------- | | `~` | Contains (case-insensitive) | `title ~ "login"` | | `!~` | Does not contain | `description !~ "deprecated"` | | `^` | Starts with | `identifier ^ "TC-10"` | | `$` | Ends with | `title $ "test"` | ### List Operators | Operator | Description | Example | | -------- | ----------------- | ----------------------------------- | | `IN` | Value in list | `priority IN (p1, p2, p3)` | | `NOT IN` | Value not in list | `status NOT IN (archived, deleted)` | ### Array Operators | Operator | Description | Example | | -------------- | ------------------------- | --------------------------------------- | | `CONTAINS` | Array contains value | `tags CONTAINS "smoke"` | | `NOT CONTAINS` | Array does not contain | `tags NOT CONTAINS "deprecated"` | | `CONTAINS ANY` | Array contains any value | `tags CONTAINS ANY (smoke, regression)` | | `CONTAINS ALL` | Array contains all values | `tags CONTAINS ALL (smoke, critical)` | ### Logical Operators | Operator | Description | Example | | -------- | ----------------------------- | ---------------------------------------------------------- | | `AND` | Both conditions must be true | `status = active AND priority = p1` | | `OR` | Either condition must be true | `priority = p1 OR priority = p2` | | `NOT` | Negate condition | `NOT status = archived` | | `( )` | Group conditions | `(status = active OR status = approved) AND priority = p1` | ## Data Types ### Strings Enclose in double or single quotes: ```oql theme={null} title = "User Login" title = 'User Login' ``` Use quotes for strings with spaces or special characters. ### Numbers No quotes needed: ```oql theme={null} version = 2 priority_number > 1 ``` ### Booleans ```oql theme={null} automated = true is_flaky = false ``` ### Dates ```oql theme={null} created_at = 2024-01-15 created_at = "2024-01-15T10:30:00Z" ``` ```oql theme={null} created_at >= -7d # 7 days ago created_at >= -2w # 2 weeks ago created_at >= -3M # 3 months ago created_at >= -1y # 1 year ago ``` Units: `d` (days), `w` (weeks), `M` (months), `y` (years) ```oql theme={null} created_at >= startOfWeek() created_at >= startOfMonth() created_at >= startOfQuarter() created_at >= startOfYear() created_at <= endOfWeek() created_at <= endOfMonth() created_at <= endOfQuarter() created_at <= endOfYear() ``` ### Null Values ```oql theme={null} last_run IS NULL description IS NOT NULL ``` ## Field Names Field names are case-sensitive: ```oql theme={null} ✅ status = active ❌ Status = active ❌ STATUS = active ``` For fields with spaces or special characters, use quotes: ```oql theme={null} "custom field" = "value" ``` ## Ordering ```oql theme={null} ORDER BY created_at DESC ORDER BY priority ASC ORDER BY title ``` Default order is `ASC` (ascending). ```oql theme={null} ORDER BY priority ASC, created_at DESC ORDER BY status, priority, title ``` Sorts by first field, then second, etc. ## Pagination ```oql theme={null} # First 50 results LIMIT 50 # Next 50 results LIMIT 50 OFFSET 50 # Third page (rows 101-150) LIMIT 50 OFFSET 100 ``` `OFFSET` requires `LIMIT` to be specified. ## Examples by Complexity ```oql theme={null} # Single condition status = active # With ordering status = active ORDER BY created_at DESC # With limit priority = p1 LIMIT 10 ``` ```oql theme={null} # AND condition status = active AND priority IN (p1, p2) # OR condition priority = p1 OR tags CONTAINS "critical" # Pattern matching title ~ "login" AND status != archived # Date filtering created_at >= -30d ORDER BY created_at DESC LIMIT 100 ``` ```oql theme={null} # Multiple conditions with grouping (status = active OR status = approved) AND priority IN (p0, p1, p2) AND tags CONTAINS ANY (smoke, regression) AND created_at >= -90d ORDER BY priority ASC, created_at DESC LIMIT 50 # Array operations tags CONTAINS ALL (smoke, critical) AND tags NOT CONTAINS "deprecated" AND last_run >= -7d # Null checks and comparisons last_run IS NOT NULL AND last_run < updated_at AND status = active ``` ## Operator Precedence From highest to lowest: 1. `( )` - Parentheses 2. `NOT` - Logical NOT 3. `=, !=, >, >=, <, <=, ~, ^, $, IN, CONTAINS` - Comparison operators 4. `AND` - Logical AND 5. `OR` - Logical OR Use parentheses to make precedence explicit: `(A OR B) AND C` ## Common Patterns ```oql theme={null} # Has all these tags tags CONTAINS ALL (smoke, critical, login) # Has any of these tags tags CONTAINS ANY (smoke, regression) # Has this tag but not that tag tags CONTAINS "smoke" AND tags NOT CONTAINS "deprecated" ``` ```oql theme={null} # This month created_at >= startOfMonth() # Last 7 days created_at >= -7d # Specific range created_at BETWEEN 2024-01-01 AND 2024-12-31 # Before a date created_at < 2024-06-01 ``` ```oql theme={null} # Any of these priorities priority IN (p0, p1, p2) # Higher priority than P2 priority < p2 # Exclude low priorities priority NOT IN (p3, p4) ``` ```oql theme={null} # Contains word title ~ "login" # Starts with identifier ^ "TC-10" # Ends with title $ "test" # Multiple words (any) title ~ "login" OR title ~ "authentication" # Multiple words (all) title ~ "login" AND description ~ "valid credentials" ``` ## Validation Rules * Field names must exist in the schema * Operators must match field types (can't use `~` on numbers) * Date formats must be valid ISO 8601 * Array operators only work on array fields * Priority values must be valid (p0-p4) ## Error Messages Common errors and how to fix them: | Error | Cause | Fix | | --------------------------------- | ------------------------ | ------------------------------------------------------------------- | | `Unknown field: xyz` | Field doesn't exist | Check [field reference](/oql/field-reference) | | `Invalid operator for field type` | Wrong operator for field | Use comparison operators for numbers, pattern operators for strings | | `Invalid date format` | Bad date syntax | Use ISO 8601: `2024-01-15` or relative: `-7d` | | `Expected AND/OR` | Missing logical operator | Add `AND` or `OR` between conditions | | `Unclosed quote` | Missing closing quote | Ensure all strings have matching quotes | ## What's Next? Real-world OQL examples All searchable fields # DeepSeek Harness Desktop Source: https://onetest.ai/products/dsh-desktop A native macOS shell for DeepSeek Harness # A desktop home for DeepSeek Harness DeepSeek Harness Desktop runs the Harness Web UI as a native macOS application. It starts and monitors the harness process, discovers its port, and loads the interface in a native window. It is a shell—not a fork. A local Harness checkout stays untouched and can continue to follow upstream normally. DeepSeek Harness Desktop workspace with AI conversation, code editor, file tree, and browser

DeepSeek Harness Desktop workspace with AI conversation, code editor, file tree, and browser

DeepSeek Harness Desktop is an unofficial community project by OneTest AI. It is not affiliated with, endorsed by, or supported by DeepSeek. Read setup instructions, inspect the source, and download available releases. Explore the upstream Harness project maintained through the OneTest fork. ## What it adds Run a local checkout or let the application install and pin a published Harness version under your DSH home directory. Use a menu-bar tray, global show/hide shortcut, lifecycle controls, and turn-completion notifications. Configure Harness plugins and local or remote Model Context Protocol servers from the application. Work with files in a Monaco editor and browse real web pages beside the Harness conversation. ## Availability The current packaging target is macOS on Apple Silicon. You can run the project from source, build a local application bundle, or use a release from GitHub when available. See the [project README](https://github.com/onetest-ai/dsh-desktop#readme) for requirements, installation, configuration, and known limitations. # Octobots Source: https://onetest.ai/products/octobots Repository-native planning and execution for AI coding teams # A project board that lives with the code Octobots is a VS Code extension for planning and running AI coding work. It turns a repository's `.octobots/` directory into a project board made of campaigns, missions, tasks, bugs, and workflows. There is no database or separate server. The YAML files are the board, making plans diffable, reviewable, portable across machines, and accessible to both people and CLI agents. Octobots campaign tree, mission acceptance criteria, workflow gates, and mission signals

Octobots campaign tree, mission acceptance criteria, workflow gates, and mission signals

The public repository is currently named `octoshell`, while the product and README use the name **Octobots**. This page follows the product name and links to the existing repository. Explore the VS Code extension, board model, workflows, architecture graph, and tokenomics packages. Learn how campaigns, missions, tasks, workflows, and agent gates fit together. ## What it provides Campaigns, missions, tasks, bugs, and attached documents remain in the repository instead of a temporary chat transcript. Every task carries a checkable definition of done that planning, QA, review, and completion gates can enforce. Install skills and hooks that let CLI agents plan work, execute tasks, record provenance, and run completion gates. Octograph finds files that change together, while Tokenomics reports tokens, cost, turns, and activity per mission. ## Board structure ```text theme={null} .octobots/campaigns//campaign.yaml /missions//mission.yaml /tasks//task.yaml /bugs//bug.yaml /workflows//workflow.js ``` Because children are derived from folders and status belongs to each entity, multiple agents can add or update separate work without rewriting a shared index. # Octodeck Source: https://onetest.ai/products/octodeck Typed, interactive presentations for people and AI agents # Deck-as-code for people and agents Octodeck is a TypeScript and Vite framework for building presentation-like webpages. Slides are typed components, so they are hot-reloadable, interactive, version-controlled, and programmable like the rest of a software project. A DeepSeek Harness plugin lets an agent create and edit a deck while it renders live beside the conversation. TypeScript slide code beside a rendered interactive presentation

TypeScript slide code beside a rendered interactive presentation

Explore the framework, examples, themes, and Harness plugin. Use the `@onetest/dsh-deck` plugin to author presentations in a Harness session. ## What makes it different Slides are ordinary TypeScript components rather than opaque objects stored in a presentation file. Use events, timers, fetched data, diagrams, fragments, and custom components directly in a slide. Compose templates, layouts, content components, diagrams, and themes through a shared visual contract. Export a deck as one self-contained HTML file with its scripts, styles, fonts, and assets included. ## Quick start ```bash theme={null} npm install npm run dev npm run new:deck -- my-deck ``` The development server provides live reloading. A single-deck build can produce an offline HTML artifact that opens in any browser. # OneTest TMS Source: https://onetest.ai/products/tms GitHub-native test management for people and agents # Test management that belongs in the repository OneTest TMS is a GitHub-native system for managing test cases, test runs, automated results, coverage, and reports without introducing a separate backend. Test assets remain as files. Runs and coordination use GitHub Issues and Projects. Automation uses GitHub Actions, while an MCP server gives coding agents a structured way to operate the same workflow. Test files connected to a GitHub test run and verified coverage report

Test files connected to a GitHub test run and verified coverage report

Read the documentation, inspect the implementation, and follow development. Access the existing OneTest test-management application. ## How it works Test cases live in `tests/`, with project configuration in `.onetest/`. They can be reviewed, versioned, and changed with the product code. Issues and Projects provide coordination, while committed reports preserve results and history. The `onetest-tms` MCP server turns agent and CLI actions into GitHub operations and committed artifacts. Ingest automated results, correlate them with test assets, and report coverage through standard developer workflows. ## Requirements The GitHub-native TMS uses tools already common in development environments: * Node.js 20 or newer * GitHub CLI (`gh`), authenticated * Git * Bash * Python 3 ## Try it ```bash theme={null} npx -y @onetest/tms ``` The project works today through its MCP server and CLI. See the [repository documentation](https://github.com/onetest-ai/tms) for provisioning, authoring cases, creating runs, recording results, and reporting coverage. # Testing Best Practices Source: https://onetest.ai/resources/best-practices Proven strategies for effective software testing This page is under construction. More content coming soon! ## Core Testing Principles Catch bugs when they're cheapest to fix—during development Automate stable, repetitive tests. Keep exploratory testing manual. Focus on critical functionality and user journeys, not every edge case Clear, simple tests are easier to maintain than clever, complex ones ## Writing Effective Test Cases ### Write Clear, Unambiguous Tests **Good test case characteristics:** * **Specific**: No room for interpretation * **Repeatable**: Anyone can execute and get same result * **Self-contained**: Includes all necessary information * **Actionable**: Clear steps to follow ```markdown ✅ Good Example theme={null} **Title**: Verify user can login with valid credentials **Preconditions**: - User account exists: test@example.com / Password123! - User is not currently logged in **Steps**: 1. Navigate to https://app.example.com/login → Login page displays with email and password fields 2. Enter email: test@example.com → Email field accepts input 3. Enter password: Password123! → Password is masked with dots 4. Click "Login" button → User is redirected to dashboard at /dashboard → Welcome message displays: "Welcome, Test User" ``` ```markdown ❌ Bad Example theme={null} **Title**: Login test **Steps**: 1. Go to the site 2. Login 3. Check if it works ``` ### One Test, One Purpose **Do**: Test one thing per test case ```markdown theme={null} ✅ Test: Verify user can login with valid credentials ✅ Test: Verify error message for invalid password ✅ Test: Verify error message for non-existent user ``` **Don't**: Test multiple things in one test ```markdown theme={null} ❌ Test: Verify entire user journey from signup to purchase ``` **Why**: When the test fails, you immediately know what broke. Large tests are hard to debug and maintain. If your test title includes "and", it's probably doing too much. ### Make Tests Independent **Each test should:** * Set up its own test data * Not depend on other tests running first * Clean up after itself (or use fresh test data) * Be runnable in any order **Example:** ```markdown theme={null} ❌ Bad: Test depends on previous test Test 1: Create user account Test 2: Login with account from Test 1 ← Fails if Test 1 is skipped ✅ Good: Each test is independent Test 1: Verify user registration - Creates test user - Verifies creation - Cleans up Test 2: Verify login with valid credentials - Uses pre-existing test account - Doesn't depend on Test 1 ``` ### Use Good Test Data **Test data best practices:** 1. **Realistic but not real**: Use fake data that looks real * ✅ `john.smith@example.com` * ❌ Real customer email addresses 2. **Descriptive**: Data should indicate its purpose * ✅ `test_user_valid@example.com` * ✅ `expired_credit_card_4111111111111111` 3. **Documented**: Include test data in test case ```markdown theme={null} **Test Data**: - Username: test@example.com - Password: ValidPassword123! - Expected result: Successful login ``` 4. **Isolated**: Don't share mutable test data between tests 5. **Fresh**: Use data that won't be affected by other tests ## Test Organization **Test cases**: Start with action verb * ✅ "Verify user can checkout with credit card" * ❌ "Checkout test #42" **Folders**: Group by feature or user journey * ✅ `/Authentication/Login/` * ❌ `/Tests/Stuff/More Stuff/` **Tags**: Be consistent * ✅ `smoke`, `regression`, `critical` * ❌ `smoke`, `Smoke`, `SMOKE`, `smoke-test` **Use priority levels:** * **P0/Critical**: Blocks releases if failing * **P1/High**: Important functionality * **P2/Medium**: Standard features * **P3/Low**: Nice to have * **P4/Trivial**: Optional **Run tests by priority:** ```oql theme={null} priority = p0 # Run first priority IN (p0, p1) # Smoke suite priority IN (p0, p1, p2) # Full regression ``` **Ensure coverage of:** * All critical user journeys * All P0/P1 requirements * Common error scenarios * Edge cases for critical features **Use OQL to find gaps:** ```oql theme={null} # High priority features without tests requirements ~ "US-" AND priority IN (p0, p1) AND test_count = 0 # Features not tested recently status = active AND last_run < -30d AND priority IN (p0, p1) ``` ## Automation Strategy ### Automate These Tests **Good candidates for automation:** * ✅ Smoke tests (run after every deployment) * ✅ Regression tests (run before release) * ✅ API tests (fast, stable, repeatable) * ✅ Data-driven tests (same steps, different data) * ✅ Tests run frequently (>5 times per week) * ✅ Stable tests (pass rate >95%) **Keep these manual:** * ❌ Exploratory testing * ❌ Usability testing * ❌ Visual design testing * ❌ Tests that change frequently * ❌ Tests run rarely * ❌ Tests with complex verification ### Test Automation Pyramid Follow the testing pyramid for balanced automation: ``` /\ /UI\ 10% - Slow, brittle, expensive /----\ / API \ 30% - Faster, more stable /------\ / Unit \ 60% - Fast, cheap, reliable /----------\ ``` **Guidelines:** * **Unit tests**: Most of your tests (60%) * **API/Integration tests**: Moderate amount (30%) * **UI tests**: Smallest amount (10%) **Why**: Lower layers are faster, more reliable, and cheaper to maintain. ### Keep Automation Healthy **Regular maintenance:** * Fix failing tests immediately * Remove flaky tests (fix or delete) * Update for product changes * Refactor for clarity * Review every sprint **Signs of poor automation:** * Tests fail for no reason * Team ignores failures * Long execution times (>1 hour) * Hard to debug failures * Requires "test environment expert" to run If your team stops trusting automated tests, they become worthless. Maintain quality rigorously. ## Test Execution **Smoke tests**: After every deployment * Quick verification (15-30 min) * Critical paths only * Blocks further testing if fails **Regression tests**: Before releases * Comprehensive validation (2-4 hours) * All important features * Blocks release if P0/P1 fail **Exploratory tests**: Ongoing * Ad-hoc testing * New features * Edge cases * Unusual scenarios **Automated tests**: Continuously * On every commit (unit tests) * Nightly (full regression) * Pre-merge (affected tests) **Environment strategy:** 1. **Development**: Developer local testing 2. **QA/Test**: QA team testing 3. **Staging**: Pre-production validation 4. **Production**: Smoke tests only **Best practices:** * Keep staging identical to production * Use production-like data (anonymized) * Isolate test data from production * Refresh test environments regularly * Document environment differences **When a test fails:** 1. **Reproduce**: Can you reproduce the failure? * Yes → Investigate * No → Might be flaky, investigate further 2. **Classify**: What type of failure? * Real bug → Create defect * Test issue → Fix the test * Environmental → Check environment * Known issue → Link to existing bug 3. **Document**: Add details: * Screenshots * Logs * Steps to reproduce * Environment details 4. **Act**: Take appropriate action: * Block release if P0/P1 * Assess risk if P2/P3 * Fix test if test issue ## Team Collaboration Everyone owns quality: devs write unit tests, QA writes integration tests, product defines acceptance criteria Use test results to communicate: pass rate, coverage, trends, risks Review test cases like code: check for clarity, completeness, maintainability Document testing strategies, share test patterns, train new team members ## Common Anti-Patterns Avoid these testing mistakes: **Problem**: Trying to test every possible scenario **Why it's bad**: Wastes time, most tests add little value **Solution**: Focus on: * Critical user journeys * High-risk areas * Recently changed code * Areas with frequent bugs **Problem**: Automating all tests blindly **Why it's bad**: Some tests cost more to automate than run manually **Solution**: Automate selectively based on: * Frequency of execution * Stability of feature * Cost of automation vs manual * ROI of automation **Problem**: "Oh, that test is flaky, just re-run it" **Why it's bad**: Erodes trust in test suite, masks real issues **Solution**: * Fix flaky tests immediately * If can't fix, remove from suite * Never accept "sometimes it fails" **Problem**: Writing tests but never updating them **Why it's bad**: Tests become outdated, irrelevant, or broken **Solution**: * Review tests quarterly * Update for product changes * Remove obsolete tests * Fix broken tests immediately **Problem**: Only testing at the end of development **Why it's bad**: Bugs are expensive to fix late in cycle **Solution**: * Test during development * Write tests first (TDD) * Review requirements before coding * Continuous testing in CI/CD ## Metrics That Matter Track these metrics to improve testing: Target: >95% for regression suite % of features with tests Bugs found in test vs production Time from code to tested % of tests automated Target: \<2% ## Further Reading Learn about OneTest's features Critical path testing workflow Comprehensive testing workflow Work together on testing ## Recommended Books By Cem Kaner, James Bach, and Bret Pettichord Classic book with 293 lessons about software testing By Lisa Crispin and Janet Gregory Testing in agile environments By Elisabeth Hendrickson Exploratory testing techniques By Glenford J. Myers Fundamentals of software testing (classic) # TMS Frequently Asked Questions Source: https://onetest.ai/resources/faq Common questions about OneTest TMS ## General Questions OneTest TMS helps teams create, organize, execute, and maintain tests. It combines test management with AI assistance, natural-language search, execution tracking, and reporting. **Key features:** * AI-powered test generation * Powerful OQL query language * Test execution tracking * Team collaboration tools * Artifacts and evidence storage **What makes OneTest unique:** * **AI-First Design**: Generate complete test cases by chatting with AI—no manual writing required * **OQL Query Language**: Search tests like you search issues in JIRA * **Modern Interface**: Clean, intuitive UI built for modern teams * **Open Architecture**: Integrations with your existing tools * **All-in-One**: Test management + artifacts + execution in one platform **No!** OneTest is designed for both technical and non-technical users: * **Manual testers**: Create and execute tests through the UI * **QA engineers**: Use OQL for advanced queries, integrate with CI/CD * **Developers**: Write automated tests, use API * **Product managers**: Review test coverage, track quality The AI Assistant makes it even easier—just describe what you want to test in plain English. Visit [onetest.ai/pricing](https://onetest.ai/pricing) for current pricing information. **Free trial:** 14 days, no credit card required **Plans:** Starter, Professional, Enterprise **Billing:** Monthly or annual OneTest uses a coin-based system for API usage tracking: * **Browser usage is free** — all UI actions cost zero coins * **API key usage is metered** — each API call costs 1 coin * **Weekly budget** — each product gets 1,000 coins per week (configurable) * **Budget resets** automatically each week When coins run out, API requests are blocked until the budget resets or you top up. UI access is never affected. See [Usage & Billing](/settings/usage-billing) for details. Use **API keys** for programmatic access: 1. Go to **Settings** > **API Keys** 2. Create a new key (shown only once — save it!) 3. Include in requests: `Authorization: Bearer ak_YOUR_KEY` API keys work with all OneTest services (test management, artifacts, receiver, etc.). Each API call costs 1 coin from your weekly budget. ## Getting Started **Easiest way (AI):** 1. Press `Cmd/Ctrl + K` to open AI chat 2. Type: "Create a test for user login with valid credentials" 3. Review the generated test 4. Click Save **Manual way:** 1. Go to Test Management 2. Click **"+ New Test"** 3. Fill in title, description, and steps 4. Click **Save** See [Your First Test](/getting-started/your-first-test) for detailed walkthrough. 1. Go to **Settings** → **Members** 2. Click **"+ Invite Member"** 3. Enter email address 4. Select role (Owner or Member) 5. Click **Send Invite** They'll receive an email invitation to join your product. **Import:** OneTest supports importing tests from: * **Excel spreadsheets (XLSX)** — direct import with collision handling (skip, overwrite, or assign new IDs) * **ZIP archives** — full backup with folders, tags, and markdown files * **AI-powered Pipelines** — multi-phase ingestion for large imports with field mapping and decision gates * **JUnit XML files** — automated results via Integrations and API Keys * **ReportPortal agents** — drop-in compatible **Export:** Export test cases to Excel (XLSX) or ZIP with flexible options: * Export all test cases * Export OQL-filtered results (e.g., `priority = p1 AND status = ready`) * Export selected tests (checkbox selection) * Export unassigned tests (not in any folder) See [Import & Export](/ui/import-export) and [Pipelines](/ui/pipelines) for details. OneTest provides **Integrations and API Keys** — ReportPortal-compatible API keys for CI/CD integration: 1. Go to **Settings** > **Integrations and API Keys** 2. Click **+ Add API Key** 3. Configure your CI/CD pipeline: ```bash theme={null} RP_ENDPOINT=https://tms.onetest.ai/api/receiver RP_PROJECT= RP_API_KEY= ``` OneTest also supports JUnit XML import via `POST /api/v1/products/{product_uuid}/import/junit`. See [Integrations and API Keys](/settings/api-integrations) for detailed setup. ## AI Assistant The AI Assistant uses Claude (by Anthropic) to understand your testing needs and generate complete test cases. **It can:** * Generate single tests or entire test suites * Search existing tests using natural language * Answer questions about your product * Suggest test improvements * Extract test cases from requirements **It understands context:** * Knows which product you're working on * Sees your current page and selections * Learns from your existing tests * Adapts to your testing style **No.** OneTest uses Claude's API with strict data protections: * Your data is never used for training * Conversations are encrypted * Data stays in your region * You control data retention See our [Privacy Policy](https://onetest.ai/privacy) for details. **Yes!** You can customize: * **Personality**: Technical, friendly, concise, etc. * **Detail level**: Brief or comprehensive test cases * **Naming style**: Your preferred test naming conventions * **Prompt templates**: Save prompts for common scenarios AI behavior is configured through **LLM Configuration** in Settings. Go to [Settings → LLM Configuration](/settings/llm-configuration) to customize. The AI is very capable but not perfect: **Always review generated tests:** * Check steps make sense * Verify expected results * Add missing details * Adjust to your standards **Improve AI output:** * Be specific in your prompts * Provide more context * Use examples * Give feedback on generated tests The AI learns from your edits and improves over time. ## OQL Query Language **OQL (OneTest Query Language)** is a simple but powerful query language for finding tests, similar to JQL in JIRA. **Example:** ```oql theme={null} status = active AND priority IN (p0, p1) AND tags CONTAINS "smoke" ORDER BY created_at DESC ``` See [OQL Overview](/oql/overview) for details. **Not required, but recommended** for power users: * **Basic users**: Use simple filters in the UI * **Advanced users**: Learn OQL for complex queries * **AI users**: Ask AI to write OQL for you! **Example with AI:** ``` "Show me high priority tests that failed last week" ``` AI generates: `priority IN (p0, p1) AND status = failed AND last_run >= -7d` **Yes!** Save frequently-used queries: 1. Write your OQL query 2. Click **"Save Query"** 3. Give it a name (e.g., "Critical Failed Tests") 4. Access from "Saved Queries" dropdown Saved queries are shared with your team. ## Test Execution **Three ways to run tests:** 1. **Individual test**: Open test → Click "Run Test" 2. **Test run**: Select multiple tests → Click "Create Run" 3. **Automated**: Schedule tests or trigger from CI/CD See [Running Tests](/ui/running-tests) for details. **Yes!** OneTest supports: * Manual test execution (click through UI) * Automated test execution (API/Selenium/Playwright) * Hybrid approach (some manual, some automated) **For automation:** * Mark tests as "automated" * Link to automation test ID * Execute via API * Track automated results **During test execution:** 1. Execute a test step 2. Click **"Add Evidence"** 3. Upload screenshot or drag & drop 4. Add optional description 5. Continue with next step **Supported formats:** PNG, JPEG, GIF, PDF **Storage:** Stored in Artifacts service **Yes!** Distribute tests across team members: 1. Create a test run 2. Assign different tests to different team members 3. Everyone executes their assigned tests simultaneously 4. Monitor progress on run dashboard This significantly speeds up test execution. ## Troubleshooting **Possible causes:** * Too many steps per test * Complex test data setup * Slow environment * Network issues **Solutions:** * Break large tests into smaller tests * Optimize test data * Use faster environment * Check network connectivity * Consider automation for repetitive tests **Try these:** 1. Check filters—clear all filters and try again 2. Check folder—tests might be in subfolder 3. Check status—test might be archived 4. Use OQL: `title ~ "your search term"` 5. Use AI: "Find tests related to login" **Check:** 1. **LLM Configuration** — ensure you've set up an AI provider in [Settings > LLM Configuration](/settings/llm-configuration) 2. Internet connection 3. Browser console for errors 4. Try refreshing the page 5. Check system status page If problem persists, contact support. The AI Assistant requires an LLM provider: 1. Go to **Settings** > **LLM Configuration** 2. Select a provider: **Azure OpenAI**, **AWS Bedrock**, **OpenAI**, or **Anthropic** 3. Enter your API credentials 4. Configure model tiers (High for reasoning, Default for general, Low for fast tasks) 5. Click **Test Connection** then **Save Configuration** See [LLM Configuration](/settings/llm-configuration) for detailed instructions. **MCP (Model Context Protocol)** is a protocol that lets AI coding assistants (like Cursor, Windsurf, or Claude Code) connect to OneTest services directly from your editor. OneTest exposes MCP endpoints for each of its services (test management, artifacts, credpool, etc.). You configure your AI assistant to connect to these endpoints using your OneTest API key. **Setup options:** 1. **Automatic (recommended):** Run `npx github:onetest-ai/qa-agent init` and follow the browser-connect flow — it configures everything for you. 2. **Manual:** Edit your `.mcp.json` file directly to add the OneTest MCP endpoints. See [MCP Connectivity](/settings/mcp-servers) for configuration details and endpoint reference. 1. Go to login page 2. Click **"Forgot Password?"** 3. Enter your email 4. Check email for reset link 5. Follow link to set new password Or contact support if you need help. ## Support [artem@onetest.ai](mailto:artem@onetest.ai) Browse our comprehensive docs Report bugs or request features Check system status ## Still have questions? Our team is here to help! Email us at [artem@onetest.ai](mailto:artem@onetest.ai) # Integrations and API Keys Source: https://onetest.ai/settings/api-integrations Configure API keys for CI/CD pipelines to push automated test results Integrations and API Keys allow external tools like **ReportPortal agents**, **JUnit reporters**, and **CI/CD pipelines** to push test results directly into OneTest. They're fully compatible with the ReportPortal v2 API. ## Setting Up Go to **Settings** > **Integrations and API Keys** from the sidebar. Click **+ Add API Key**, give it a name (e.g., "CI Pipeline" or "Staging"), and save. The key is shown only once — copy it immediately. Use the generated API key in your CI/CD configuration: ```bash theme={null} # ReportPortal Agent Configuration RP_ENDPOINT=https://tms.onetest.ai/api/receiver RP_PROJECT= RP_API_KEY= ``` Your **Product UUID** is displayed on the Integrations and API Keys page and can be copied with one click. Execute your automated tests. Results will automatically appear in your **Test Runs** dashboard alongside manual test runs. ## API Endpoints OneTest provides ReportPortal v2 compatible endpoints: | Endpoint | Description | | --------------------------------------------------- | ------------------------ | | `POST /api/v2/{product_uuid}/launch` | Start a new test launch | | `POST /api/v2/{product_uuid}/item` | Report a test item | | `POST /api/v2/{product_uuid}/log` | Add a log entry | | `POST /api/v1/products/{product_uuid}/import/junit` | Import JUnit XML results | ## Managing API Keys Each API key has: * **Status** — Active or expired * **Retention period** — How long test results are kept (default: 90 days) * **Expiration date** — When the key expires * **Allowed Scopes** — Granular permissions controlling which APIs this key can access * **Last Active** — When the key was last used You can **regenerate** or **delete** keys at any time. Deleting a key immediately revokes access for any pipelines using it. API keys are shown only once when created. Store them securely in your CI/CD secrets (e.g., GitHub Secrets, GitLab CI Variables). ## JUnit XML Import For frameworks that don't support ReportPortal agents, you can import JUnit XML files directly: ```bash theme={null} curl -X POST "https://tms.onetest.ai/api/v1/products/{product_uuid}/import/junit" \ -H "Authorization: Bearer {api_key}" \ -F "file=@test-results.xml" ``` Each API call costs 1 coin from your weekly budget. Browser UI usage is always free. See [Usage & Billing](/settings/usage-billing) for details. ## What's Next? See your automated test results alongside manual runs Monitor API usage and coin consumption # LLM Configuration Source: https://onetest.ai/settings/llm-configuration Configure AI providers and model tiers for the OneTest AI Assistant The AI Assistant requires an LLM provider to be configured before it can generate tests, answer questions, or perform analysis. OneTest supports multiple providers so you can use your existing AI infrastructure. LLM Configuration page showing AWS Bedrock setup

LLM Configuration page showing AWS Bedrock setup

The AI Assistant will not work until you configure an LLM provider. This is the first step to enabling AI features in OneTest. ## Supported Providers **Required fields:** * Access Key ID * Secret Access Key * Region (e.g., `us-east-1`, `eu-central-1`) **Recommended models:** * High-Tier: `anthropic.claude-sonnet-4-5-20250929-v1:0` * Default: `anthropic.claude-sonnet-4-5-20250929-v1:0` * Low-Tier: `anthropic.claude-haiku-4-5-20251001-v1:0` **Required fields:** * API Key **Recommended models:** * High-Tier: `gpt-4o` * Default: `gpt-4o` * Low-Tier: `gpt-4o-mini` **Required fields:** * API Key **Recommended models:** * High-Tier: `claude-sonnet-4-5-20250929` * Default: `claude-sonnet-4-5-20250929` * Low-Tier: `claude-haiku-4-5-20251001` **Required fields:** * API Key * Endpoint URL * Deployment Name Use your Azure OpenAI deployment names for each model tier. ## Model Tiers OneTest uses three model tiers for different task complexities: | Tier | Purpose | Examples | | ------------------------- | --------------------------------------------------- | ------------------------------------------------------------- | | **High-Tier** (reasoning) | Complex analysis, test generation from requirements | Generating comprehensive test suites, analyzing coverage gaps | | **Default** (balanced) | General-purpose tasks | Answering questions, searching tests, simple generation | | **Low-Tier** (fast) | Quick, simple operations | Summarization, classification, simple lookups | ## Setup Steps Go to **Settings** > **LLM Configuration** from the sidebar. Click your preferred provider (Azure, AWS Bedrock, OpenAI, or Anthropic). Fill in the required credentials for your chosen provider. Enter model identifiers for each tier. Use the recommended models above or your own deployments. Click **Test Connection** to validate your credentials and model access. Click **Save Configuration** to activate the AI Assistant. Once configured, you can manage your LLM setup with **Revalidate** (test again), **Edit** (change settings), or **Delete** (remove configuration). ## What's Next? Start using the AI Assistant to generate and search tests Connect CI/CD pipelines and external tools via API keys # MCP Connectivity Source: https://onetest.ai/settings/mcp-servers Connect AI coding assistants to OneTest services via Model Context Protocol OneTest exposes MCP (Model Context Protocol) endpoints so AI coding assistants — such as Cursor, Windsurf, Claude Code, or any MCP-compatible tool — can interact with OneTest services directly from your editor. ## Available Endpoints Each OneTest service has its own MCP endpoint at `/mcp/{service-name}`: | Service | Endpoint | | --------------- | -------------------------------------------- | | Test Management | `https://tms.onetest.ai/mcp/test-management` | | Artifacts | `https://tms.onetest.ai/mcp/artifacts` | | Membership | `https://tms.onetest.ai/mcp/membership` | | Metering | `https://tms.onetest.ai/mcp/metering` | | Credential Pool | `https://tms.onetest.ai/mcp/credpool` | ## Authentication All MCP endpoints authenticate via API key using a Bearer token. Include your OneTest API key in the `Authorization` header: ``` Authorization: Bearer ak_YOUR_API_KEY ``` To get an API key, go to **Settings** > **API Keys**. ## Setup ### Option 1: Automatic (Recommended) Run the QA Agent init command and follow the browser-connect flow. It detects your editor, authenticates you, and writes the MCP configuration automatically: ```bash theme={null} npx github:onetest-ai/qa-agent init ``` ### Option 2: Manual `.mcp.json` Add OneTest endpoints directly to your `.mcp.json` configuration file. The example below connects to the Test Management service: ```json theme={null} { "mcpServers": { "onetest": { "url": "https://tms.onetest.ai/mcp/test-management", "headers": { "Authorization": "Bearer ak_YOUR_API_KEY" } } } } ``` Add additional entries for other services (artifacts, credpool, etc.) as needed. The `npx github:onetest-ai/qa-agent init` flow is the easiest way to get started — it handles authentication and writes the correct `.mcp.json` entries for all services. ## What's Next? Use the AI with your connected tools Configure the AI provider powering the assistant # Product Settings Source: https://onetest.ai/settings/products Manage environments, releases, sprints, builds, and integrations Product settings let you configure the context around your testing: environments where tests run, releases being tested, sprint cycles, and external integrations. Product Settings page

Navigate to **Settings** > **Manage Products** to access these settings. ## Environments Environments define where your tests are executed (e.g., Development, Staging, Production). Click **+ Environment** and provide a name and URL. When creating test runs, select the target environment to track which environment was tested. ## Releases Releases track software versions being tested. * Click **+ Release** to create a new release * Assign a version number and description * Link test runs to releases for coverage tracking ## Sprints Sprints help organize testing around agile development cycles. * Sprints are created under a release * Track test progress per sprint * View sprint-level quality metrics Sprint and Build creation buttons become active after you create at least one Release. ## Builds Builds track specific CI/CD build versions. * Created under a sprint * Link automated test results to specific builds * Track quality trends across builds ## Integrations ### Langfuse Connect [Langfuse](https://langfuse.com) for LLM observability — monitor AI Assistant usage, track token consumption, and analyze conversation quality. **Required fields:** * Langfuse Host URL * Public Key * Secret Key ## Danger Zone The Danger Zone contains irreversible actions like deleting the product and all its data. Use with extreme caution. ## What's Next? Create test runs linked to environments and releases Monitor product usage and billing # Usage & Billing Source: https://onetest.ai/settings/usage-billing Plans, coins, storage limits, and how billing works in OneTest OneTest uses a **coin-based** system to meter AI-powered operations. Creating test cases, running tests, and managing your product through the UI is always free — coins only apply to AI operations like pipeline runs, MCP tool calls, and file uploads. ## Plans OneTest is billed **per product** — each product you create can be on its own plan. Upgrade individual products as your team grows. | | **Free** | **Pro** | **Team** | **Enterprise** | | ----------------------- | --------- | ------------------ | ------------------- | ------------------ | | **Price** | \$0 | \$9 / product / mo | \$19 / product / mo | Custom | | **Members included** | 3 | 10 | 30 | Custom | | **Extra members** | — | +5 seats = \$4/mo | +10 seats = \$6/mo | Custom | | **Coins / month** | 500 | 5,000 | 15,000 | Custom | | **Coin top-ups** | — | \$5 / 500 coins | \$5 / 500 coins | Custom | | **Max file size** | 10 MB | 10 MB | 10 MB | 10 MB | | **Storage per product** | 100 MB | 5 GB | 15 GB | Custom | | **Extra storage** | — | \$1 / GB / mo | \$1 / GB / mo | Custom | | **Data retention** | 90 days | 90 days | 90 days | 90 days + archival | | **Audit logs** | — | — | ✅ | ✅ | | **SSO (SAML/OIDC)** | — | — | — | ✅ | | **Self-hosted** | — | — | — | ✅ | | **Support** | Community | Email | Priority email | Dedicated SLA | Billing is per product, not per workspace. A team working on three products on the Pro plan pays \$27/month total. ### Why Team is Better Value The Team plan gives you the same resources as three stacked Pro products at a lower price: ``` 3 × Pro = $27/mo → 30 members, 15,000 coins, 15 GB storage 1 × Team = $19/mo → 30 members, 15,000 coins, 15 GB storage ``` ### Extra Member Packs If you need more members without upgrading your plan, you can add member packs: * **Pro:** +5 members for \$4/month per pack * **Team:** +10 members for \$6/month per pack The math is designed so that stacking packs naturally becomes more expensive than upgrading: ``` Pro + 3 packs = $21/mo (25 members) → upgrade to Team at $19 makes more sense ``` *** ## Coins Coins are the unit of AI consumption in OneTest. Your monthly coin balance resets at the start of each billing cycle. ### What Costs Coins | Operation | Cost | | ---------------------------------------------- | ----------- | | MCP AI tool call | 1 coin | | File upload (screenshot, artifact, attachment) | 1 coin | | Pipeline run — ≤ 50 test cases extracted | 100 coins | | Pipeline run — \~100 test cases extracted | 200 coins | | Pipeline run — \~200 test cases extracted | 400 coins | | Pipeline run — \~500 test cases extracted | 1,000 coins | ### What is Always Free * All UI operations (create, edit, view, search) * Test run execution and result viewing * MCP server access (connection setup) * REST API reads (non-AI) * Member management ### Monthly Coin Budget | Plan | Coins / month | Approximate pipeline capacity | | ---- | ------------- | ----------------------------- | | Free | 500 | \~2 runs of 100 TCs | | Pro | 5,000 | \~25 runs of 100 TCs | | Team | 15,000 | \~75 runs of 100 TCs | Coins are shared across all users on a product. A product owner can monitor consumption and top up at any time from **Settings → Usage & Billing**. ### Topping Up Coins When you run out of coins mid-cycle, you can top up without waiting for the reset: Navigate to **Settings** → **Usage & Billing** for your product. Select the number of coin packs to add. Each pack is **500 coins for \$5**. Coins are added immediately to your available balance and expire at the end of the current billing cycle. Top-up coins are available on Pro and Team plans only. Free plan users must upgrade to access top-ups. *** ## Storage Each product has a storage quota for uploaded files — screenshots, attachments, test artifacts, and pipeline input files. | Plan | Storage per product | Extra storage | | ---------- | ------------------- | -------------------- | | Free | 100 MB | — (upgrade required) | | Pro | 5 GB | \$1 / GB / month | | Team | 15 GB | \$1 / GB / month | | Enterprise | Custom | Custom | **File size limit: 10 MB per file** across all plans. Storage is consumed by: * Files uploaded via the UI or API * Screenshots captured during test runs * Pipeline input files (Excel, CSV) * Test result attachments Uploads that exceed the product storage quota will be rejected. Free plan products cannot purchase additional storage — upgrade to Pro or Team to expand. *** ## Data Retention All plans retain data for **90 days**. After 90 days, test runs, executions, activity logs, and uploaded files are permanently deleted. Enterprise plans can configure extended archival retention on custom terms. Test cases, releases, environments, and product configuration are **never deleted** — only time-series data like runs, executions, and activity logs age out. *** ## Viewing Usage Navigate to **Settings** → **Usage & Billing** to monitor your product's consumption. ### Coin Balance At a glance you can see: * **Coins used** this billing cycle * **Coins remaining** * **Cycle reset date** * **Breakdown by operation type** ### Activity Log Every coin-consuming event is logged in real-time: * **User** who triggered the operation * **Operation type** (pipeline run, MCP call, file upload) * **Coin cost** * **Timestamp** Use date filters to narrow activity to a specific period. Export to CSV for reporting: 1. Go to **Settings** → **Usage & Billing** 2. Set the date range (up to 90 days) 3. Click **Export CSV** The export includes: timestamp, user, operation type, service, and coin cost. ### Storage Usage The storage panel shows: * **Total used** vs your plan quota * **Top consumers** by file type * **Recent uploads** with individual file sizes *** ## FAQ Pipeline runs and MCP AI tool calls will be blocked until you either top up coins or wait for the monthly reset. UI access, test execution, and result viewing are never affected. Free plan users must upgrade to top up — there is no overage on Free. No. Your coin balance resets at the start of each billing cycle. Top-up coins also expire at the end of the cycle in which they were purchased. Pipeline coins are based on the number of test cases **extracted** from your file, not the file size. After the pipeline analyses your file, it shows you the estimated coin cost before processing begins so you can confirm. No. Free products are capped at 100 MB with no option to purchase more. Upgrade to Pro ($9/product/month) to get 5 GB and the option to add extra storage at $1/GB/month. Billing is **per product**. Each product is its own billing unit with its own plan, coin balance, and storage quota. Members themselves are not billed — only the product owner is charged. Yes. The activity log records the user behind every coin-consuming event. Export to CSV and filter by user to see individual consumption. Enterprise is OneTest deployed on your own infrastructure. You control the data, retention policies, and user limits. Pricing is custom based on team size and usage. [Contact us](mailto:hello@onetest.ai) to discuss. # Automation Coverage Source: https://onetest.ai/ui/automation-coverage Understand your test automation gap and track CI/CD evidence The Automation Coverage report answers a critical question: **are the tests you marked as "automated" actually running in CI/CD?** Many teams mark test cases as automated when they write the code, but automation can silently break — tests get renamed, CI configs change, or entire suites get disabled. The coverage report surfaces these gaps. ## Accessing the Report Navigate to **Analytics** from the main navigation. The Automation Coverage dashboard loads automatically for the current product. ## Dashboard Overview The report has four sections: ### Summary Cards All non-archived test cases in the product Percentage of test cases with execution type set to "automated" Test cases with actual CI/CD execution evidence Test cases marked automated but with no CI/CD evidence (highlighted in red when > 0) ### Distribution Charts Two charts show how your test suite is composed: * **Execution Type** (pie chart) — Manual vs Automated split * **Test Category** (bar chart) — Breakdown by category: functional, performance, security, accessibility, exploratory ### Gap Analysis Bar The most important visualization. A stacked horizontal bar that breaks down your "marked automated" test cases into three segments: | Segment | Color | Meaning | | --------------------------- | ------ | ------------------------------------------------------------------------------------- | | **Linked to CI/CD** | Green | Test has matching execution results from your CI/CD pipeline | | **No test ref set** | Orange | Test is marked automated but has no `automation_test_id` configured — link is missing | | **Has ref, no CI/CD match** | Red | Test has an automation reference but it doesn't match any code\_ref in CI/CD results | ### Coverage by Priority & Category Two stacked bar charts showing automated vs manual distribution: * **By Priority** — Are your P0/P1 critical tests automated? * **By Category** — Which test categories have the best automation coverage? ## Understanding the Gap These test cases have confirmed execution evidence. OneTest matched the test case's `automation_test_id` to a `code_ref` in your CI/CD test results. **No action needed** — these tests are running. These test cases are marked as automated (`execution_type = automated`) but don't have an `automation_test_id` set. Without this reference, OneTest can't match them to CI/CD results. **Action:** Set the `automation_test_id` field on these test cases. The value should match the test's code reference in your framework: ``` test_login.py:TestLogin.test_valid_login ``` If you use pytest with pytest-reportportal, the code\_ref is automatically sent in the format `file.py:Class.method`. Set your `automation_test_id` to match this format. These are the real **automation gaps**. The test case has an `automation_test_id` but it doesn't match any `code_ref` in your CI/CD results. Common causes: * **Test was renamed** — the class or method name changed but `automation_test_id` wasn't updated * **Test was deleted** — the automation code was removed * **Test is disabled** — skipped or excluded from CI/CD runs * **Format mismatch** — `automation_test_id` uses `::` separator but CI/CD sends `:` (OneTest normalizes these, but edge cases exist) **Action:** Investigate each red-flagged test case. Update the `automation_test_id` to match the current test name, or change `execution_type` back to "manual" if the automation was removed. ## How Matching Works OneTest links test cases to CI/CD results by comparing two values: | Source | Field | Format | Example | | ----------------- | -------------------- | ------------------- | ------------------------------------------ | | **Test Case** | `automation_test_id` | `file:Class.method` | `test_login.py:TestLogin.test_valid_login` | | **CI/CD Results** | `code_ref` | `file:Class.method` | `test_login.py:TestLogin.test_valid_login` | When these match, the test case is "linked to results." ### Format Normalization OneTest automatically normalizes common format differences: | Input Format | Normalized | | -------------------------------------------- | ------------------------------------------ | | `test_login.py::TestLogin::test_valid_login` | `test_login.py:TestLogin.test_valid_login` | | `test_login.py:TestLogin.test_valid_login` | `test_login.py:TestLogin.test_valid_login` | The `code_ref` values come from your test reporting agent (e.g., pytest-reportportal). OneTest doesn't control this format — it adapts to what your CI/CD sends. ### Multiple References A single test case can map to multiple automated tests. When `automation_test_id` contains multiple references (one per line), a match on **any** reference counts as linked: ``` test_login.py:TestLogin.test_valid_login test_login.py:TestLogin.test_login_with_mfa test_login.py:TestLogin.test_login_rate_limit ``` ## Setting Up Automation References ### From the UI 1. Open a test case 2. Set **Execution Type** to "Automated" 3. Fill in the **Automation Test ID** field with the test's code reference 4. Save ### Via API ```bash theme={null} curl -X PUT \ https://tms.onetest.ai/api/test-management/api/v1/test-cases/{id} \ -H "Authorization: Bearer ak_YOUR_KEY" \ -H "Content-Type: application/json" \ -d '{ "execution_type": "automated", "automation_test_id": "test_login.py:TestLogin.test_valid_login" }' ``` ### From CI/CD (Automatic) When using pytest-reportportal or similar agents, test results include `code_ref` automatically. OneTest matches these to test cases with matching `automation_test_id` values. ## API Reference ``` GET /api/v1/products/{product_id}/test-cases/automation-coverage ``` Query parameters: * `no_cache=true` — Bypass the 5-minute cache for fresh data Response: ```json theme={null} { "total_test_cases": 500, "execution_type_distribution": { "manual": 174, "automated": 326 }, "marked_automated": 326, "linked_to_results": 245, "no_automation_ref": 46, "automation_gap": 35, "automation_gap_percentage": 10.7, "coverage_by_priority": [ { "priority": "p0", "total": 45, "automated": 40, "gap_percentage": 5.0 }, { "priority": "p1", "total": 120, "automated": 95, "gap_percentage": 8.4 } ], "coverage_by_category": [ { "category": "functional", "total": 300, "automated": 200, "gap_percentage": 12.0 }, { "category": "api", "total": 100, "automated": 85, "gap_percentage": 5.9 } ] } ``` ## Best Practices When you write automation code, immediately set the `automation_test_id` on the matching test case. Don't wait for CI/CD to catch up. Check the Automation Coverage report regularly. A growing red segment means automation is silently breaking. Use `file.py:Class.method` format for `automation_test_id`. This matches what pytest-reportportal sends as `code_ref`. Use the Coverage by Priority chart to ensure critical tests have the highest automation coverage and lowest gap. ## What's Next? Understand test execution results and trends Execute tests and report results to OneTest # CredPool Source: https://onetest.ai/ui/credpool Manage and distribute test credentials securely across your team CredPool is a credential vending machine for test environments. It lets teams register, share, and track test credentials without copying passwords into Slack or spreadsheets. ## Why CredPool? Testing often requires shared accounts (staging logins, API keys, service credentials). Without a system, teams face: * **Credential collisions** — two testers use the same account simultaneously * **Lost passwords** — scattered across notes, chats, and wikis * **No audit trail** — who used what, and when? CredPool solves this with a checkout/release model: credentials are checked out exclusively (or concurrently), with automatic expiry and a full audit log. ## Core Concepts Templates that define what a credential looks like — its fields (username, password, URL, API key, etc.) and which fields are sensitive. Named collections of interchangeable credentials. For example, a "Staging Jira Accounts" pool with 5 accounts that testers can check out. Individual credentials inside a pool. Each entry has values matching its pool's credential type. Values are encrypted at rest (AES-256-GCM). The core workflow: check out a credential, use it, release it when done. Locks prevent collisions. Auto-release prevents forgotten locks. ## Getting Started Go to **Test Data** > **CredPool** > **Credential Types** tab and click **Create Type**. Define the fields your credentials need: | Field | Type | Sensitive | Required | | ------------ | ------ | --------- | -------- | | Username | text | No | Yes | | Password | secret | Yes | Yes | | Instance URL | url | No | Yes | | Project Key | text | No | No | Field types: `text`, `secret`, `url`, `email`, `number`, `textarea`, `select` Fields marked **sensitive** are masked in the UI (shown as dots) until explicitly revealed. Go back to the **Pools** tab and click **Create Pool**. * **Name**: "Staging Jira Accounts" * **Credential Type**: Select the type you just created * **Environment**: staging * **Allow concurrent usage**: Off (exclusive checkout) * **Max lock time**: 240 minutes (4 hours) When concurrent usage is **off**, only one person can use each credential at a time. When **on**, multiple people can check out the same credential simultaneously. Open the pool and click **Add Entry**. Fill in the credential values: * **Label**: "Admin Account" (human-readable identifier) * **Username**: `svc_test_01` * **Password**: `xK9#mP2$vL` * **Instance URL**: `https://staging.jira.example.com` Add as many entries as you have accounts. More entries = less waiting for your team. Click **Get Random Credentials** on any pool card. CredPool picks an available entry, locks it to you, and displays the credential values. Copy what you need, then release when done. If you forget, auto-release kicks in after the max lock time. ## Dashboard The CredPool dashboard shows all pools as cards with real-time availability: Each pool card shows: * **Pool name** and environment * **Availability**: e.g., "3 of 5 available" * **Credential type** name * **Get Random Credentials** button for quick checkout Below the pool grid, **My Active Checkouts** lists credentials you currently have locked, with a countdown timer until auto-release. ## Checking Out Credentials The most common way to get credentials: 1. Find your pool on the dashboard 2. Click **Get Random Credentials** 3. CredPool picks any available entry and locks it to you 4. A dialog shows the decrypted credential values 5. Copy what you need (click the copy icon next to each field) Random checkout is the fastest path — you don't need to know which specific account to use. CredPool picks one for you. When you need a particular credential: 1. Open the pool detail page 2. Find the entry you need in the table 3. Click the lock icon on that row 4. The credential values are displayed This is useful when a test requires a specific account (e.g., an admin vs. read-only user). For CI/CD pipelines and automation scripts: ```bash theme={null} # Random checkout curl -X POST \ https://tms.onetest.ai/api/credpool/api/v1/products/{product_id}/credpool/pools/{pool_id}/checkout \ -H "Authorization: Bearer ak_YOUR_KEY" ``` Response: ```json theme={null} { "checkout_id": "log-uuid", "pool": { "id": "pool-uuid", "name": "Staging Jira Accounts" }, "entry": { "id": "entry-uuid", "label": "Admin Account" }, "credentials": { "username": "svc_test_03", "password": "xK9#mP2$vL", "instance_url": "https://staging.jira.example.com" }, "lock": { "locked_at": "2026-02-21T10:30:00Z", "expires_at": "2026-02-21T14:30:00Z", "auto_release": true } } ``` ## Releasing Credentials When you're done with a credential: * **From the dashboard**: Click the release button on your active checkout * **From pool detail**: Click the unlock icon on the locked entry * **Via API**: `POST /credpool/entries/{entry_id}/release` * **Automatically**: Credentials auto-release after the max lock time expires If you forget to release, the credential stays locked until the auto-release timer expires. Other team members will see it as unavailable during this time. ## Exclusive vs Concurrent Mode | Mode | Behavior | Use Case | | ----------------------- | -------------------------------------------------------------- | -------------------------------------------- | | **Exclusive** (default) | One person at a time per credential. Checkout locks the entry. | Accounts that break with concurrent sessions | | **Concurrent** | Multiple people can check out the same credential. No locking. | Read-only API keys, shared service accounts | Set the mode when creating a pool via the **Allow concurrent usage** toggle. ## Credential Types Credential types are reusable templates. Create them once, use across multiple pools. ### Field Types | Type | Renders As | Example | | ---------- | --------------------------- | ---------------------- | | `text` | Text input | Username, project key | | `secret` | Password input (masked) | Passwords, tokens | | `url` | URL input with validation | Instance URLs | | `email` | Email input with validation | Service account emails | | `number` | Numeric input | Port numbers | | `textarea` | Multi-line text | SSH keys, certificates | | `select` | Dropdown with options | Environment selector | ### Sensitive Fields Fields marked **sensitive** are: * Masked in the UI (shown as dots until revealed) * Encrypted at rest alongside all other values * Never logged in the audit trail Fields of type `secret` are automatically marked as sensitive. You can also mark `text` or `textarea` fields as sensitive for additional protection. ## Admin Features ### Active Locks View all currently locked credentials across all pools. Useful for understanding team capacity and identifying forgotten checkouts. ### Force Release Product owners and admins can force-release a locked credential when: * A team member is unavailable and their credential is needed * A CI pipeline crashed without releasing its checkout ### Audit Log Every checkout and release is logged with: * **Who** checked out / released * **When** the action occurred * **Which** credential (pool + entry) * **Action type**: random checkout, specific checkout, release, force-release, auto-expired * **IP address** of the client ## Encryption All credential values are encrypted at rest using **AES-256-GCM** (authenticated encryption). Each entry uses a unique initialization vector (IV), so identical values produce different ciphertext. Values are only decrypted when returned during a checkout response. ## API Reference ### Credential Types ``` POST /api/v1/products/{id}/credpool/types # Create type GET /api/v1/products/{id}/credpool/types # List types GET /api/v1/products/{id}/credpool/types/{id} # Get type PUT /api/v1/products/{id}/credpool/types/{id} # Update type DELETE /api/v1/products/{id}/credpool/types/{id} # Delete type ``` ### Pools ``` POST /api/v1/products/{id}/credpool/pools # Create pool GET /api/v1/products/{id}/credpool/pools # List pools GET /api/v1/products/{id}/credpool/pools/{id} # Get pool PUT /api/v1/products/{id}/credpool/pools/{id} # Update pool DELETE /api/v1/products/{id}/credpool/pools/{id} # Delete pool ``` ### Entries ``` POST /api/v1/products/{id}/credpool/pools/{id}/entries # Add entry GET /api/v1/products/{id}/credpool/pools/{id}/entries # List entries PUT /api/v1/products/{id}/credpool/entries/{id} # Update entry PATCH /api/v1/products/{id}/credpool/entries/{id}/status # Enable/disable DELETE /api/v1/products/{id}/credpool/entries/{id} # Delete entry ``` ### Checkout & Release ``` POST /api/v1/products/{id}/credpool/pools/{id}/checkout # Random checkout POST /api/v1/products/{id}/credpool/entries/{id}/checkout # Specific checkout POST /api/v1/products/{id}/credpool/entries/{id}/release # Release GET /api/v1/products/{id}/credpool/me/checkouts # My checkouts ``` ### Admin ``` GET /api/v1/products/{id}/credpool/admin/locks # Active locks POST /api/v1/products/{id}/credpool/admin/entries/{id}/force-release # Force release GET /api/v1/products/{id}/credpool/admin/audit-log # Audit log POST /api/v1/products/{id}/credpool/admin/release-expired # Cleanup expired ``` ## What's Next? Track which tests are actually running in CI/CD # Import & Export Source: https://onetest.ai/ui/import-export Import test cases from Excel or ZIP and export for sharing or backup OneTest supports importing and exporting test cases in multiple formats, making it easy to migrate from other tools or share test suites across teams. Import format selection dialog

## Importing Test Cases Click the **Import** button (upload icon) in the Test Cases toolbar, or go to **Pipelines** > **Import Test Cases** for bulk imports. Choose your import format: * **ZIP Archive** — Full restore with folders, tags, and markdown files * **Excel (XLSX)** — Spreadsheet with test cases, steps, folders, and tags Click **Choose File** or drag and drop your file into the upload area. Choose how to handle duplicate test cases: | Strategy | Description | | ---------------------- | -------------------------------------------- | | **Skip duplicates** | Leave existing test cases unchanged | | **Overwrite existing** | Create a new version of existing test cases | | **Assign new IDs** | Import all test cases with fresh identifiers | Click **Import** to begin. You'll see a summary of imported test cases when complete. Import upload step with collision handling

Import upload step with collision handling

For large or complex imports with hundreds of test cases, use [Pipelines](/ui/pipelines) instead. Pipelines provide AI-powered field mapping, decision gates, and multi-phase progress tracking. Pipeline import flow

## Exporting Test Cases Click the **Export** button (download icon) in the Test Cases toolbar. ### Export Formats | Format | Contents | | ---------------- | --------------------------------------------------------------------- | | **Excel (XLSX)** | Multi-sheet workbook with Test Cases, Steps, Folders, and Tags sheets | | **ZIP Archive** | Full backup with folder structure, tags, and markdown files | ### Export Scopes * **All test cases** — Export everything in the current product * **OQL-filtered results** — Export only test cases matching your current query (e.g., `priority = p1 AND status = ready`) * **Selected tests** — Export only test cases you've checked in the table * **Unassigned tests** — Export test cases not assigned to any folder ## Preparing Import Files Your Excel file should contain these sheets: * **Test Cases** — One row per test case with columns: Title, Description, Priority, Type, Status, Category, Tags * **Steps** (optional) — Test steps with columns: Test Case Title, Step Number, Action, Expected Result * **Folders** (optional) — Folder hierarchy with columns: Name, Parent Folder * **Tags** (optional) — Tag definitions with columns: Name, Color Export an existing test suite first to see the expected format, then use it as a template for your import file. ZIP archives should contain: * Markdown (`.md`) files for each test case * A `folders.json` file defining the folder structure * A `tags.json` file defining tags This format is ideal for full backup/restore scenarios. ## What's Next? Use AI-powered pipelines for complex bulk imports Organize and manage your imported test cases # Pipelines Source: https://onetest.ai/ui/pipelines Multi-phase test case ingestion with AI-powered field mapping and decision gates Pipelines provide an AI-powered, multi-phase process for importing large volumes of test cases from Excel or CSV files. Unlike [simple import](/ui/import-export), pipelines analyze your file structure, map columns intelligently, and let you review decisions at each stage. Pipelines page showing pipeline list

## Accessing Pipelines Navigate to **Tasks** in the sidebar to see all your pipelines. Each pipeline tracks the progress of a test case import from upload to completion. ## Creating a Pipeline Click **Import Test Cases** and upload an Excel (.xlsx) or CSV file containing test cases. The pipeline will analyze the file structure automatically. Pipeline upload dialog

The AI analyzes your spreadsheet columns and suggests mappings to OneTest fields (Title, Description, Priority, Steps, etc.). Review and adjust the mappings as needed, then click **Confirm** to proceed. Pipeline field mapping gate

The pipeline parses and validates your test cases based on the field mapping. Review any warnings or issues before proceeding. Pipeline analysis phase

Test cases are created in OneTest. The pipeline then presents **Review Results** — AI-detected quality findings such as: * **Flow continuations** — shorter test cases that overlap with longer ones and can be merged * **Contradictions** — tests with conflicting expected outcomes for the same action * **Merge proposals** — duplicate or redundant tests that should be combined For each finding, choose to merge, keep separate, or skip. Use **Confirm All** to accept the AI's recommendations in bulk. Pipeline execution review results with merge proposals

Pipeline execution review results with merge proposals

A comprehensive **Health Report** compares source file quality against the migrated result: * **Health Score** — overall quality rating (e.g., 78.1/100) * **TC Completeness** — field coverage improvements (description, priority, pass/fail criteria) * **Structural Quality** — naming consistency, step counts, tag coverage * **Quality Findings** — resolved overlaps and contradictions Pipeline health report with before/after comparison

Pipeline health report with before/after comparison

Your test cases are now available in Test Cases. Navigate there to review and organize them. ## Pipeline Phases Each pipeline progresses through these phases: | Phase | Description | | ---------------------- | --------------------------------------------------------------------------------------- | | **Uploading** | File is being uploaded and validated | | **Planning (Gate 1)** | Field mapping and configuration | | **Analyzing (Gate 2)** | AI parses and validates test cases | | **Executing (Gate 3)** | Test cases are imported into OneTest, with review of merge proposals and contradictions | | **Reporting** | Health report with before/after quality comparison | | **Completed** | Import finished successfully | | **Failed** | Import failed — check error details | ## Decision Gates At each gate, the pipeline may present **decision items** that require your input: * **Blocking items** — Must be resolved before the pipeline can advance * **Non-blocking items** — Can be resolved or skipped You can resolve items individually or use **Batch Resolve** to handle multiple items at once. ## Pipeline Actions | Action | Description | | ------------- | ------------------------------------------------ | | **Advance** | Move to the next phase after reviewing decisions | | **Cancel** | Stop the pipeline (imported data remains) | | **Resume** | Continue a paused pipeline | | **Rollback** | Undo the import and remove created test cases | | **Delete** | Remove the pipeline record | | **Duplicate** | Create a new pipeline with the same settings | ## When to Use Pipelines vs Simple Import | Feature | Simple Import | Pipeline | | ----------------- | ------------------------- | --------------------------- | | Best for | Small files, known format | Large files, unknown format | | Field mapping | Automatic (fixed format) | AI-powered with review | | Decision gates | No | Yes | | Progress tracking | Basic | Detailed per-phase | | Rollback | No | Yes | Use simple [Import & Export](/ui/import-export) for quick imports of files in the standard OneTest format. Use Pipelines when importing from external tools or dealing with hundreds of test cases. ## What's Next? Quick import for standard format files Manage your imported test cases # Reporting Automated Results Source: https://onetest.ai/ui/reporting-results Push automated test results from CI/CD pipelines into OneTest ## Overview OneTest accepts automated test results from any CI/CD pipeline. Results from your automated tests appear alongside manual and AI-driven test runs in a unified dashboard — giving you a single view of quality across all execution methods. There are three ways to report automated results: Drop-in compatible with pytest-reportportal, Java agent, and other RP agents Upload JUnit XML files from any test framework Use the OneTest MCP tools directly from Claude Code or any MCP client ## Prerequisites Before reporting results, you need an API key: 1. Go to **[tms.onetest.ai](https://tms.onetest.ai) -> Settings -> [Integrations and API Keys](/settings/api-integrations)** 2. Click **+ Add API Key** and copy the key immediately (it is shown only once) 3. Note your **Product UUID** displayed on the same page ## Method 1: ReportPortal Agents OneTest is fully compatible with the [ReportPortal](https://reportportal.io) v2 API. If your team already uses a ReportPortal agent, just point it at OneTest — no code changes required. ### Configuration Set these environment variables in your CI/CD pipeline: ```bash theme={null} RP_ENDPOINT=https://tms.onetest.ai/api/receiver RP_PROJECT= RP_API_KEY= ``` ### Supported Agents Any ReportPortal agent works out of the box: | Framework | Agent | | --------------- | ------------------------------------------------------------------------------------------ | | pytest | [pytest-reportportal](https://github.com/reportportal/agent-python-pytest) | | JUnit 5 | [agent-java-junit5](https://github.com/reportportal/agent-java-junit5) | | TestNG | [agent-java-testng](https://github.com/reportportal/agent-java-testng) | | Cypress | [agent-js-cypress](https://github.com/reportportal/agent-js-cypress) | | Playwright | [agent-js-playwright](https://github.com/reportportal/agent-js-playwright) | | Robot Framework | [agent-Python-RobotFramework](https://github.com/reportportal/agent-Python-RobotFramework) | ### Example: pytest ```ini theme={null} # pytest.ini [pytest] rp_endpoint = https://tms.onetest.ai/api/receiver rp_project = a1b2c3d4-e5f6-7890-abcd-ef1234567890 rp_api_key = ak_your_api_key_here rp_launch = Nightly Regression rp_launch_attributes = env:staging build:2.1.0 ``` ```bash theme={null} pytest --reportportal ``` ### Example: GitHub Actions ```yaml theme={null} - name: Run tests with reporting env: RP_ENDPOINT: https://tms.onetest.ai/api/receiver RP_PROJECT: ${{ secrets.ONETEST_PRODUCT_UUID }} RP_API_KEY: ${{ secrets.ONETEST_API_KEY }} run: pytest --reportportal ``` ### What Gets Reported The ReportPortal agent sends a structured hierarchy: ``` Launch (test run) └── Suite └── Test └── Step └── Log (with screenshots, console output) ``` Each test item includes: * **`code_ref`** — the test's fully qualified name (e.g., `test_login.py:TestLogin.test_valid_login`) * **Status** — passed, failed, skipped * **Duration** — execution time * **Logs** — console output, stack traces, screenshots The `code_ref` value is what OneTest uses to link automated results to test cases. Set the **Automation Test ID** field on your test cases to match the `code_ref` format your agent sends. See [Automation Coverage](/ui/automation-coverage) for details. ## Method 2: JUnit XML Import For frameworks that don't support ReportPortal agents, upload JUnit XML results directly: ```bash theme={null} curl -X POST "https://tms.onetest.ai/api/v1/products/{product_uuid}/import/junit" \ -H "Authorization: Bearer {api_key}" \ -F "file=@test-results.xml" ``` ### Example: GitHub Actions ```yaml theme={null} - name: Run tests run: pytest --junitxml=results.xml - name: Upload results to OneTest if: always() run: | curl -X POST "https://tms.onetest.ai/api/v1/products/${{ secrets.ONETEST_PRODUCT_UUID }}/import/junit" \ -H "Authorization: Bearer ${{ secrets.ONETEST_API_KEY }}" \ -F "file=@results.xml" ``` JUnit XML import creates a single launch with all test cases as flat items (no suite hierarchy). For hierarchical reporting, use a ReportPortal agent instead. ## Method 3: MCP API If you use the [OneTest QA Agent](https://github.com/onetest-ai/qa-agent) or any MCP-compatible client, you can record results directly through MCP tools. ### Setup Add the MCP server to your `.mcp.json`: ```json theme={null} { "mcpServers": { "test-management": { "type": "http", "url": "https://tms.onetest.ai/mcp/test-management", "headers": { "Authorization": "Bearer " } } } } ``` ### Recording Results Use `/qa-onetest run` in Claude Code to execute and record a full test run. MCP tools focus on test case management and search (creating, finding, and updating test cases). For the full run lifecycle — creating runs, recording execution results, and completing runs — use the REST API (see Method 1 and Method 2 above) or the ReportPortal integration. See [Running Tests](/ui/running-tests) for the full `/qa-onetest` workflow. ## API Endpoints Reference | Endpoint | Method | Description | | ---------------------------------------------- | ------ | ---------------------------------------- | | `/api/v2/{product_uuid}/launch` | POST | Start a new test launch | | `/api/v2/{product_uuid}/item` | POST | Report a test item | | `/api/v2/{product_uuid}/log` | POST | Add a log entry (text, screenshot, etc.) | | `/api/v1/products/{product_uuid}/import/junit` | POST | Import JUnit XML results | ## Viewing Automated Results Once results are reported, they appear in the **Test Runs** section alongside manual and AI-driven runs. You can filter by source: * **All runs** — see everything together * **Automated** — only CI/CD results * **Manual** — only manual/AI-driven runs Each automated run shows: * **Pass/fail/skip breakdown** with percentages * **Hierarchical test items** — drill into suites, tests, and steps * **Logs and screenshots** — console output, stack traces, failure screenshots * **Duration and timing** — per-test and total execution time ### Linking to Test Cases When a `code_ref` from an automated result matches a test case's `automation_test_id`, OneTest links them automatically. This enables: * Tracking automated vs. manual coverage on the [Automation Coverage](/ui/automation-coverage) dashboard * Seeing automated results directly on the test case detail page * Identifying automation gaps (tests marked automated but not running in CI/CD) ## Cost Each API call costs 1 coin from your weekly budget. Browser UI usage is always free. See [Usage & Billing](/settings/usage-billing) for details. ## What's Next? Manage API keys and view your Product UUID Track which automated tests are actually running Analyze results and trends in the dashboard # Running Tests Source: https://onetest.ai/ui/running-tests Execute test runs using the OneTest Skill in Claude Code ## Overview OneTest uses an AI-powered test execution engine through the [OneTest QA Agent](https://github.com/onetest-ai/qa-agent). Tests stored in OneTest are executed via the `/qa-onetest` skill in Claude Code, which drives a real browser, records results per step, and reports everything back to OneTest automatically. ## Prerequisites Install [Claude Code](https://docs.anthropic.com/en/docs/claude-code) — the CLI tool that runs the QA agent. Clone the QA agent skills into your project: ```bash theme={null} git clone https://github.com/onetest-ai/qa-agent.git ``` **Tip:** Run `npx github:onetest-ai/qa-agent init` and choose "Connect via browser" for one-click setup — it configures the MCP connection automatically. Or add the `test-management` MCP server manually to your `.mcp.json` (project root): ```json theme={null} { "mcpServers": { "test-management": { "type": "http", "url": "https://tms.onetest.ai/mcp/test-management", "headers": { "Authorization": "Bearer " } } } } ``` Get your API key from **[tms.onetest.ai](https://tms.onetest.ai) -> Settings -> API Keys**. Restart Claude Code so it picks up the new MCP server configuration. ## Running a Test Run Use the `/qa-onetest run` command in Claude Code to create, execute, and complete a full test run. ``` /qa-onetest run ``` The agent will prompt you for which tests to include, then automatically: Creates a test run in OneTest with the selected test cases. Transitions the run from `planned` to `in_progress` and freezes the scope. For each test case in the run: * Navigates to the target URL in a real browser * Executes each test step via Chrome DevTools Protocol (CDP) * Captures screenshots before and after each step * Checks the browser console for errors after each step * Validates actual results against expected outcomes Records the result for each test execution: * **Pass** / **Fail** / **Blocked** / **Skipped** status * Step-level results with screenshots * Failure reasons and classification * Links to defects if applicable Marks the run as complete and generates analytics — pass/fail rates, completion percentage, duration, and failure breakdown. ## All Commands The `/qa-onetest` skill supports four commands: Create, execute, and complete a full test run. The agent drives a real browser and records all results back to OneTest. ``` /qa-onetest run ``` Convert QA audit findings into OneTest test cases. Maps priorities (p0-p3) and categories (accessibility, security, performance, etc.) automatically. ``` /qa-onetest push findings ``` Fetch test cases from OneTest for local browser execution. ``` /qa-onetest pull tests ``` Show the execution queue and any active runs. ``` /qa-onetest status ``` ## How Execution Works ### Browser-Based Testing The agent executes tests in a real Chrome browser using the Chrome DevTools Protocol (CDP). This means tests interact with your application exactly as a user would — clicking buttons, filling forms, navigating pages, and verifying visual output. ### Exploratory Findings If the agent discovers issues during test execution that aren't covered by existing test cases, it can record them as exploratory findings using `record_exploratory_result`. These appear in your test run alongside the planned test results. ### Parallel Execution Non-conflicting tests can run simultaneously using separate Chrome instances on different CDP ports. This speeds up execution when tests target different URLs or isolated user sessions. ## Recording Results Every test execution records detailed results back to OneTest: Each step in a test case gets its own pass/fail status, along with: * Screenshots captured before and after the step * Console output from the browser * Actual vs. expected result comparison When a test fails, the agent records: * **Failure reason**: What went wrong * **Failure classification**: Environment issue, product defect, test data issue, etc. * **Defect links**: Links to related bug tracker issues After the run completes, OneTest provides analytics: * Pass/fail/skip/blocked rates * Completion percentage * Total execution duration * Failure breakdown by category ## Typical Workflow A common end-to-end workflow looks like this: 1. **Audit your site** with QA specialist skills (`/qa-accessibility`, `/qa-security`, etc.) 2. **Push findings** to OneTest as test cases with `/qa-onetest push findings` 3. **Run tests** with `/qa-onetest run` to execute them in a real browser 4. **View results** in the [OneTest dashboard](https://tms.onetest.ai) to analyze trends and track regressions ## What's Next? Analyze test results and trends in the dashboard Full documentation for the QA agent and all skills # Test Management Source: https://onetest.ai/ui/test-management Create, organize, and manage your test cases ## Overview OneTest's test management system helps you create, organize, track, and maintain your test cases efficiently. Create and manage test cases with steps, preconditions, and expected results Organize tests hierarchically with folders and categorize them with tags Find tests instantly with a powerful query language ## Creating Test Cases There are multiple ways to create test cases in OneTest. ### Generate Tests with AI The fastest way to create tests is using the AI Assistant: Press `Cmd/Ctrl + K` or click the chat icon ``` Create a test for user login with valid credentials ``` The AI generates a complete test case. Review and click **Save**. The AI understands context and can generate multiple related tests at once: ``` Create 5 tests for the checkout flow covering happy path and error cases ``` ### Manual Test Creation Navigate to Test Management and click **"+ New Test"** * **Title**: Clear, descriptive title * **Description**: What this test validates * **Priority**: P0 (critical) to P4 (nice-to-have) * **Type**: Manual, Automated, or API Define step-by-step instructions: 1. Action to perform 2. Expected result * **Tags**: `smoke`, `regression`, `login`, etc. * **Folder**: Organize by feature area * **Requirements**: Link to user stories (e.g., `US-123`) Click **Save** to create the test case ### Import Tests Supported formats: CSV, Excel, or JSON Go to Test Management and click **Import** Map your columns to OneTest fields Review the preview and click **Import** See [Import & Export](/ui/import-export) for details. ## Test Case Components **Title**: Short, clear summary of what's being tested ``` Good: "Verify user can login with valid credentials" Bad: "Login test" ``` **Description**: Detailed explanation of the test purpose and scope Each step should have: * **Action**: What the tester should do * **Expected Result**: What should happen Example: ``` 1. Navigate to login page → Login form is displayed 2. Enter valid username → Username field accepts input 3. Enter valid password → Password is masked 4. Click Login button → User is redirected to dashboard ``` What must be true before running this test: * User account exists * User is not logged in * Test data is available Specific data needed for this test: * Username: `test@example.com` * Password: `Test123!` * Expected role: `Admin` **Tags**: Categorize for easy filtering * `smoke`: Critical path tests * `regression`: Run before releases * `feature-name`: Group by feature **Folders**: Hierarchical organization * Authentication/Login/Valid Credentials * Checkout/Payment/Credit Card ## Organizing Tests ### Folders Create folders to organize tests by feature, module, or any structure that fits your team: Access the Test Management section from the main navigation Create folders to organize your tests by feature or module Drag and drop or use bulk operations to move tests into folders ### Bulk Operations Use checkboxes to select test cases for bulk actions: * **Click a checkbox** to select a single test * **Select multiple** tests across the list * **Bulk actions** appear in the toolbar: Move, Tag, Export ## Exporting Test Cases Export test cases to **Excel (XLSX)** format for reporting, sharing, or offline review. Export all test cases in the current product: 1. Go to the Test Cases list 2. Click the **Export** button in the toolbar 3. Choose **"All Test Cases"** 4. An XLSX file downloads with all test data Export a filtered subset using OQL: 1. Enter an OQL query in the search bar (e.g., `priority IN (p1, p2) AND status = ready`) 2. Click the **Export** button 3. Choose **"Filtered Results"** 4. Only matching test cases are exported Export specific test cases: 1. Select test cases using checkboxes 2. Click the **Export** button 3. Choose **"Selected Tests"** 4. Only checked test cases are exported Export test cases not assigned to any folder: 1. Click the **Export** button 2. Choose **"Unassigned Tests"** 3. Tests without a folder are exported ### XLSX Export Contents The exported Excel file includes: | Column | Description | | ------------- | ---------------------------------- | | Identifier | TC-0001, TC-0002, etc. | | Title | Test case title | | Type | manual, automated, exploratory | | Status | draft, ready, deprecated, archived | | Priority | p1, p2, p3, p4 | | Description | Full description text | | Preconditions | Setup requirements | | Steps | Action and expected result pairs | | Tags | Comma-separated tags | | Folder | Folder path | | Created | Creation date | | Updated | Last modification date | ## Best Practices Clear, unambiguous steps that anyone can follow One test case should verify one thing Tag appropriately for easy filtering and test suite creation Connect tests to user stories or requirements for traceability ## What's Next? Execute and monitor test runs Bulk import and export test cases Generate tests with AI Master OQL for powerful searches # Viewing Results Source: https://onetest.ai/ui/viewing-results Analyze test results and execution metrics This page is under construction. More content coming soon! ## Test Results Overview OneTest provides comprehensive views of test execution results, trends, and analytics. ## Result Views ### Test Run Summary High-level metrics for each test run: Percentage of passed tests Number of tests executed Total execution time In Progress, Completed, or Aborted **Status Breakdown:** * ✅ **Passed**: Tests that passed all steps * ❌ **Failed**: Tests with one or more failed steps * ⏸️ **Skipped**: Tests that were skipped * 🚫 **Blocked**: Tests blocked by dependencies ### Individual Test Results View execution details for each test: * **Execution timestamp**: When the test ran * **Executor**: Who ran the test * **Duration**: How long it took * **Step results**: Pass/fail status for each step * **Evidence**: Screenshots, logs, videos * **Defects**: Linked bugs or issues * **Environment**: Where it ran * **Build**: Version tested ### Historical Trends Track quality over time: * **Pass rate trend**: Are we improving? * **Flaky tests**: Tests with inconsistent results * **Slow tests**: Execution time increases * **Failure patterns**: Common failure reasons * **Coverage trends**: Test execution frequency ### Test Analytics Deeper insights into test quality: * **Coverage by feature**: Which areas are well-tested? * **Risk assessment**: High-priority untested areas * **Automation candidates**: Frequently-run manual tests * **Test debt**: Outdated or deprecated tests * **Team productivity**: Tests created/executed by team ## Result Dashboards Real-time view of active test runs: **Widgets:** * Progress bar (% complete) * Status pie chart (pass/fail/skip/blocked) * Test list with individual status * Assignee breakdown * Timeline view * Recent failures Overall product quality metrics: **Metrics:** * Pass rate over time (line chart) * Test execution velocity * Defect trends * Coverage by priority * Risk areas (low-coverage high-priority features) Team productivity and workload: **Insights:** * Tests executed per person * Execution time by team member * Test creation velocity * Review backlog * Assigned vs completed tests Quality per build/release: **Track:** * Pass rate by build * New failures introduced * Regression detection * Build quality trends * Release readiness ## Filtering & Sorting Results Use powerful filters to find what matters: ```oql Find Recent Failures theme={null} status = failed AND execution_date >= -7d ORDER BY execution_date DESC ``` ```oql Flaky Tests theme={null} pass_rate < 0.8 AND pass_rate > 0 AND execution_count >= 5 ORDER BY pass_rate ASC ``` ```oql Slow Tests theme={null} avg_duration > 300 ORDER BY avg_duration DESC ``` ```oql Untested High Priority theme={null} status = active AND priority IN (p0, p1) AND last_run IS NULL ORDER BY priority ASC ``` ## Exporting Results ### Generate Reports Create shareable reports: * **Executive Summary**: High-level metrics for stakeholders * **Detailed Test Report**: Complete execution details * **Failure Analysis**: Deep dive into failures * **Trend Report**: Historical quality trends * **Coverage Report**: Feature coverage analysis **Formats:** PDF, Excel, HTML ### Push to Other Tools Send results to external systems: * **Slack**: Notify team on completion * **JIRA**: Create/update issues for failures * **Email**: Send reports to stakeholders * **Webhook**: Custom integrations ### Programmatic Access Access results via API: ```bash theme={null} # Get test run results GET /api/v1/runs/{run_id}/results # Get execution details GET /api/v1/executions/{execution_id} # Query results with OQL POST /api/v1/products/{id}/search/executions ``` ## Understanding Metrics **Formula:** `(Passed Tests / Total Tests) × 100` **Interpretation:** * **≥95%**: Excellent quality * **90-94%**: Good quality * **80-89%**: Needs attention * **\<80%**: Critical issues Track pass rate trends, not just absolute values. A declining trend indicates growing technical debt. Tests that pass sometimes and fail other times without code changes. **Detection:** Tests with: * Pass rate between 20% and 80% * At least 5 executions * No pattern to failures **Action:** Fix or remove flaky tests—they erode confidence in your test suite. **Formula:** `Tests Executed / Time Period` Measures how quickly you're validating changes. Higher velocity means faster feedback loops. Percentage of features/requirements that have associated tests. **Types:** * **Feature coverage**: Features with tests * **Requirement coverage**: Requirements with tests * **Priority coverage**: P0/P1 features with tests * **Execution coverage**: Tests run in last N days ## What's Next? Back to test management overview Learn advanced filtering with OQL Common testing workflows Testing best practices # Organizing Tests Source: https://onetest.ai/workflows/organizing-tests Best practices for structuring your test suite This page is under construction. More content coming soon! ## Why Organization Matters A well-organized test suite is: * **Easy to navigate**: Find tests quickly * **Maintainable**: Clear structure makes updates simple * **Scalable**: Grows cleanly as product expands * **Collaborative**: Team can work efficiently Poor organization leads to duplicate tests, missed coverage, and wasted time. ## Organization Methods OneTest provides multiple ways to organize tests: Hierarchical structure for logical grouping Flexible labels for categorization Product-specific metadata ## Folder Structure Strategies ### Organize by Product Features Recommended for most teams. ``` 📁 Authentication 📁 Login ✅ Valid credentials ✅ Invalid password ✅ Locked account 📁 Registration ✅ New user signup ✅ Email verification 📁 Password Reset ✅ Request reset link ✅ Reset with valid token 📁 User Profile 📁 View Profile 📁 Edit Profile 📁 Upload Avatar 📁 Products 📁 Browse Products 📁 Search Products 📁 Product Details 📁 Reviews 📁 Shopping Cart 📁 Add to Cart 📁 Update Quantity 📁 Remove Items 📁 Checkout 📁 Shipping Information 📁 Payment 📁 Order Confirmation ``` **Pros:** * Mirrors product structure * Easy for new team members * Scales well **Cons:** * May need reorganization after refactoring ### Organize by User Flows Good for customer-centric products. ``` 📁 New User Journey ✅ Discover product ✅ Sign up ✅ Complete onboarding ✅ First action 📁 Purchase Journey ✅ Browse catalog ✅ Add to cart ✅ Checkout ✅ Payment ✅ Confirmation 📁 Return User Journey ✅ Login ✅ View history ✅ Repeat action ✅ Logout 📁 Admin Journey ✅ Login as admin ✅ Access dashboard ✅ Manage users ✅ Generate reports ``` **Pros:** * Reflects real user behavior * Easy to identify gaps * Clear end-to-end scenarios **Cons:** * Tests may span multiple features * Can have overlapping tests ### Organize by Testing Type Common in mature testing organizations. ``` 📁 Functional Tests 📁 Smoke Tests 📁 Regression Tests 📁 Integration Tests 📁 Non-Functional Tests 📁 Performance Tests 📁 Security Tests 📁 Usability Tests 📁 Accessibility Tests 📁 Platform Tests 📁 Web Tests 📁 Mobile - iOS 📁 Mobile - Android 📁 API Tests 📁 Environment Tests 📁 Dev Tests 📁 Staging Tests 📁 Production Smoke Tests ``` **Pros:** * Clear separation of concerns * Easy to run specific test types **Cons:** * May duplicate structure * Less intuitive for new team members ### Combine Multiple Strategies Most flexible approach. ``` 📁 Smoke Tests (by type) ✅ Login ✅ Homepage loads ✅ Add to cart ✅ Checkout 📁 Features (by feature) 📁 Authentication 📁 Products 📁 Orders 📁 User Journeys (by flow) 📁 First-time Buyer 📁 Returning Customer 📁 Platforms (by platform) 📁 Web 📁 iOS App 📁 Android App ``` **Pros:** * Maximum flexibility * Accommodates different needs **Cons:** * Can become confusing * Requires discipline to maintain ## Effective Tagging Strategy Tags provide cross-cutting categorization: **Purpose:** Indicate criticality * `p0` or `critical`: Must pass before release * `p1` or `high`: Important, impacts core functionality * `p2` or `medium`: Standard functionality * `p3` or `low`: Nice to have * `p4` or `trivial`: Optional ```oql theme={null} priority = p0 # Built-in field # OR tags CONTAINS "critical" ``` **Purpose:** Categorize by testing approach * `smoke`: Critical path verification * `regression`: Full suite before release * `sanity`: Quick verification * `exploratory`: Ad-hoc testing * `security`: Security-focused tests * `performance`: Load/stress tests * `accessibility`: A11y tests ```oql theme={null} tags CONTAINS "smoke" OR tags CONTAINS "regression" ``` **Purpose:** Identify target platform * `web`: Web application tests * `mobile`: Mobile-specific * `ios`: iOS app * `android`: Android app * `api`: API/backend tests * `desktop`: Desktop application ```oql theme={null} tags CONTAINS ANY (ios, android) ``` **Purpose:** Link to features/components * `auth`, `login`, `signup` * `cart`, `checkout`, `payment` * `search`, `filter`, `sort` * `profile`, `settings` * `notifications`, `email` ```oql theme={null} tags CONTAINS "checkout" AND tags CONTAINS "payment" ``` **Purpose:** Track when tests were added * `sprint-24`: Tests for sprint 24 * `v2.1.0`: Tests for version 2.1.0 * `q1-2024`: Tests added in Q1 ```oql theme={null} tags CONTAINS "sprint-24" ``` **Purpose:** Track test lifecycle * `draft`: Work in progress * `ready-for-review`: Needs review * `approved`: Ready to use * `automated`: Has automation * `flaky`: Inconsistent results * `deprecated`: Being phased out ```oql theme={null} tags CONTAINS "flaky" AND last_run >= -7d ``` Many of these have built-in status fields. Use tags only if you need custom statuses. ## Naming Conventions ### Clear, Descriptive Titles **Format:** `Verify [action] [condition] [expected result]` * `Verify user can login with valid credentials` * `Verify error message displays for invalid email format` * `Verify shopping cart updates when quantity is changed` * `Verify checkout completes successfully with credit card` * `Login test` ← Too vague * `Test #47` ← No context * `Check if it works` ← What is "it"? * `User does something and sees a thing` ← Too generic **Tips:** * Start with action verb: "Verify", "Validate", "Confirm" * Include condition being tested * Be specific about expected outcome * Keep under 80 characters if possible ### Folder Naming **Rules:** * Use Title Case: `User Profile`, not `user profile` * Be concise but descriptive * Avoid abbreviations unless universal (e.g., "API" is okay) * Use plurals for collections: `Products`, not `Product` **Examples:** ``` ✅ Good: 📁 Authentication 📁 User Management 📁 Product Catalog 📁 Payment Processing ❌ Bad: 📁 auth (unclear abbreviation) 📁 User_Management (underscores) 📁 product catalog (not title case) 📁 paymentprocessing (no spaces) ``` ### Tag Naming **Rules:** * Use lowercase * Use hyphens for multi-word tags: `smoke-test`, not `smokeTest` * Be consistent across team * Avoid redundant tags **Examples:** ``` ✅ Good: smoke, regression, critical, login, checkout, ios, android ❌ Bad: Smoke (capitalized) smoke_test (underscore) smokeTest (camelCase) smoke-tests (plural when others are singular) ``` ## Maintenance Best Practices Review test suite quarterly: * Remove obsolete tests * Update outdated tests * Consolidate duplicates * Fix organizational issues ```oql theme={null} # Find old draft tests status = draft AND created_at < -90d # Find tests without folders folder_id IS NULL # Find untagged tests tags IS NULL OR tags = [] ``` Assign test ownership: * Feature teams own their feature tests * QA owns smoke/regression suites * Individuals own exploratory tests Use custom fields to track ownership: `owned_by: "payments-team"` Document your structure: * Create folder descriptions * Define tag meanings * Maintain test suite README * Train new team members Automate organization tasks: * Auto-tag based on folder location * Bulk move tests between folders * Generate coverage reports * Alert on orphaned tests ## Common Anti-Patterns Avoid these common mistakes: **Problem:** Deeply nested folders (>5 levels) ``` ❌ Bad: 📁 Web 📁 Desktop 📁 Chrome 📁 Windows 📁 Authentication 📁 Login 📁 Valid Credentials ✅ Test ``` **Solution:** Flatten structure, use tags for additional categorization **Problem:** Some tests organized one way, others differently **Solution:** Choose a strategy and stick to it. Migrate existing tests. **Problem:** Too many tags (>10 per test) **Solution:** Use only meaningful tags. Remove redundant ones. **Problem:** All tests in root folder **Solution:** Start organizing today! Create basic folders. ## What's Next? Work together effectively Find tests with queries General testing best practices Back to test management # Regression Testing Workflow Source: https://onetest.ai/workflows/regression-tests Comprehensive testing before releases This page is under construction. More content coming soon! ## What is Regression Testing? Regression testing ensures that new code changes haven't broken existing functionality. It's typically run before releases or after significant code changes. While smoke tests verify "can we continue testing?", regression tests verify "are we ready to ship?" ## When to Run Regression Tests Comprehensive validation before shipping to production Verify large features or refactoring didn't break anything Validate all sprint work before demo/release Ensure hotfix didn't introduce new issues ## Building a Regression Suite Include all smoke tests (P0/P1 critical paths) Include tests for all major features: ```oql theme={null} priority IN (p0, p1, p2) AND status = active ``` Include tests that have failed before: ```oql theme={null} failure_count > 0 AND status = active ORDER BY failure_count DESC ``` Tag tests with `regression` for easy filtering: ```oql theme={null} tags CONTAINS "regression" AND status = active ``` Target suite that completes in 2-4 hours maximum ## Regression Testing Strategy ### Complete Test Suite Run all regression tests before major releases. **When to use:** * Major version releases * Quarterly releases * After significant architecture changes **Characteristics:** * 100+ tests * 2-4 hours execution time * All features covered * All priority levels (P0-P3) Full regression is time-consuming. Reserve for major releases only. ### Risk-Based Testing Test only areas affected by recent changes. **When to use:** * Sprint releases * Minor version updates * Targeted hotfixes **Strategy:** ```oql theme={null} # Tests in affected components component IN (authentication, checkout) AND tags CONTAINS "regression" AND status = active ``` **Benefits:** * Faster execution (30-60 minutes) * Focused on risk areas * More frequent testing possible ### Continuous Regression Automated tests run on every commit or nightly. **Setup:** * Automate all P0/P1 tests * Run on CI/CD pipeline * Integrate with version control * Alert team on failures **Benefits:** * Catch regressions early * Fast feedback loop * No manual effort required * Consistent execution ## Running Regression Tests Choose appropriate regression level: ```oql theme={null} # Full regression tags CONTAINS "regression" AND status = active ORDER BY priority ASC # Selective regression (e.g., auth module) tags CONTAINS "regression" AND component = "authentication" AND status = active ``` Configure the test run: * **Name**: `Regression - v2.1.0 Release` * **Environment**: `Staging` * **Build**: `2.1.0-rc1` * **Assigned to**: Distribute among team Team executes assigned tests in parallel Monitor run dashboard for: * Completion percentage * Pass/fail rate * Blockers * Failed tests needing triage For each failure: * Is it a real bug? Create defect. * Is it a test issue? Fix the test. * Is it environmental? Investigate environment. Decide go/no-go based on results: * All P0 tests passed? ✅ Can release * Any P0 failures? ❌ Block release * P1/P2 failures? Assess risk ## Example Regression Suites **Scope:** Full website functionality **Categories:** * Authentication (10 tests) * Navigation (15 tests) * User Profile (12 tests) * Search (8 tests) * Content Management (20 tests) * Forms & Validation (18 tests) * Notifications (7 tests) * Settings (10 tests) **Total:** \~100 tests **Time:** \~3 hours **Tags:** `regression`, `web` **Scope:** All API endpoints **Categories:** * Authentication endpoints (5 tests) * User CRUD operations (12 tests) * Product endpoints (15 tests) * Order processing (10 tests) * Search & filtering (8 tests) * File uploads (6 tests) * Webhooks (5 tests) **Total:** \~60 tests **Time:** \~45 minutes (automated) **Tags:** `regression`, `api`, `automated` **Scope:** iOS & Android apps **Categories (per platform):** * App launch & onboarding (5 tests) * Login & authentication (8 tests) * Main navigation (10 tests) * Core features (25 tests) * Settings & preferences (7 tests) * Offline functionality (10 tests) * Push notifications (5 tests) **Total:** \~70 tests per platform **Time:** \~2 hours per platform **Tags:** `regression`, `mobile`, `ios`/`android` ## Optimizing Regression Testing Test high-risk, frequently-changed areas first Move stable, repetitive tests to automation Distribute tests across team members Archive tests for deprecated features Fix flaky tests immediately Monitor pass rates and execution times ## Regression Test Maintenance When to add tests to regression suite: * New feature released * Bug found in production * Area lacking coverage * High-value user workflow **Process:** 1. Create test case 2. Execute to validate it works 3. Tag with `regression` 4. Add to appropriate folder When to remove tests from regression suite: * Feature deprecated/removed * Test is consistently flaky * Test is redundant with other tests * Feature changed significantly **Process:** 1. Change status to `archived` (don't delete) 2. Remove `regression` tag 3. Document why it was removed Keep tests current with product changes: * Review test suite quarterly * Update after major feature changes * Fix broken tests immediately * Refine test steps for clarity * Update expected results Assign test ownership to feature teams for ongoing maintenance ## Integration with Release Process ```mermaid theme={null} graph LR A[Code Complete] --> B[Smoke Tests] B --> C{Pass?} C -->|No| D[Fix & Retry] D --> B C -->|Yes| E[Regression Tests] E --> F{Pass?} F -->|No| G[Triage Failures] G --> H{Blocking?} H -->|Yes| D H -->|No| I[Accept Risk] I --> J[Release] F -->|Yes| J ``` ## What's Next? Quick verification tests Structure your test suite Work together on testing Testing best practices # Smoke Testing Workflow Source: https://onetest.ai/workflows/smoke-tests Run critical path tests after every deployment This page is under construction. More content coming soon! ## What are Smoke Tests? Smoke tests are a subset of critical tests that verify the most important functionality of your application. They ensure the system is stable enough for further testing. Think of smoke tests as "build verification tests"—if smoke tests fail, the build is not ready for deeper testing. ## When to Run Smoke Tests Verify deployment was successful and core features work Validate new build before handing off to QA Automated nightly runs to catch environmental issues Final sanity check before releasing to production ## Creating a Smoke Test Suite What functionality is absolutely essential? Examples: * User can log in * User can view homepage * User can add item to cart * User can complete checkout * User can log out Use OQL to find and tag your critical tests: ```oql theme={null} priority IN (p0, p1) AND component IN (login, checkout, core) ``` Then bulk-tag them with `smoke` Smoke tests should be: * **Fast**: Complete in \<15 minutes * **Focused**: Only critical functionality * **Stable**: No flaky tests allowed * **Independent**: No complex dependencies If your smoke suite takes >30 minutes, it's too large. Split into smoke + sanity suites. Automate smoke tests to run: * After every deployment (CI/CD pipeline) * On a schedule (nightly builds) * Before release (release gate) ## Running Smoke Tests ### Run Smoke Suite Manually Use OQL to filter smoke tests: ```oql theme={null} tags CONTAINS "smoke" AND status = active ORDER BY priority ASC ``` Click **"+ New Test Run"**: * **Name**: `Smoke Tests - Build 2.1.0` * **Environment**: `Staging` or `Production` * **Build**: `2.1.0` Run tests in order of priority (P0 first, then P1) If any test fails, **STOP** and report immediately. Don't proceed with further testing until smoke tests pass. ### Integrate with CI/CD Pipeline ```yaml theme={null} # Example: GitHub Actions workflow name: Smoke Tests on: deployment_status jobs: smoke-tests: runs-on: ubuntu-latest if: github.event.deployment_status.state == 'success' steps: - name: Run Smoke Tests run: | # Use OneTest API to trigger smoke test run curl -X POST https://api.onetest.ai/v1/runs \ -H "Authorization: Bearer ${{ secrets.ONETEST_API_KEY }}" \ -d '{ "name": "Smoke Tests - ${{ github.sha }}", "environment": "${{ github.event.deployment_status.environment }}", "filters": "tags CONTAINS \"smoke\" AND status = active" }' - name: Wait for Results run: | # Poll for results and fail if smoke tests fail # ... polling logic ... - name: Notify Team if: failure() uses: slack-notify-action ``` ### Schedule Regular Smoke Tests Set up automated recurring runs: Save your smoke test filter: ```oql theme={null} tags CONTAINS "smoke" AND status = active ``` * **Frequency**: Nightly, or after business hours * **Environment**: Staging or Production * **Notification**: Alert on failure Check dashboard each morning for overnight results ## Example Smoke Test Suite **Critical Paths:** 1. Homepage loads successfully 2. User can search for products 3. User can view product details 4. User can add product to cart 5. User can view cart 6. User can proceed to checkout 7. User can complete payment 8. User receives order confirmation **Total Time:** \~8 minutes **Priority:** All P0 tests **Critical Paths:** 1. Landing page loads 2. User can sign up 3. User can log in 4. Dashboard displays data 5. User can create new item 6. User can edit item 7. User can delete item 8. User can log out **Total Time:** \~10 minutes **Priority:** All P0 tests **Critical Endpoints:** 1. Health check returns 200 2. Authentication endpoint works 3. GET /users returns data 4. POST /users creates user 5. PUT /users/ updates user 6. DELETE /users/ removes user 7. Database connection is healthy 8. Cache is accessible **Total Time:** \~2 minutes **Priority:** All P0 tests ## Smoke Test Best Practices Target \<15 minutes total execution time Remove or fix any flaky tests immediately Test on staging or production-like environments Failing smoke tests should block deployment Track which build/version was tested Make results visible to entire team ## Troubleshooting **Problem:** Smoke suite takes >30 minutes **Solutions:** * Review tests—are they all truly critical? * Remove redundant test steps * Parallelize test execution * Consider splitting into smoke + sanity suites **Problem:** Smoke tests sometimes pass, sometimes fail **Solutions:** * Identify the flaky tests using pass rate analysis * Fix root cause (timing issues, environment problems, etc.) * Remove flaky tests from smoke suite temporarily * Add explicit waits/retries in test steps **Problem:** Smoke tests consistently fail after deployment **Solutions:** * Check if deployment actually completed * Verify environment is accessible * Review recent code changes * Check for database migration issues * Validate configuration/environment variables ## What's Next? Comprehensive testing workflow Best practices for test organization Learn to filter tests with OQL General testing best practices # Team Collaboration Source: https://onetest.ai/workflows/team-collaboration Work together effectively on testing This page is under construction. More content coming soon! ## Collaborative Testing OneTest enables teams to work together seamlessly on test creation, execution, and analysis. ## Team Features Invite team members with role-based access (Owner, Member) Assign tests to specific team members for execution See test run progress as team members execute tests Discuss test cases and results within the platform ## Team Workflows ### Collaborative Test Writing Team lead creates test outline or uses AI to generate drafts Assign incomplete tests to domain experts: * Frontend tests → Frontend engineers * API tests → Backend engineers * Mobile tests → Mobile team Team members review each other's tests: * Check for completeness * Verify steps are clear * Ensure expected results are specific QA lead or team lead approves final tests Use the AI Assistant to generate initial test drafts, then have team members refine the details. ### Distributed Test Execution QA lead or release manager creates test run for sprint/release Assign tests to team members: * Balance workload * Consider expertise (assign API tests to API experts) * Account for time zones (for global teams) Team members execute their assigned tests simultaneously Team lead monitors run dashboard: * Track completion percentage * Identify blockers * Reassign stuck tests Team discusses failures: * Real bugs → Create defects * Test issues → Fix tests * Environmental → Investigate environment ### Team Test Maintenance Assign test folders to feature teams: * Authentication → Auth team * Payment → Payments team * Search → Search team Teams review their tests quarterly: * Update for product changes * Remove obsolete tests * Fix flaky tests * Improve clarity For tests spanning multiple features: * Tag with multiple teams * Assign primary owner * Schedule joint reviews ## Best Practices for Teams **Why:** Prevents "bystander effect" where no one maintains tests **How:** * Assign tests to specific people or teams * Use custom field: `owner: "auth-team"` * Make ownership visible in test metadata * Review ownership quarterly ```oql theme={null} # Find unowned tests status = active AND owner IS NULL ``` **Why:** Ensures consistency across team **How:** * Create test case templates * Document test writing guidelines * Use AI prompts for consistent generation * Review new tests for adherence Save common AI prompts as prompt templates for the whole team to use. **Why:** Keeps team aligned on testing strategy **Agenda:** * Review pass rates and trends * Discuss flaky tests * Identify coverage gaps * Plan upcoming test work * Share best practices **Frequency:** Weekly or bi-weekly **Why:** Prevents duplicate effort **How:** * Create reusable test steps * Share test data sets * Document common workflows * Use saved OQL queries * Build test case templates **Why:** Improves team capability **How:** * Document domain knowledge * Pair testing sessions * Internal testing workshops * Share test case examples * Code review for test automation ## Team Roles & Responsibilities ### QA Lead Responsibilities **Test Strategy:** * Define testing approach * Set quality gates * Choose test types and coverage targets **Team Management:** * Assign tests to team members * Monitor test execution progress * Balance workload **Quality Oversight:** * Review test results * Triage failures with team * Make release decisions ### QA Engineer Responsibilities **Test Creation:** * Write detailed test cases * Create automation scripts * Document edge cases **Test Execution:** * Execute assigned tests * Document results thoroughly * Report bugs with evidence **Maintenance:** * Keep tests up to date * Fix flaky tests * Improve test coverage ### Developer Involvement **Unit/Integration Tests:** * Write automated tests for code changes * Ensure tests pass before committing **Collaboration:** * Review test cases for new features * Help QA understand technical implementation * Fix failing tests **Knowledge Sharing:** * Document APIs and behaviors * Explain complex logic * Pair with QA on technical tests ### Product Owner Role **Acceptance Criteria:** * Define clear acceptance criteria for features * Review test coverage for user stories **Prioritization:** * Help prioritize test execution * Identify critical user paths **Sign-off:** * Review test results before release * Make go/no-go decisions ## Communication Tools Comment directly on test cases and results Get notifications in your team Slack channel Receive alerts for test failures and completions Shared dashboards for team visibility ## Handling Conflicts **Problem:** Two people edit the same test simultaneously **Solution:** * OneTest uses versioning—both edits are saved * Compare versions and merge manually * Communicate before major test updates * Use assignment to indicate "I'm working on this" **Problem:** Team members have different opinions on how to test **Solution:** * Discuss in team meeting * Document testing standards * Defer to QA lead for final decision * Try both approaches and compare **Problem:** Unclear who owns certain tests **Solution:** * Default to feature team ownership * For cross-cutting tests, assign primary owner * Document ownership in test metadata * Review ownership during test audits ## Metrics for Team Performance Track these metrics to improve team effectiveness: Tests created/executed per sprint Percentage of features with tests Bugs found vs. bugs escaped to production Time from test creation to execution Quality improvement over time Automated vs. manual tests ## What's Next? Structure for team collaboration Testing best practices for teams Critical path testing workflow Comprehensive testing workflow