About Company/Product

• Company: ConformIQ (sometimes referenced as “Conformic” historically)

• Product: A model-based testing (MBT) platform enhanced by symbolic AI and LLM integration (often referred to simply as ConformIQ or Creator/Visualizer).

• Host/Organizer: Test Guild (with Joe as the host).

Object of the Webinar

The main goal was to discuss how artificial intelligence (AI), particularly symbolic AI and generative AI, can be applied to test case generation and test automation. Key objectives included:

• Showing differences between symbolic AI and generative AI (Gen AI).

• Demonstrating ConformIQ’s approach to combining MBT (model-based testing) with AI to optimize test creation.

• Explaining how test coverage can be guaranteed (or validated) with symbolic AI, and how Gen AI can help in earlier stages (e.g., generating Gherkin from requirements).

Webinar 6. To AI or Not to AI in Testing (ConformIQ)

Why Test Software When AI Can Create and Test It Too?

About Company/Product

• Company: ConformIQ (sometimes referenced as “Conformic” historically)

• Product: A model-based testing (MBT) platform enhanced by symbolic AI and LLM integration (often referred to simply as ConformIQ or Creator/Visualizer).

• Host/Organizer: Test Guild (with Joe as the host).

Objective of the Webinar

• The main goal was to discuss how artificial intelligence (AI), particularly symbolic AI and generative AI, can be applied to test case generation and test automation. Key objectives included:

• Showing differences between symbolic AI and generative AI (Gen AI).

• Demonstrating ConformIQ’s approach to combining MBT (model-based testing) with AI to optimize test creation.

• Explaining how test coverage can be guaranteed (or validated) with symbolic AI, and how Gen AI can help in earlier stages (e.g., generating Gherkin from requirements).

Presenting the Webinar

• Mark (ConformIQ Representative):

  • Over 12 years at ConformIQ.

  • Background in AI (wrote papers on AI applications as early as 1983).

  • Explained the overall approach and strategic use of AI in testing.

• Joe (Test Guild Host):

  • Facilitated the session, asked clarifying questions, and moderated the Q&A.

Brief Summary of the Webinar

• Overview of AI Approaches

  • Symbolic AI: Used by ConformIQ for 20+ years to generate test cases deterministically (like IBM’s “Deep Blue” for chess).

  • Generative AI (Gen AI): Large language models (LLMs) such as ChatGPT, used for code/text generation but can be non-deterministic.

• ConformIQ’s Process

  • Requirements to Gherkin: The system can use an LLM to convert natural-language requirements into Gherkin scenarios.

  • Gherkin to MBT: Imported into ConformIQ’s Creator tool to build a system-level model.

  • Optimized Test Generation: Symbolic AI ensures coverage and deduces minimal test sets with 100% coverage.

  • Automated Script Output: Exports code for any chosen test execution framework (e.g., Selenium, Playwright, etc.).

• Benefits & Challenges

  • Control Points: AI-generated artifacts (e.g., Gherkin) must be reviewed to avoid “hallucinations” or missing data.

  • Complex Systems: Symbolic AI can handle system-level complexity and guaranteed coverage better than raw Gen AI alone.

  • Integration: ConformIQ integrates with popular tools (JIRA, ALM, Jenkins, etc.) for end-to-end automation.

Features and Technical Aspects

• Symbolic AI

  • Deterministic approach for model-based test generation, guaranteeing coverage and minimizing test sets.

  • Ideal for compliance-heavy industries needing robust coverage.

• Gen AI / LLM Integration

  • Used to generate or enhance Gherkin from textual requirements.

  • Simplifies initial scenario creation but requires verification to prevent errors.

• Model-Based Testing (MBT)

  • ConformIQ’s Creator tool merges multiple Gherkin scenarios into a system-level model.

  • Allows loops and advanced logic for complex applications (beyond typical BDD).

  • Generates step definitions and locators for chosen frameworks.

• Visualizer

  • Provides a flow diagram of the generated Gherkin code, helping teams validate AI outputs.

  • Ensures BA/Dev/QA alignment (“three amigos”).

• CI/CD Integration

  • Produces executable scripts for various frameworks (Selenium, Playwright, WebdriverIO, etc.).

  • Aligns with existing test management solutions (JIRA, ALM, TestRail, etc.).

Job Specifications

• Testers as Domain Experts: Less focus on raw scripting; more on validating AI outputs, domain logic, and coverage analysis.

• AI Familiarity: Understanding symbolic AI vs. Gen AI helps teams decide best approach for different test scenarios.

• Cross-Team Collaboration: BAs, Devs, and QA coordinate using MBT, Gherkin, and AI-driven modeling.

How GoTestPro Can Compete

• Offer a Hybrid AI Approach

  • Combine Gen AI for quick Gherkin generation with symbolic AI for deterministic coverage in complex systems.

  • Provide an easy path to system-level modeling or scenario consolidation.

• Control Points & Verification

  • Include a visual review mechanism (flow diagrams, coverage matrix) so users can validate AI outputs.

  • Avoid “AI hallucinations” by integrating robust checks for missing or made-up data.

• Deterministic Coverage & Risk Management

  • Emphasize guaranteed coverage for mission-critical or compliance-heavy domains.

  • Provide metrics (like 100% coverage, minimal test sets) to highlight ROI.

• Flexible Integrations

  • Ensure out-of-the-box compatibility with major frameworks (Selenium, Playwright) and DevOps pipelines.

  • Support advanced test data handling (API, scanning, custom DBs).

• Consulting & Onboarding

  • Offer readiness assessments, training, and best practices for AI-based test generation.

  • Show how domain experts, testers, and DevOps can adopt the tool seamlessly.

Additional Important Points

• Data Handling: Large LLMs can invent or “hallucinate” data, so a governance process is crucial.

• Enterprise Infrastructure: For big systems, cost and computational requirements of Gen AI can be high. Symbolic AI may be more resource-efficient.

• Adaptability: Tools must integrate with existing user stories, JIRA items, or Word docs.

• Expert Review: Even with AI, domain expertise and human oversight remain essential to confirm correctness of generated tests.