...

• Deliver AI functionality for automated test step creation.

• Internal beta by mid-February; public release by late March.

LLM Testing

• LLM Accuracy Tool: Release a tool to validate LLM responses against expected outputs.

• LLM Price Tradeoff Tool: Develop a tool comparing performance across LLMs at identical price points. Beta launch by June.

ServiceNow: Core Features (March Deadline)

...

• Finalize SAP and Salesforce discovery phases for upcoming initiatives.

2025 - Q2

LLM Testing

...

LLM Accuracy Tool: Release a tool to validate LLM responses against expected outputs.

...

• SAP Integration: AI for Manual Test Conversion

  • Deliver functionality to convert manual SAP tests into automated AI-driven test steps.

  • Collaborate with pilot customers for real-world validation.

Info
Metrics for Success: Successful beta tests of LLM tools (target: 3 active enterprise users for feedback). Number of manual SAP test cases converted (target: 1,00 cases).

2025 - Q3

...

• AI Parity Enhancements

  • Collect and implement feedback from Q1 AI test step feature users.

  • Add advanced customization options for AI-generated test cases.

• Salesforce: Autogenerate Test Cases

  • Launch functionality to create test cases automatically from Salesforce product databases.

  • Include configurable mappings for custom Salesforce environments.

  LLM Tools Expansion

  • Iterate on LLM accuracy and pricing tools based on Q2 beta feedback.

  • Launch broader marketing campaign targeting AI/LLM-focused customers.

Info
Metrics for Success: Salesforce integration adoption (target: onboard 10 new enterprise users). LLM tools accuracy improvement (target: >90% success rate in testing scenarios).

2025 - Q4

• AI Parity: Advanced Features

  • Introduce advanced AI algorithms for predictive testing and failure analysis.

  • Develop integrations for additional platforms based on customer demand.

• LLM Testing Expansion

  • Broaden support for emerging LLMs and optimize performance benchmarking tools.

• User Feedback and Market Insights

  Host customer workshops and gather end-of-year feedback for the 2026 roadmap.

  Jira integration

...