DeviQA and Soapbox: WMS functional testing case analysed
Verdict: what is checkable
The evidence grade (this site's label for how a case is sourced: primary with client or press confirmation, secondary when vendor-hosted only.) is secondary: the only source is the vendor's own page; of 18 metrics, 0 are verifiable yes, 18 partial and 0 no. [ev-0891]
Client profile as published
- Client
- Soapbox
- Segment
- Ecommerce and D2C (direct to consumer: a brand that sells its own products online to shoppers without a retailer in between.)
- Region
- not stated in the source
- Channels
- ecommerce
- Size indicators
- none given
The client description and size indicators are quoted from the source; segment, region and channels are this site's classification. [ev-0891]
Systems in scope
- WMS (warehouse management system: software that runs receiving, storage, picking and shipping inside a distribution center.): named in the title and the About project section ("e-commerce fulfillment and warehouse management platform")
- OMS (order management system: software that routes, splits and tracks customer orders across stores, warehouses and carriers.): named in the About project section ("order processing")
Where the source names each system is quoted or summarised from the case page. [ev-0891]
Problem as stated
Before the engagement Soapbox ran tests by hand, test coverage stood at 40%, a regression run took about 1.5 weeks and stories took about two weeks to move forward. The page describes the QA process as poor and the Android app code as carrying many major and critical issues. Soapbox asked DeviQA to structure test documentation, widen coverage and cut testing time on the web platform and the Android app prototype. [ev-0891]
A huge amount of major/critical issues in the code base of the mobile app(quoted from the source)
Approach
- Testing types
- functional, automation and mobile (main scope: functional)
- Team shape
- 3 manual QA engineers, 1 automation QA engineer, 1 mobile developer
- Duration
- Since 2021
- Engagement model
- dedicated-team
- Tooling named in the source
- Figma, GitLab, GitLab issues, Sentry, Datadog, Android Studio, Cypress, MS Visual Studio Code, Google Docs, Google Sheets, React Native, React Native Reanimated, React Navigation, App Center, Swagger
Metrics claimed
| Metric | Value | Unit | Claimed by | Verifiable | Evidence |
|---|---|---|---|---|---|
| test coverage before DeviQA | 40 | % | vendor | partial | ev-0892 |
| software covered with test cases | >90 | % | vendor | partial | ev-0893 |
| test cases and automated scripts created | 2,500+ | test cases and automated scripts | vendor | partial | ev-0894 |
| web platform coverage | 90 | % | vendor | partial | ev-0894 |
| test cases created (stat block) | 2.5k+ | test cases | vendor | partial | ev-0895 |
| automated scripts in the first 6 months | >400 | automated scripts | vendor | partial | ev-0896 |
| regression run time | 1.5 weeks to three days | vendor | partial | ev-0897 | |
| regression testing speed (stat block) | X3 | x | vendor | partial | ev-0898 |
| time to move stories forward before DeviQA | ~2 | weeks | vendor | partial | ev-0899 |
| task handling speed | 2x | x | vendor | partial | ev-0900 |
| share of code refactored | 60 | % | vendor | partial | ev-0901 |
| mobile app files refactored | 200+ | files | vendor | partial | ev-0902 |
| successful releases | 95+ | releases | vendor | partial | ev-0903 |
| web platform releases supported | 80+ | releases | vendor | partial | ev-0904 |
| Android app releases supported | 15+ | releases | vendor | partial | ev-0904 |
| critical bugs reported | 550+ | bugs | vendor | partial | ev-0905 |
| defects reported | 1,500+ | defects | vendor | partial | ev-0906 |
| share of defects blocker, critical or major | 60 | % | vendor | partial | ev-0906 |
What the case does not say
- Measurement method.
- Duration over which the metric held.
- Cost.
- What was out of scope.
Reading notes for a buyer
This case speaks to the scenario “OMS or WMS replacement” and feeds dimension D2 (Retail system coverage) of the methodology. A question to put to DeviQA:
Which warehouse and order flows did the web coverage figure include, and how was coverage counted? [ev-0891]
Source and verification
- Source
- QA services for warehouse management software | DeviQA case study
- Published
- not dated by the vendor
- Verified
- 2026-10-30
- Evidence grade
- secondary
- Corroborating sources
- none found
| Id | Claim | Source | Class | Grade | Published | Verified |
|---|---|---|---|---|---|---|
| ev-0891 | QA services for warehouse management software | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0892 | 40% test coverage | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0893 | >90% of the software is covered with detailed and relevant test cases | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0894 | DeviQA created 2,500+ test cases and automated scripts, achieving 90% coverage of the Soapbox web platform. | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0895 | 2.5k+ Test cases created | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0896 | >400 automated scripts were created in the first 6 months | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0897 | Regression testing decreased from 1.5 weeks to three days. | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0898 | X3 Faster regression testing | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0899 | ~2 weeks to move stories forward | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0900 | 2x faster task handling | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0901 | 60% of the code was refactored to improve application stability | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0902 | 200+ Refactored files of the mobile app | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0903 | 95+ successful releases | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0904 | DeviQA supported 80+ releases of the Soapbox web platform and 15+ releases of its Android application. | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0905 | 550+ Critical bugs reported | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0906 | The team reported 1,500+ defects, with 60% classified as blocker, critical, or major. | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 |
| Id | Claim | Source | Class | Grade | Published | Verified |
|---|---|---|---|---|---|---|
| ev-0891 | QA services for warehouse management software | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0892 | 40% test coverage | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0893 | >90% of the software is covered with detailed and relevant test cases | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0894 | DeviQA created 2,500+ test cases and automated scripts, achieving 90% coverage of the Soapbox web platform. | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0895 | 2.5k+ Test cases created | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0896 | >400 automated scripts were created in the first 6 months | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0897 | Regression testing decreased from 1.5 weeks to three days. | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0898 | X3 Faster regression testing | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0899 | ~2 weeks to move stories forward | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0900 | 2x faster task handling | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0901 | 60% of the code was refactored to improve application stability | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0902 | 200+ Refactored files of the mobile app | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 |
Show 4 more evidence records
| Id | Claim | Source | Class | Grade | Published | Verified |
|---|---|---|---|---|---|---|
| ev-0903 | 95+ successful releases | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0904 | DeviQA supported 80+ releases of the Soapbox web platform and 15+ releases of its Android application. | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0905 | 550+ Critical bugs reported | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 | |
| ev-0906 | The team reported 1,500+ defects, with 60% classified as blocker, critical, or major. | QA services for warehouse management software | DeviQA case study | vendor-case-study | secondary | 2026-10-30 |
Vendor profile: DeviQA. System facet: OMS and WMS testing vendors.
Positions on this page are computed from the methodology published at /methodology/ and from the evidence records linked on this page. Case study figures are the vendor's or the client's claims as published at the linked source; the "verifiable" column states what this site could check. The site does not recommend vendors, does not accept paid placement and does not publish rates. Last computed: 2026-11-03.
Published by the editor of retailqavendors.com. Last reviewed .