DeviQA and an anonymised specialty retailer: OMS functional testing case analysed

Verdict: what is checkable

The evidence grade (this site's label for how a case is sourced: primary with client or press confirmation, secondary when vendor-hosted only.) is secondary: the only source is the vendor's own page; of 14 metrics, 0 are verifiable yes, 0 partial and 14 no. [ev-0921]

Client profile as published

Client
A New York art gallery (the vendor's description; the vendor does not name the client)
Segment
Specialty retail
Region
United States
Channels
not stated
Size indicators
none given

The client description and size indicators are quoted from the source; segment, region and channels are this site's classification. [ev-0921]

Systems in scope

  • OMS (order management system: software that routes, splits and tracks customer orders across stores, warehouses and carriers.): named in the client description ("From managing orders and private sales to public exhibitions")

Where the source names each system is quoted or summarised from the case page. [ev-0921]

Problem as stated

The gallery runs a digital platform for managing, presenting and selling artworks, books and art objects, including orders and private sales. QA ran apart from the development and release flow, and testing happened mostly at the UAT stage. User stories had no structure, bug reports lacked context and many test cases were out of date. DeviQA was asked to tie QA into the release cycle, cut testing time and improve issue tracking. [ev-0921]

The QA process ran in isolation from the development and release flow (quoted from the source)

Approach

Testing types
functional (main scope: functional)
Team shape
1 manual QA engineer
Duration
Since 2023
Engagement model
dedicated-team
Tooling named in the source
Jira, Confluence, GitHub, Slack, 1Password, Mailtrap, Google Docs, Postman, Microsoft D365

Metrics claimed

Metrics as published, with who claims each and what this site could check.
MetricValueUnitClaimed byVerifiableEvidence
release velocity30%vendornoev-0922
critical production bugs25%vendornoev-0923
overall testing time40%vendornoev-0924
early-stage defect detection60%vendornoev-0925
lost or unclear tickets95%vendornoev-0926
task processing speed20%vendornoev-0927
bug reports with screenshots or videos100%vendornoev-0928
ticket reopen rate50%vendornoev-0929
defect reproduction time30%vendornoev-0930
regression risk70%vendornoev-0931
test execution time35%vendornoev-0932
test cases created515test casesvendornoev-0933
test coverage90%vendornoev-0934
bugs logged1k+bugsvendornoev-0935

What the case does not say

  • Baseline before the engagement.
  • Measurement method.
  • Duration over which the metric held.
  • Cost.
  • What was out of scope.

Reading notes for a buyer

This case speaks to the scenario “First outsourced QA team for a mid-size retailer” and feeds dimension D1 (Retail case study evidence) of the methodology. A question to put to DeviQA:

How were the percentage changes measured, over what period, and what were the starting values? [ev-0921]

Source and verification

Source
QA for an art management and sales platform | DeviQA case study
Published
not dated by the vendor
Verified
2026-10-30
Evidence grade
secondary
Corroborating sources
none found
Evidence records for this case. First 12 of 15 shown; the rest open below.
IdClaimSourceClassGradePublishedVerified
ev-0921QA for an art management and sales platformQA for an art management and sales platform | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0922Release velocity increased by 30%QA for an art management and sales platform | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0923Critical production bugs decreased by 25%QA for an art management and sales platform | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0924Overall testing time decreased by 40%QA for an art management and sales platform | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0925Early-stage defect detection rose by 60%QA for an art management and sales platform | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0926Lost/unclear tickets decreased by 95%QA for an art management and sales platform | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0927Task processing speed increased by 20%QA for an art management and sales platform | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0928100% of reports include screenshots or videosQA for an art management and sales platform | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0929Ticket reopen rate decreased by 50%QA for an art management and sales platform | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0930Defect reproduction time decreased by 30%QA for an art management and sales platform | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0931Regression risk decreased by 70%QA for an art management and sales platform | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-0932Test execution time decreased by 35%QA for an art management and sales platform | DeviQA case studyvendor-case-studysecondary2026-10-30
Show 3 more evidence records
Remaining evidence records.
IdClaimSourceClassGradePublishedVerified
ev-0933515 Test cases createdQA for an art management and sales platform | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-093490% Test coverageQA for an art management and sales platform | DeviQA case studyvendor-case-studysecondary2026-10-30
ev-09351k+ Number of bugs loggedQA for an art management and sales platform | DeviQA case studyvendor-case-studysecondary2026-10-30

Vendor profile: DeviQA. System facet: OMS and WMS testing vendors.

Positions on this page are computed from the methodology published at /methodology/ and from the evidence records linked on this page. Case study figures are the vendor's or the client's claims as published at the linked source; the "verifiable" column states what this site could check. The site does not recommend vendors, does not accept paid placement and does not publish rates. Last computed: 2026-11-03.

Published by the editor of retailqavendors.com. Last reviewed .