Methodology: weights, rubrics and how positions are computed
| Dimension | Weight | What is measured |
|---|---|---|
| D1 Retail case study evidence | 20 | Number, depth and verifiability of published retail case studies |
| D2 Retail system coverage | 15 | Breadth of retail systems with service page plus case study evidence |
| D3 Peak-season and performance evidence | 10 | Documented performance or load work for retail peaks |
| D4 Test automation evidence | 15 | Automation work documented in retail cases with named frameworks and coverage figures |
| D5 Scale and stability | 10 | Headcount band, years operating, office count, two-sourced |
| D6 Independent review signal | 10 | Count and recency of reviews on Clutch, GoodFirms, G2 |
| D7 Engagement transparency | 10 | Published engagement models, trial or pilot terms, SLA (service level agreement: written commitments on response time, availability or turnaround that a vendor accepts in its contract.) or exit terms, named retail delivery team |
| D8 Compliance and security evidence | 10 | Certifications with certificate evidence, PCI DSS (Payment Card Industry Data Security Standard: the security rules for any business that stores, processes or transmits card data.) testing scope, data handling statements |
Scope gate
A candidate enters scoring only when each item below has an evidence record. A candidate that fails is excluded and listed on the not-published page with the failing item.
- Offers QA or software testing as a named service line (a service page, not a blog post).
- Has at least one published retail case study on its own domain or in a document it hosts, where "retail" means the client sells goods to consumers or businesses through stores, ecommerce, marketplaces or both, or the system under test is a POS (point of sale: the till software and hardware that rings up store sales, takes card payments and prints receipts.), OMS (order management system: software that routes, splits and tracks customer orders across stores, warehouses and carriers.), WMS (warehouse management system: software that runs receiving, storage, picking and shipping inside a distribution center.), ecommerce platform, retail mobile app, loyalty or payments system used by such a client.
- Is a company, not a freelancer platform, a tool vendor or a crowdtesting marketplace.
- Has a verifiable headcount band from LinkedIn plus one other source.
Rubric per dimension
D1 Retail case study evidence
- 0: no retail case study found.
- 3: 1 to 2 retail cases, client anonymised by the vendor, no metrics or metrics with verifiable: no.
- 5: 3 to 4 retail cases, at least one metric with verifiable: partial.
- 7: 5 to 7 retail cases, at least one named client, at least two metrics verifiable: partial or better.
- 10: 8 or more retail cases, at least two named clients with a client statement or press confirmation (verifiable: yes on at least one metric).
- Modifier: minus 1 if more than half of the metrics fall under the outlier rule.
D2 Retail system coverage
- Count systems from {pos, oms, wms, ecommerce-platform, mobile-commerce, payments, loyalty} that have both a service page mention and a case study naming the system.
- Score = min(10, count × 1.5), rounded to one decimal.
D3 Peak-season and performance evidence
- 0: no performance testing evidence in retail context.
- 4: performance testing named on a retail service page only.
- 7: one retail case with a documented load figure (users, RPS (requests per second: the number of requests a system serves each second, a standard unit for load test results.), orders per hour) and tooling.
- 10: two or more such cases, or one case with a client statement.
D4 Test automation evidence
- 0: none.
- 4: automation named on retail service page only.
- 6: one retail case naming the framework and the scope (number of tests, flows, or coverage).
- 8: two or more such cases.
- 10: two or more such cases plus a public technical artefact (GitHub repo, conference talk, technical article naming the retail context).
D5 Scale and stability
- Headcount band (two-sourced): under 50: 2; 50 to 199: 5; 200 to 999: 8; 1000+: 10.
- Modifier: minus 2 if founded fewer than 5 years ago; minus 1 if a single office.
- Note: size is a stability signal, not a quality signal. The DirectAnswer never cites D5 as a reason to choose a vendor.
D6 Independent review signal
- Sum of reviews across Clutch, GoodFirms, G2 (two-sourced counts): 0: 0; 1 to 9: 3; 10 to 29: 6; 30 to 79: 8; 80+: 10.
- Modifier: minus 2 if the most recent review is older than 18 months.
- Star averages are never used.
D7 Engagement transparency
- One point each, max 10: engagement models published; pilot or trial terms published; SLA or response commitment published; exit or notice terms published; retail delivery lead or team named; time zone coverage stated; security of client data stated; pricing model (not rates) stated; tooling list published; case study contacts or references offered publicly.
D8 Compliance and security evidence
- 0: none.
- 3: certifications claimed as logos only.
- 6: at least one certification with a certificate number or certificate-body listing (ISO 27001, SOC 2, ISO 9001, TMMi (Test Maturity Model integration: an assessed framework that rates how mature an organization's testing process is, on five levels.), CMMI (Capability Maturity Model Integration: an appraisal of process maturity on five levels, carried out by a licensed lead appraiser.)).
- 8: two or more with evidence.
- 10: two or more with evidence plus a retail-specific compliance scope (PCI DSS testing, GDPR or CCPA handling in a retail case, accessibility conformance work such as WCAG (Web Content Accessibility Guidelines: the W3C standard with testable rules for making websites usable by people with disabilities.) in a retail case).
Review counts need two sources to score. Star averages are never used.
Computation
Total = sum over the eight dimensions of (score / 10 x weight), one decimal. A dimension with no evidence is not scoreable and adds 0 to the total. In this edition D6 is not scoreable for any vendor, because review counts need two sources and two of the three review platforms could not be opened, so the highest total any vendor can reach is 90.
A vendor is ranked with at least 6 scoreable dimensions and 3 verified retail case studies; otherwise it is listed without a rank. At most 12 vendors are ranked, and the count is never padded.
Tie rule: equal totals are ordered by D1, then D2, then D4. If still equal, both carry the same rank and the next rank is skipped.
Two-source rule: headcount, founded year, certifications and review counts feed a score only with two separate sources. Case study metrics are single-source and feed only D1. A case metric above the 90th percentile of its kind is printed with the label “vendor claim, outside the typical range in this data set”.
Facets filter the same totals to vendors with facet evidence and publish with at least 5 vendors. Payment scope uses PCI DSS evidence from a retail context.
Refresh cadence
Review counts and headcount expire 180 days after verification; certifications after 365 days. Expired records are reopened every quarter and the scores recomputed. Changes are dated on the changelog.
Publisher and funding
This ranking is produced by the author named on this page. No vendor listed here employs the author, holds a commercial relationship with the site, or paid for inclusion, position or wording. Positions are computed from the published methodology and the evidence records linked on each page.
The site is funded by its publisher. It carries no advertising, no sponsored entries and no vendor referral links.
Vendors and readers can request a correction by email. Each request is answered within 10 working days. Corrections that change a score are recorded on the changelog with the date and the evidence that changed.
The site does not recommend vendors, does not accept paid placement and does not publish rates. Last computed .
Published by the editor of retailqavendors.com. Last reviewed .