Guide to BLI’s Revamped Solutions Test Reports

Dec 22, 2015 11:22:28 AM

By Jamie Bsales

BLI analysts, editors and art directors have been working the past several months to debut a revamped BLI Solutions Report format, the first of which (on PaperCut MF 15) debuted in September. The goal was to make the report easier to read and digest for our subscribers and their clients, to surface the most pertinent benefits a product delivers, and to provide more differentiation in our ratings scale.

Our new rating system is designed to indicate how a solution fits in among competitive solutions. A solution that meets the industry standard, has no clear shortcomings or outstanding differentiators in a given area, would fall right in the middle of our five-point scale (represented by "bars" on the cover of the report). All scores are relative to other solutions in the same category and in the same class (for instance, a single-user desktop document management application would not be compared to an enterprise-class ECM platform).

First bar: Falls significantly short of the industry standard
Second bar: On the low end of the competitive range, falling short in key areas
Third bar: Meets the industry standard
Fourth bar: On the high end of the competitive range, with clear strengths
Fifth bar: Offers differentiators unique to this category of solution, above and beyond the industry standard, and/or is “best of breed”

Based on performance in each of these test categories, each of which carry different weight, solutions are given an overall rating.

Platinum: A best-in-class solution that goes above and beyond the industry standard with a lack of any substantial negatives
Gold: An excellent solution with clear differentiators
Silver: A competitive offering that meets the industry standard
Bronze: Falls behind in some categories, so take caution if one of those areas is something your business cares strongly about

If a solution does not meet any of the above scores, they will not receive Buyers Lab’s seal.

Note: These scores are not meant to line up with BLI’s old star-based system, which has been discontinued. Additionally, since the categories have been revamped, they do not rate head-to-head with previous reports.

What’s inside?
Our evaluations cover five key areas important to buyers when considering a software solution. Each area is evaluated and rated comparatively to similar solutions.

Features & Productivity: What key features does the solution offer? How do these features help increase productivity in the workplace?
Usability: How easy is the solution to use?
IT Admin & Security: How much IT intervention/administration is required for a solution? How easy is it to administer? What security measures can admins turn on/off?
Support & Training: How accessible is a solution’s help desk? What training is available for customers and resellers?
Value: An evaluation of a product’s pricing and functionality compared to a competitive group.

In addition to the ratings, our reports include several key components, all located within the first four pages.

Overview: Objective overview of the solution, how it’s deployed, differences compared to other versions, etc.
Chief benefits: Benefits the solution has for a customer, not against competitive solutions
Our Take: Our overall assessment of the solution
Strengths & Weaknesses: Positive and negative statements about a solution and how it compares to other solutions in its category
Product Profile: Specs including pricing, operating systems supported, languages available, software integration and more

If you have any questions about the new solutions report, please call 973-797-2100 or email support@buyerslab.com