Blog
An ERP Selection Scorecard: How to Evaluate Vendors
A practical, weighted ERP selection scorecard for SMBs: what to score, how to weight it, how to run demos fairly and how to avoid a biased shortlist.
- bottom
Short answer: a good ERP scorecard is a weighted table with five to seven categories, scored by a small cross-functional group against scripted demos of your own processes, with a written note behind every score. Weights come from your priorities, not from a template. The scorecard does not pick the winner for you. It makes the disagreement visible early, while it is still cheap to resolve.
This guide covers what to put in the table, how to weight it, and the traps that skew the result.
Why a scorecard at all
Most ERP shortlists are decided by whoever gave the best demo. That is a presentation skill, not a fit signal.
Failure statistics get quoted a lot. Figures like “55 to 75 percent of ERP projects fail to meet their objectives” are commonly attributed to Gartner, but we could not find a primary Gartner document for them. They circulate secondhand, for example on an implementation partner’s blog, and definitions of “failure” vary (missed objectives, delays, budget overruns, abandonment). Sources like that are often vendors or implementation partners with a commercial interest. Treat the percentages as directional at best.
What is less controversial is the mechanism: many of the causes cited are decided before implementation starts, such as an unstructured selection process, unclear requirements and the wrong partner. A scorecard addresses exactly that stage.
Step 1: Write requirements before you meet vendors
Do this first, or the scorecard just encodes whatever the first vendor showed you.
- List 15 to 30 business processes you run today (quote to cash, procure to pay, stock moves, month-end close, and so on).
- Mark each as must-have, should-have or nice-to-have.
- Note the pain points you are buying ERP to fix. “Inventory is wrong in three spreadsheets” is a requirement. “Modern UI” is a preference.
- Capture compliance needs for your markets: sales tax or VAT, e-invoicing, GDPR, industry rules.
Keep must-haves short. If everything is a must-have, nothing is.
Step 2: Choose categories and weights
There is no standard weighting. One published guide suggests a starting point of 40% functional fit, 20% technical fit, 15% vendor viability, 15% total cost and 10% implementation risk, scored 1 to 5. That is a reasonable starting template, and it is only that. A regulated manufacturer will weight compliance and traceability far higher than a ten-person services firm will.
Here is a layout we use as a starting point for SMBs:
| Category | Starting weight | What it covers |
|---|---|---|
| Functional fit | 30% | Coverage of your must-have and should-have processes, out of the box |
| Implementation partner | 20% | Experience in your industry, references, team continuity, method |
| Total cost of ownership | 15% | Licences, implementation, hosting, support, upgrades over 5 years |
| Technical fit | 10% | Integrations, APIs, data migration, customisation approach |
| Vendor and ecosystem health | 10% | Roadmap, release cadence, community, long-term viability |
| Usability and adoption | 10% | How fast a non-expert can finish a real task |
| Security and compliance | 5% | Hosting region, certifications, audit trails, data protection |
Adjust it. The point is to agree on the weights before any demo, and to write them down. Changing weights after you have seen scores is how a preferred vendor wins on paper.
One more rule: if a vendor fails a hard requirement, such as a legally required e-invoicing format in your market, mark it as a knockout rather than letting strong scores elsewhere average it away.
Step 3: Use a consistent scale
Use 1 to 5 and define each point so scorers mean the same thing:
- 1: Not supported, or needs a workaround we would not accept.
- 2: Possible only with significant custom development.
- 3: Supported with configuration or a well-known third-party module.
- 4: Supported out of the box, with minor gaps.
- 5: Supported out of the box and demonstrated on our data or scenario.
Then multiply each score by its weight and sum. Have each scorer fill in the sheet independently first, then compare. Big gaps between scorers are the most useful output you will get.
Step 4: Run scripted demos
Send every vendor the same three to five scenarios in advance, drawn from your own processes. For example:
- A customer order with a partial shipment, a backorder and a credit note.
- A purchase receipt with a price variance, flowing through to the ledger.
- A month-end close with intercompany or multi-currency entries, if relevant.
Ask them to show it live, in the product, with your sample data where possible. Slides do not count. Anything the vendor says is “on the roadmap” scores as not supported today.
Include the people who will use the system daily. Their question “how many clicks is that?” is worth more than most feature checklists.
Step 5: Score the partner as hard as the software
With many ERPs, including Odoo, the software is only half the decision. The people who configure, migrate and support it determine most of your experience.
Ask for:
- Two or three reference customers of similar size and industry, and actually call them.
- Who will do the work, and whether the people presenting are the people delivering.
- A sample project plan, a sample statement of work and how change requests are priced.
- What happens after go-live: support hours, response targets and upgrade handling.
References are the cheapest due diligence available, and most buyers skip them.
Step 6: Compare cost on the same basis
Licence price is the easy number and the least informative. Compare over five years, using the same assumptions for every option:
- Licences or subscriptions, including the tier you actually need. Check the vendor’s own pricing page for your region, since prices differ by plan, billing period and country. For example, Odoo publishes its per-user plans on its pricing page, and third-party sites sometimes quote introductory discounts that differ from the renewal price.
- Implementation services, including data migration, training and testing.
- Hosting and environments (production, staging), if not included.
- Custom development and its ongoing maintenance, especially across version upgrades.
- Support and internal staff time.
Implementation cost often exceeds licence cost for SMBs, and it depends heavily on how much you customise. Ask each bidder for a fixed-scope estimate with stated assumptions, not a single hourly rate.
Step 7: Sanity-check the result
Before you sign:
- Sensitivity test. Change the top two weights by 5 to 10 points. If the winner flips, the vendors are closer than the sheet suggests, and the tie-breaker is the partner and the references.
- Gut check. If the winner surprises you, find out why. Either the scorecard has a flaw or your instinct does.
- Risk review. List the three most likely ways this project could go wrong and ask each finalist how they would handle them.
- Exit plan. Confirm you can export your data in a usable format. Open-source options such as Odoo Community reduce lock-in at the software layer, but you still depend on whoever maintains your customisations.
Academic work exists on formal multi-criteria methods for ERP selection, such as best-worst method and weighted sum-product ranking. They are useful for large procurements. For most SMBs, a transparent weighted table that people actually understand beats a sophisticated model nobody trusts.
Common mistakes
- Weighting after demos instead of before.
- Letting one executive’s favourite drive the scores.
- Scoring features instead of processes.
- Ignoring the cost of customisation and upgrades.
- Treating the partner as an afterthought.
- Running a 40-vendor long list. Three to five finalists is enough.
A note on our own bias
We implement Odoo, so we obviously think it is a good fit for many SMBs. It will not be the best answer for every company, and an honest scorecard should be able to rule us out. If it does, you are better off knowing that now than six months into a project.
Next step
If you would like a second pair of eyes on your requirements list, weights or vendor shortlist, we are happy to have a no-charge conversation with you. No obligation, and we will tell you plainly if another route fits better. Get in touch with Teknologia Solutions.
Sources: Rand Group, What percentage of ERP implementations fail?; ERP Research, Why do ERP projects fail; ERP Research, ERP selection criteria; Odoo pricing; PeerJ Computer Science via PMC, rough multi-criteria ERP selection model. Several sources are commercial vendors or consultancies, and the failure-rate figures are secondhand. Weights shown are starting points, not standards. Please verify prices, features and statistics against primary sources before making a decision.