3c. Bid evaluation

This section will help you:

Conduct an objective, fair, quantitative and qualitative assessment of bids

Consider direct and indirect costs associated with full software lifetime

Conduct thorough technical evaluation that mitigates downstream contract issues

Common challenges

You may encounter these frictions as you do the work of bid evaluation. These are challenges the Recommended actions are designed to solve, or that may arise as you take those actions.

Cost eclipsing quality

Quality, user experience, and integrity are very difficult to quantify. In an effort to maintain objectivity, evaluation processes often default to cost-only comparison, without adequately accounting for the actual value that the solution will generate.

Fear of the unknown

Choosing legacy solutions can feel safer—they appear proven or dependable. These legacy systems may be easier to understand and evaluate, but may not be the best solution for the problem statement.

Business model pigeonholes

Procurement norms were established for awarding high-value contracts to private sector companies. But defaulting to private sector vendors, without adequately considering non-profit or open source solutions, may sacrifice cost, quality, or opportunities for value-creating solutions.

Poor vendor interactions

Ineffective or disrespectful communications can result in bad relationships and compromise future procurement.

Recommended actions

1. Creating a selection panel

The selection panel will review bids using the evaluation framework stated in the RFP. Each member will give their professional assessment of the quality of each bid, working separately and together to compare options and assign scores.

Form the selection panel

(see guidelines below)

Guidelines for creating a selection panel

The panel should reflect the stakeholder group, including:

A representative from the primary team or business unit that is leading the procurement

A technical expert

A representative from the group of end users

Additional individuals who were not involved with writing the tender

Do not include people who have conflicts of interest.

Anyone who stands to benefit or lose, on a personal or professional level, from a particular outcome of the RFP award

Panelists should be briefed on:

A summary of the discovery research

The evaluation metrics

A summary of the vendor interactions

2. Conducting a quantitative and technical evaluation

It is important to conduct a full technical evaluation of software on offer. Does it match up to vendors’ claims? Will it work within the city’s digital environment? Is it secure?

You many need to replicate the city’s digital environment outside the city firewall

This will enable vendors (or other developers) to run demonstrations on it.

Not every municipality has technical staff who can perform technical analysis

You shouldn’t be responsible for full technical auditing. Consider hiring outside impartial technical consultants.

Software may not be testable out of the box

You can require vendors to demonstrate a clear integration and deployment plan and include the test as a contingency in the contract.

Form a technical committee

Internal staff or independent experts should perform a technical review. Explore hidden costs and issues, and determine the truth of vendors’ claims.

Involve your IT department

Test the software and ensure that it is compatible with existing technical infrastructure at the city.

3. Conducting a qualitative, user-oriented evaluation

If you are comparing existing solutions, you can test how well they address your problem statement by involving the end users.

The most important evaluation will come from end users

These are the people who will eventually be using the software, so they should be directly involved in the evaluation process.

Ask vendors for access to the software and/or schedule demonstrations

Run the potential software through the scenarios included in the RFP (literally or as a thought experiment). Invite all of the end users who contributed to the discovery research and project scoping.

4. Considering value beyond cost

Because you introduced multiple types of criteria into the RFP, you can evaluate proposals based on a number of important factors beyond cost, such as quality, long-term value creation opportunities, alignment with organizational priorities, and risk.

Panelists should review all bids individually and separately

This should happen before sharing scores and comparing options through discussion.

Start by scoring the optional criteria

These should give you a good sense of value beyond cost. Consider also your interactions with each vendor. Can you envision them as good partners?

5. Post-selection

The actions you take after making a selection are crucial for building long-term relationships with the software development community—which is important for the success of future projects.

Continue to communicate with vendors

Even those who do not win. It is a demonstration of respect.

Give all vendors a post-award debrief

Send every bidder a copy of the evaluation and the winning proposal, and offer to do debrief calls.

Conduct a selection process evaluation

Did the selection panel find the evaluation criteria effective? Were the categories limiting? Were the end users involved in a meaningful way?

Takeaways

Create or complete the following outputs before moving on to the next step.