A software firm writing a guide to choosing a software firm has an obvious conflict of interest, so let’s start with the credential that matters: a meaningful share of our work is inheriting projects that went wrong somewhere else. Rescues, rebuilds, modernisations of systems three years old and already legacy. We see what the wrong choice looks like from the inside, eighteen months after the contract was signed — and the pattern is remarkably consistent. The buyer almost never chose badly out of carelessness. They chose badly because the standard evaluation process measures the wrong things.
From the outside, development firms are nearly indistinguishable. The websites show the same case studies, the same logos, the same claims about senior engineers and agile delivery. Ours included. And the tools buyers reach for to break the tie — portfolios, references, rate cards — are precisely the tools every firm has spent years polishing. This is a checklist for looking past them.
Why references and portfolios mislead
A portfolio is a highlight reel. Every firm shows its best three projects, and the selection tells you nothing about the median engagement — a portfolio is survivorship bias by construction. The projects that ran a year over, shipped and were quietly abandoned, or ended in a dispute are not on the website, and confidentiality hands every firm a respectable reason for the gaps. Reading a portfolio tells you the firm’s ceiling. You are going to live at its median.
References have the same flaw with a friendlier face: the firm picks them. You will be given the three happiest clients of the last five years, and you will never be given the number of the one whose project was abandoned. Even a genuinely glowing reference is weak evidence, because the outcome depended on the specific team staffed that year — and the people who built the flagship case study may have moved on twice over since.
References are still worth calling — you just have to ask different questions. Not “were you happy”, which invites a rehearsed yes, but “tell me about the worst month of the project, and what they did about it”. Every engagement of any size has a bad month. Whether the firm surfaced the problem early, repriced honestly and fixed it — or went quiet and kept invoicing through it — is the single most informative thing a reference can tell you.
Four signals that actually predict outcomes
The first is how they estimate. A firm that gives you a number before it understands your operation is telling you how it works. A real estimate is downstream of real questions — about your workflows, your data, the systems you already run, the exceptions that make your business yours. If the price arrives before those questions do, it was not derived from your project; it was set to win it, and the difference between those two numbers will surface later, dressed as change requests.
Estimation behaviour predicts delivery behaviour. A firm that guesses at the price will guess at the build.
The second is what they say no to. Ask directly: what projects have you turned down, and why? A firm that cannot name one either takes everything that pays — which means your project will be staffed by whoever is free, not whoever fits — or has never thought hard about where it is actually good. The shape of a firm’s refusals is the most honest map of its competence you will get in a sales process.
The third is who writes the code. The people in your first three meetings are usually not the people who will build your system. Ask to meet the tech lead who would own your project — the actual engineer, not the head of delivery — and put technical questions about your problem to them. A firm confident in its engineers will put them in front of you without hesitation. A firm that keeps you at the account layer is managing an impression.
The fourth is what handover looks like. The end of an engagement is where incentives diverge most: everything before it, the vendor is motivated to do well; documentation, handover and your independence are the parts that only cost them. So ask for evidence — a redacted architecture record, runbook or handover pack from a finished project. A firm that can produce one has done this before and expects you to be able to run without them. A firm that talks about partnership instead of producing a document is describing lock-in politely.
Red flags worth walking away from
Bait-and-switch seniority is the most common. Senior engineers run the pitch and the discovery; the commit history tells a different story by month two. Protect against it structurally, not through trust: named individuals in the agreement, the right to meet any replacement before they are staffed and a look at who actually turns up to the standups.
Hourly billing with no scope discipline.Hourly is not inherently wrong — but hourly with no milestones, no written scope and no definition of done means the vendor earns more the longer your project takes, and nothing in the structure pushes back. The question to ask is not “hourly or fixed” but “where does accountability for the estimate sit”. If the answer is with you, then you are the scope discipline.
“We can start Monday.” A firm whose whole team is available tomorrow is telling you something about demand for its work. Genuine gaps happen — a project ends, a start date slips — but an entire senior team idling on the bench is rare at any firm worth hiring. Treat instant availability at scale the way you would treat a surgeon with no waiting list.
No opinion about the boring parts of your industry. Every domain has unglamorous machinery — reconciliation, regulatory edge cases, pricing exceptions, scheduling rules — and it is where projects actually sink. A partner does not need prior experience in your industry, but they do need to light up at the boring parts, because that is what they will spend most of the project building. If the conversation keeps drifting to the exciting surface of the product, the dull nine-tenths will be learnt at your expense.
Eight questions for the first call
Signals are easier to read when you ask for them directly. These are worth asking verbatim, in roughly this order.
- “What would you need to learn about our operation before you’d put a number on this?”
- “What projects have you turned down in the last year, and why?”
- “Who exactly would write the code — can we meet them before we sign?”
- “Tell me about a project that went badly. What did you change afterwards?”
- “What does handover look like — can we see a redacted example from a real project?”
- “What would you cut from our brief to get a first version live sooner?”
- “What happens if we want to stop after three months?”
- “How would we know, six months in, whether this is working?”
Listen for specificity and mild discomfort. Good answers name real projects, real mistakes and real constraints; bad answers are smooth, generic and instantly agreeable. Question six tests scope judgement — the right partner will argue for a focused first version in 2–3 months over a platform in a year, and will tell you which parts of your brief can wait. Question seven is the lock-in test: a partner planning to earn the renewal makes leaving cheap — your repositories, your infrastructure, documentation as a deliverable — while a vendor planning to own you will get vague.
De-risk the decision: start smaller than the contract
The deepest problem with partner selection is not any single signal; it is that the standard process asks you to decide everything at once — two calls, a proposal, a six-figure commitment. You cannot interview your way to certainty about how a firm behaves under real conditions, but you can buy a small sample of it. Instead of signing the big contract, start with a short diagnostic engagement: a week or two inside one workflow, ending in an estimate, an architecture sketch and a payback case you own regardless of what happens next.
A diagnostic converts every signal in this post from a claim into an observation. You watch how they estimate instead of asking about it. You meet the engineers who would actually build, because a diagnostic is technical work the account team cannot do. And you end up holding an artefact you can take to a different firm — which is the acid test. If the output is only useful if you hire them, it was a sales document with a deliverable’s haircut. Some firms charge for this and some run a version of it free — we do, and it is how every engagement of ours begins — but paid or free matters less than the shape: small, bounded and with an output that survives the relationship.
It is also the cheapest way to find out that a firm is wrong for you — including ours. Run this checklist properly and you may well conclude that a different partner fits your project better than we do. That is not a failure of the checklist; that is the checklist working. The point of starting small is that the conclusion, whichever way it lands, costs you two weeks instead of two quarters.
The compressed version: ignore the highlight reel and read the firm through its behaviour. The partner you want prices only after understanding, can tell you what it refuses, puts engineers in front of you, shows you a real handover document and structures the engagement so that leaving is easy. Because how a firm behaves before the contract — while it is still trying to win you — is the ceiling of how it will behave after.