A pilot order is a measuring instrument, and like any instrument it has a resolution. Twenty units placed in one store for a month will tell you whether the hardware fits your software and your staff. The same twenty units will tell you nothing about a failure rate below fifteen percent, and nothing at all about how the hardware behaves in year four. Most pilots fail before the first unit ships, not because the product was wrong but because the order was never designed to answer a question.
What a pilot can and cannot prove
The arithmetic is simpler than the argument around it. If a pilot runs n units and none of them fails, the upper bound of the 95 percent confidence interval on the failure rate is roughly three divided by n, a relationship usually called the rule of three. It is the reason a small pilot feels reassuring and proves very little.
| Units in the pilot | Failures observed | Failure rate still consistent with that result |
|---|---|---|
| 5 | 0 | up to 60 percent |
| 10 | 0 | up to 30 percent |
| 20 | 0 | up to 15 percent |
| 30 | 0 | up to 10 percent |
| 60 | 0 | up to 5 percent |
| 100 | 0 | up to 3 percent |
Read the 20 unit row again. Twenty units, zero failures, and the true failure rate could still be fifteen percent. On a 500 unit rollout that is 75 dead or faulty units. The pilot did not lie; the pilot was never big enough to see them.
That is not an argument for ordering 100 units to test a product. It is an argument for deciding what the pilot is for. If the question is whether the hardware integrates with the point of sale software, whether the staff can use it, and whether the supplier ships what was ordered, 20 units answer it well. If the question is the failure rate, only a sample of 60 or more begins to answer it, and a warranty with a clear dead-on-arrival window answers it more cheaply.
Size the order from the defect rate, not from the budget
Turned around, the same rule sizes the pilot. Divide three by the failure rate you are unwilling to miss and you have the smallest sample that can detect it.
| Failure rate you need to detect | Units required with zero failures | What that size of pilot is good for |
|---|---|---|
| 20 percent | 15 units | Dead units, wrong firmware, missing accessories |
| 10 percent | 30 units | Connector faults, configuration errors, mounting problems |
| 5 percent | 60 units | Intermittent faults that appear under a specific store condition |
| 2 percent | 150 units | Rare faults, and only if every unit is logged properly |
The practical band for a first pilot is 15 to 30 units. Below 15, the result is compatible with almost any failure rate and the exercise becomes a demo. Above 30, the pilot starts to cost like a rollout and the argument for calling it a trial weakens.
What twenty units cannot tell you, and why that is fine
A pilot measures integration and handling. It does not measure wear, and the arithmetic makes that obvious. Twenty units running 200 transactions a day for 30 days accumulate 120,000 transactions between them, which is 6,000 per unit. At 200 transactions a day, a cash drawer rated past one million operations needs about fourteen years of trading to reach that rating, which is 119 pilot-lengths of six weeks.
So the question "will it last five years" cannot be answered by running the pilot longer. It is answered by the cycle ratings on the datasheet, by the warranty terms, and by the supplier's own test regime: the drawer cycle count, the printer's cutter life, the scanner's drop rating. The pilot's job is to confirm that those numbers apply to your environment, not to generate them. When a supplier offers to prove durability with a six week trial, what is being offered is a demo with better stationery.
Where the units go
A pilot placed entirely in the best store tells you what the hardware does in the best store. The distribution matters more than the count, and a twenty unit pilot can cover three store types and a bench.
| Location | Units | What it stresses |
|---|---|---|
| Highest volume store | 6 | Throughput, heat, printer duty cycle, queue behavior |
| Average store | 6 | Normal daily use, the case that has to work for everyone else |
| Hardest environment | 5 | Grease, dust, sunlight, cold room, whatever the estate actually contains |
| Integrator bench | 3 | Configuration, imaging, software updates, a unit to break on purpose |
Two rules make that distribution work. Every site runs the same build and the same software version, or the results cannot be compared. And at least one unit is left unboxed at the integrator's bench as a swap unit, because a pilot with no spare stops being a measurement the first time a unit fails and becomes a support case.
How long, and the window that expires while you run it
Four weeks is the minimum useful duration: it covers a monthly close, a stock count and at least one peak day. Six weeks is better, because most retail estates have a payroll or ordering cycle that only comes around twice a month, and the failures that matter tend to sit on those days.
Duration has a commercial consequence that is easy to miss. Our dead-on-arrival window is 30 days, and a six week pilot runs past it. If the window is read from delivery, the last two weeks of the pilot are outside it. The fix is one sentence in the order: the dead-on-arrival window starts when the pilot starts, or is extended to the end of the pilot. Suppliers who understand pilots agree to that without argument; a supplier who will not is telling you something about the after-sales process.
The same logic applies to the production lead time. A pilot at 20 units sits inside the 7 to 20 working day band, so the units can be on a counter about a month after the order. A 500 unit rollout sits in the 15 to 50 working day band, which means the pilot's result arrives while the rollout order is already being placed. That overlap is normal, and it is the reason the acceptance criteria have to be agreed before the pilot ships rather than after the results are in. Comparing the two quotations on one basis is a separate exercise, covered in reading several quotations side by side.
The four numbers worth logging
Pilot reports usually arrive as a list of incidents, and a list of incidents cannot be compared with anything. Convert to rates and the numbers become decisions. Every rate needs a denominator, and the denominator in retail hardware is transactions, not days.
| Metric | Denominator | What it reveals |
|---|---|---|
| Failure events | per 1,000 transactions | The headline number, comparable across sites and against the rollout |
| First-read rate | per 1,000 scans | Scanner and label quality; the number that shows up as queue time |
| Drawer open failures | per 1,000 cash sales | Kick-out voltage, cable seating, solenoid wear |
| Receipt jams and paper-outs | per 10,000 receipts | Printer duty cycle and paper specification |
With 20 units at 200 transactions a day for 30 days the pilot generates 120,000 transactions, or 6,000 per unit. That denominator is what makes the result portable: one failure per 6,000 transactions is 0.17 per 1,000, and the same rate on a 500 unit estate at 200 transactions a day is 17 failures a day. Presented that way, a pilot result becomes an operating number rather than an anecdote, which is also the form in which it survives the meeting where the rollout budget is decided.
Acceptance criteria written before the order
The most common pilot failure is not a hardware fault. It is a pilot that ends without a decision, because nobody agreed in advance what a pass would look like. The criteria belong in the order document, next to the quantity.
| Criterion | Typical target | Owner |
|---|---|---|
| Integration | Every pilot unit completes a full trading day on the live point of sale | IT or integrator |
| Failure rate | No more than one hardware failure per 1,000 transactions | Operations |
| Staff acceptance | No manual workaround used after week two | Store manager |
| Supplier response | Replacement unit dispatched within an agreed number of working days | Purchasing |
Separately from the targets, write a stop list: the faults that end the pilot immediately rather than counting against the rate. Payment failure, any event that loses a transaction, any safety issue. Those are not statistics, and treating them as a percentage is how a rollout learns to live with them.
Spares and dead-on-arrival units at pilot scale
The spare parts ratio that works at volume does not survive the arithmetic at pilot size. Our standard is two percent of units, shipped in the same carton. On a 500 unit order that is 10 spare units, which is a sensible buffer. On a 20 unit pilot it is 0.4 of a unit, which is nothing at all.
So the pilot needs a spare rule written for its own size: one spare per ten units, or one per site, whichever is greater. Past the pilot, on an estate of several hundred units, the ratio starts to work as intended, and what the two percent allowance actually covers becomes the number worth arguing about. Twenty units across three sites becomes two to three spare units, and at least one of them stays unboxed at the bench so a swap takes minutes rather than a support ticket. The cost of those units is trivial next to a pilot that stops for a week waiting for a replacement.
The dead-on-arrival definition matters at the same scale. A unit that arrives dead in a 500 unit order is a statistic. A unit that arrives dead in a 20 unit pilot removes five percent of the sample on day one, and if it is replaced slowly the pilot quietly loses a week of the calendar it was designed around.
The commercial shape of a trial
Two order sizes sit on either side of a line that changes the terms. Up to five units counts as a sample: charged, freight not refunded, and the sample charge credited against a follow-on order of about one hundred units. More than five units counts as a trial: priced at production rates, and not refundable.
That line explains a question buyers ask often, which is why a six unit pilot costs proportionally more than a five unit one. It is not a pricing quirk; it is the boundary between buying a sample and buying a small production run, and the commercial risk moves with it.
There is a second boundary above it. Customization carries a 50 unit minimum on the OEM and ODM program, so a 20 unit pilot is either a standard configuration or a pre-production build priced above the production rate, and the follow-on order still carries the 50 unit minimum. The sensible sequence for most projects is a standard-configuration pilot to prove integration, a customized order at 50 units or more once the configuration is settled, and a phased rollout from there. The ordering terms set out the current numbers for minimums, lead times, dead-on-arrival handling and spares.
What changes at five hundred units
A rollout is not a large pilot, and the first thing to fix is the number of terminals each site actually needs, which the lane sizing formula answers before any hardware is ordered. Five things then change at volume, and all five are easier to plan while the pilot is still running than after the purchase order is signed.
| Item | Pilot, 20 units | Rollout, 500 units |
|---|---|---|
| Build | Standard configuration | Customized, 50 unit minimum per configuration |
| Lead time | 7 to 20 working days | 15 to 50 working days, staged by site |
| Spares | One per ten units, by agreement | Two percent, shipped with the goods |
| Configuration | Done on the bench, unit by unit | Pre-loaded before dispatch, serialised and logged |
| Certification | Whatever the pilot country requires | Per product and per market, confirmed before production |
The certification row is the one that catches projects late. A pilot in one country proves nothing about the paperwork for the next four, and approvals are the longest lead item in most rollouts because they sit outside the factory's control. The question to ask during the pilot is not whether the product is certified, but which markets the rollout will touch and whether the certificates for those markets exist or have to be applied for.
Five lines for the pilot order
- Units and configuration. The exact count, the exact build, and whether it is a standard configuration or a pre-production run.
- Sites and distribution. Which stores, how many units each, and which unit stays at the bench as a swap.
- Duration and the dead-on-arrival window. Start date, end date, and one sentence moving the dead-on-arrival window to the pilot period.
- Acceptance criteria. The rate thresholds, the stop list, and the name of the person who signs the result off.
- What happens next. The follow-on quantity, the credit arrangement if the pilot was a sample order, and the date by which the rollout decision is made.
The five lines are the same document as the RFQ that precedes the order, carried one stage further. The fifth line is the one that turns a pilot into a decision. Without a date and a number, a successful pilot becomes a permanent trial, and the estate ends up with twenty units proving something nobody has written down. A pilot designed this way costs the same as one designed badly; the difference is that it ends with an answer. Where the pilot is the first step of a multi-site program, planning the rollout and testing hardware before a bulk order cover the stages on either side of it.
Frequently asked questions
How many units should a pilot order have?
Set the number from the failure rate you are willing to miss, not from the budget. With zero failures, the upper 95 percent confidence bound is roughly three divided by the sample size, so 20 units with no failures is still consistent with a 15 percent failure rate, 30 units with 10 percent, and 60 units with 5 percent. A 15 to 30 unit pilot is enough to catch dead units, wrong configurations and integration faults. It is not enough to prove that a product is reliable.
Is a pilot order refundable?
It depends which side of the line it falls on. Up to five units counts as a sample order: the units are charged, freight is not refunded, and the sample charge is credited back against a follow-on order of about one hundred units. More than five units counts as a trial: units are priced at production rates and are not refundable. That boundary is worth knowing before the pilot is sized, because a six unit pilot is commercially a different thing from a five unit one.
How long should a pilot run?
Four weeks is the minimum that covers a full monthly close and at least one peak day, and six weeks is better because it covers two payroll or ordering cycles. Check the dead-on-arrival window against that duration before the units ship. Our window is 30 days, so a six week pilot runs past it, and the sensible move is to agree in writing that the window starts when the pilot starts or runs to the end of it.
Does a small order cost more per unit?
Usually, and for a reason worth understanding rather than arguing about. Customization has a 50 unit minimum, so a 20 unit pilot is either a standard configuration or a pre-production run priced above the production rate, and the follow-on order still carries the 50 unit minimum. A pilot placed as a standard configuration at production price is the cheapest way to test integration, and it leaves the customization decision for the order that follows.
What should a pilot actually measure?
Rates, not counts. The denominator is transactions, and without it a report of three failures means nothing: three failures in 2,000 transactions is a different product from three in 200,000. Log failures per 1,000 transactions, first-read rate on scanners, drawer open failures per 1,000 cash sales, and receipt jams per 10,000 receipts. Twenty units running 200 transactions a day for 30 days produces 120,000 transactions, which is a denominator worth comparing against the rollout.
Can a pilot prove the hardware will last five years?
No, and it is worth being explicit about why. Twenty units for six weeks at 200 transactions a day accumulate about 8,400 transactions each, against a drawer rated past one million operations. At that pace the rating is 119 pilot-lengths away, or about fourteen years of trading. A pilot answers whether the hardware fits the store, the software and the staff. Wear-out is answered by the cycle ratings on the datasheet and by the warranty terms, not by six weeks on a counter.
