Introduction
Over the years of his work in the field of IT and especially in IT sales, he saw many pilot projects, but most of them ended in nothing with a significant expenditure of time.
At the same time, if we are talking about testing hardware solutions, such as storage systems, for each demo system there is usually a queue almost a year in advance. And each test in the schedule can bring a sale or, on the contrary, screw up a sale. It makes no sense to consider a situation in which testing does not affect the sale, since testing does not make sense either - this is a waste of time and the occupation of a demo system.
So, how do you do everything wisely and make everything happen?
Prepare
Pilot Goals
Where does the pilot start? Not from connecting equipment to a rack, not at all. Before starting any work with the equipment, there is work with documents. And we start by defining the goals of the pilot.
The goal of the pilot is to eliminate objections from the end customer. No objections - no need for a pilot. Yes Yes exactly.
But what are the main classes of objections we can see?
* We doubt the reliability
* We doubt the performance
* We doubt scalability
* We doubt compatibility and ability to work with our systems
* We do not believe in your slides and want to make sure in practice that your system really knows how to do all this
* It will all be very difficult, our engineers are already loaded and it will be hard for them
In total, in the end, we get three main types of pilot testing and, as a special case of the pilot, proof of concept (PoC - proof of concept):
* Load testing (+ scalability)
* Functional testing
* Fault tolerance testing
In a specific case, depending on the doubts of a particular customer, different goals can be combined in the pilot, or, on the contrary, only one of them can be present.
The pilot begins with a document describing in Russian in white - why this testing is being carried out. It also includes a mandatory set of measurable criteria that allow you to unambiguously say whether the pilot passed successfully or what specifically was not passed. Measurable criteria are either numeric (such as latency in ms, IOPS) or binary (yes/no). If your pilot has an unmeasured value as a criterion, there is no point in the pilot, it is only a manipulation tool.
Equipment
The pilot can be conducted on the demo equipment of the vendor / distributor / partner or on the customer's equipment. Strictly speaking, the difference is small, the general approach is the same.
The main equipment question BEFORE starting the pilot is is the complete set of equipment present (including switches, data cables, power cables)? Is the equipment ready for testing (correct firmware versions, everything is supported, all lights are green)?
The correct sequence of actions after determining the goals of testing is to fully prepare the equipment for testing BEFORE it is handed over to the customer. Of course, there are loyal customers without haste, but this is rather an exception. Those. the complete set must be assembled at the partner's site, everything is checked and assembled. Without fail, the system must be running and you must make sure that everything works, the software is distributed without errors, and so on. It would seem nothing complicated, but 3 out of 4 pilots start by looking for cables or SFP transceivers.
Separately, it should be emphasized that as part of checking the demo system, you must make sure that it is clean. All data from previous testing must be deleted from the system without fail before transfer. It is possible that it was tested on real data, and there can be anything, both trade secrets and personal data.
Test program
Before the equipment is handed over to the customer, a testing program that meets the testing objectives must be prepared without fail. Each test should have a measurable result and clear criteria for success.
The testing program can be prepared by the vendor, partner, customer or jointly - but always BEFORE the tests start. And without fail, the customer must sign that he is satisfied with this program.
People
As part of the preparation for the pilot, it is necessary to agree on the dates of the pilot and the presence of all the necessary persons and their readiness for testing, both from the vendor / partner and from the customer. Oh, how many pilots started with the departure of the main person in the pilot from the customer on vacation the next day after the installation of the equipment!
Areas of responsibility / access
The pilot's program should clearly understand and ideally describe the responsibilities of all those involved. If necessary, coordinated remote or physical access of vendor / partner engineers to the customer's systems and data with the customer's security service.
Pilot
If we have completed all the previous points, then the most boring part is the pilot himself. But it has to go like it's on rails. If not, then part of the preparation was screwed up.
Pilot Completion
Upon completion of the pilot, a document on the testing carried out is prepared. Ideally with all the tests in the program with a green checkmark PASSED. It is possible to prepare a presentation for senior management to make a positive decision on the purchase or inclusion in the list of systems allowed for purchase.
If you donβt have a document with a list of completed tests and marks passed at the end of the pilot, the pilot failed and it should not have been started at all.
Source: habr.com