<<Project Name>>

Pilot Review


Customer Name




Directions for using template:

Read the Guidance (Arial blue font in brackets) to understand the information that should be placed in each section of this template. Then delete the Guidance and replace the placeholder within <<Begin text here>> with your response. There may be additional Guidance in the Appendix of some documents, which should also be deleted once it has been used.


Some templates have four levels of headings.  They are not indented, but can be differentiated by font type and size:

You may elect to indent sections for readability.






Author Position








Version: 1.0





Ó 2002 Microsoft Corporation. All rights reserved.

The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication.

This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS DOCUMENT.

Microsoft and Visual Basic are either registered trademarks or trademarks of Microsoft in the United States and/or other countries.



Revision & Sign-off Sheet

Change Record




Change Reference
































Version Approved
































Document Properties



Document Title

Pilot Review



Creation Date


Last Updated





Table of Contents

 TOC \o "1-3" \h \z

Summary. PAGEREF _Toc13632450 \h 3

Objectives. PAGEREF _Toc13632451 \h 3

Success Factors and Metrics. PAGEREF _Toc13632452 \h 3

Pilot Feedback Summaries. PAGEREF _Toc13632453 \h 3

Sessions with the User Community. PAGEREF _Toc13632454 \h 3

Problem Reports. PAGEREF _Toc13632455 \h 4

End-user Surveys. PAGEREF _Toc13632456 \h 4

Observations of the Project Team.. PAGEREF _Toc13632457 \h 4

Statistical Operations Reports. PAGEREF _Toc13632458 \h 4

Project Team Feedback. PAGEREF _Toc13632459 \h 4

Training Feedback. PAGEREF _Toc13632460 \h 5

Deployment Feedback. PAGEREF _Toc13632461 \h 5

Support Feedback. PAGEREF _Toc13632462 \h 5

Communications Feedback. PAGEREF _Toc13632463 \h 5

Problems Encountered. PAGEREF _Toc13632464 \h 5

Suggestions for Improvements. PAGEREF _Toc13632465 \h 6

Pilot Results and Recommendations. PAGEREF _Toc13632466 \h 6



[Introduction to the Template


Description: The Pilot Review document records the results of the process that validates that what is delivered during the pilot meets specifications and business requirements.


Justification: The pilot’s success or failure is typically indicative of the progress of the overall project. Capturing the results of the pilot provides valuable insight into the solution’s value. If the development and testing activities rigorously followed the specifications and design, the pilot should go smoothly. If the pilot results are less than favorable, changes to the project, small or large, can be made as appropriate.


{Team Role Primary: Program Management will facilitate the creation of the pilot review document, using the pilot plan (developed during the planning phase) as context. Program management should collect input from representatives of all pilot participants (team members and end-users) regarding their feedback on the pilot.


Team Role Secondary: Product Management will employ the pilot review content that describes end-user satisfaction and make any necessary changes to communication or other customer centric activities. Development will need to act on any solution anomalies reported in this review. Test may need to modify testing procedures to either shrink or enlarge the testing criteria. User Experience addresses any issues that are negatively affecting users. Release Management will review the pilot results to understand if any part of the deployment process or operational environment needs modification to ensure a successful implementation of the entire solution.}]





[Description: The Summary section summarizes the results of the Pilot Review document, identifying whether the pilot was approved or received conditional approval and including key recommendations and required action items.


Justification: Some project participants may need to know only the highlights of the review, and summarizing these creates that user view. It also enables the full reader to know the essence of the document before they examine the details.]

<<Begin text here>>



[Description: The Objectives section lists and describes the key objectives to be met by the pilot review process. This information is a re-statement of the pilot objectives found in the Pilot Plan, which was created during the planning phase.


Justification: Restating pilot objectives provides the reader with sufficient context to make the remainder of the document meaningful.]

<<Begin text here>>


Success Factors and Metrics

[Description: The Success Factors and Metrics section restates the success factors and metrics contained within the Pilot Plan. This includes a list of the factors that will be used to judge the pilot as a success, partial success, or failure and the metrics to be used to make these judgments. This section also records the results of the judgments.


Justification: Restating pilot success factors and metrics provides the reader with sufficient context to make the remainder of the document meaningful.]

<<Begin text here>>


Pilot Feedback Summaries

[Description: The Pilot Feedback Summaries section contains summaries of the feedback obtained from all the pilot participants. An efficient way of collecting the original feedback data is via web site forms.


Justification: Each participating team is assessing pilot results from a different perspective. By including feedback from all teams, the project ensures that the pilot was performed and assessed from a broad range of viewpoints.]


Sessions with the User Community

[Description: The project team holds many sessions with the user community during the planning, development, deployment, and operation of a pilot. The Sessions with the User Community section lists and describes each session; defining its objectives, listing user feedback and recommendations, and defining responses and actions taken by the pilot team.]

<<Begin text here>>


Problem Reports

[Description: Pilots help to identify and define many types of problems that might occur during the development and deployment of a full solution at a far smaller cost than the development of a full solution. Problems can be documented using project problem report templates to ensure that all participants are recording standard information. The Problem Reports section contains a summary of all problem reports.]

<<Begin text here>>


End-user Surveys

[Description: A primary objective of pilots is to gain information from users about the usability of the proposed solution. The End-user Surveys section should include the objectives and a description of each survey, the information gathered and recorded in the survey, and conclusions reached from analyzing survey results.]

<<Begin text here>>


Observations of the Project Team

[Description: The project team observes the deployment and operation of the pilot to gather and record specific sets of required information as well as ad hoc observations. The Observations of the Project Team section includes the information gathered and recorded by the project team and their conclusions and recommendations.]

<<Begin text here>>


Statistical Operations Reports

[Description: The Statistical Operations Reports section defines the statistics that were both planned and gathered during pilot operations, summaries of the statistics, and conclusions based on the statistics.]

<<Begin text here>>


Project Team Feedback

[Description: The Project Team Feedback section contains feedback from project teams. This feedback is collected during and at the end of the pilot in order to evaluate its success.


Justification: Each participating team is assessing pilot results from a different perspective. By including feedback from all teams, the project ensures that the pilot was performed and assessed from a broad range of viewpoints.]


Training Feedback

[Description: The Training section describes the training provided to end-users, operations personnel, and other groups that participated in the pilot program. It describes the effectiveness of the training, identifies and describes training problems, lists lessons learned, and makes recommendations that will make the training on the actual solution effective.]

<<Begin text here>>


Deployment Feedback

[Description: The Deployment section summarizes the pilot deployment plan and compares the actual deployment with the plan, listing and defining problems encountered and solved, lessons learned, and recommendations for deployment of the full solution.]

<<Begin text here>>


Support Feedback

[Description: The Support section describes the support planned and provided for the pilot solution. It includes the actual support needed to conduct the pilot, support problems encountered and solved, and lessons learned that can be extrapolated into support for the actual solution.]

<<Begin text here>>


Communications Feedback

[Description:  The Communications section summarizes the pilot communication plan and describes actual communication experiences with pilot stakeholders. It includes the actual communications that occurred during the pilot, communication problems encountered and solved, and lessons learned that can be extrapolated into communication for the actual solution.]

<<Begin text here>>


Problems Encountered

[Description: The Problems Encountered section lists and describes the problems encountered in every aspect of the pilot (planning, development, deployment, training, operations, management, user involvement, support, etc.). It also describes the impact of the problems, solutions applied to the problems, and lists all unsolved problems and the challenges to their solutions.]

<<Begin text here>>


Suggestions for Improvements

[Description: The Suggestions for Improvements section lists and describes all recommended improvements to the pilot, including extensions to the current pilot and any additional pilots. Identify and define their objectives for each extension and additional pilot.]

<<Begin text here>>


Pilot Results and Recommendations

[Description: The Pilot Results and Recommendations section contains a complete evaluation of the pilot's success and makes all necessary recommendations. It describes how well the pilot was received by the users and any usability issues or problems that need to be corrected prior to developing the full solution. This section should identify the specific changes or improvements that need to be made from both a technical and project management perspective.


Justification: The quality goals of the project and the business objectives of the solution are furthered by a complete and objective assessment of the pilot.]

<<Begin text here>>