Skip to Main Content
Auckland Transport

Supplier performanceSupplier Performance

Auckland Transport Performance Assessment by Coordinated Evaluation (ATPACE) monitors supplier performance.

It's a joint initiative between AT Procurement and AT’s business units.

The purpose of ATPACE is:

  • To provide a means of monitoring performance of AT suppliers.
  • To enable continuous dialogue with suppliers and encourage high levels of performance by challenging them to exceed key objectives.
  • To provide an historical database to assist in track record assessment in future tender evaluations.

Print AT’s supplier performance management guide (PDF 466KB)


Supplier performance management guide

PR04-503 (August 2016)


1. Introduction

Auckland Transport Performance Assessment by Coordinated Evaluation (ATPACE) is joint initiative between Auckland Transport Procurement and Auckland Transport’s Business Units.

The purpose of ATPACE is to:

  • Enable continuous dialogue with Suppliers to encourage high levels of performance by challenging them to exceed the key objectives.
  • Provide a means of systematically monitoring performance of AT’s Suppliers.
  • Provide a historical database to assist in Track Record assessment in future Tender Evaluations.

The intention of this guide is to describe the process used in the evaluation of Supplier performance, and provide information about the performance levels and behaviours that AT is encouraging under each performance criteria.

From this guide, the Supplier, who is the subject of the evaluation, will get an understanding where their focus should be to enable the successful delivery of their contract.

1.1 ATPACE responsibilities

Responsibilities for ATPACE evaluation assessments are as follows:

  • AT’s Project Managers who are the AT Representative for the relevant contracts are responsible for completing ATPACE evaluations via ATPACE System.
  • AT Procurement is responsible for running the ATPACE System and relevant processes associated with ATPACE.
  • AT’s Group Managers (GMs) of the relevant Department will sign off the final evaluation via the system.
  • AT’s GMs of the relevant Department and the relevant Procurement Manager are responsible for moderation of evaluations, when required.

2. Completing performance evaluations

 2.1 Performance evaluation process

Performance evaluations shall be completed using the ATPACE System.

For a step-by-step process guide of how to use ATPACE System please see Help Guide posted under ATPACE System library.

2.2 Evaluation frequency

The four main ATPACE performance criteria are:

  • Management.
  • Production.
  • Health and safety.
  • Administration.

Performance criteria, performance evaluation examples and grading scale guidance are listed in the Appendix 1 - Performance Criteria. 

Details of each performance criteria topic relevant to the contract being assessed will be agreed at the first interim ATPACE evaluation. 

2.3 Evaluation frequency

The Procurement Strategy defines the frequency of the ATPACE evaluation.

There is an expectation that performance evaluations will be subject to monthly interim evaluations over the duration of the contract if the contract term is greater than two months. The final evaluation will be completed at contract completion.

  • Monthly For capital development contracts where term is defined by the completion of works/ projects.
  • Quarterly For operational expenditure.
  • Annually For master services agreements (MSA) and general services contracts.

2.3 Performance criteria

The four ATPACE performance criteria are:

  • Management - Skill levels and competency, risk management, creating a ‘no surprises’ environment, responsiveness, creating innovation and proactiveness.
  • Production - Ability to meet programme, timely outputs, achieve the specified standard, accurate outputs to required standards and defect management systems.
  • Health and Safety - Safe work practices and traffic management control.
  • Administration - QA documentation, monthly reporting, financial and handling of variations.

Details of each performance criteria topic relevant to the contract being assessed will be explained at the first interim ATPACE evaluation. If not relevant the score will be ‘not applicable’.

2.4 Evaluation forms and types of evaluation

Evaluation Form are predetermined by the procurement category and the contract type.

2.4.1 Interim evaluations

Interim evaluations are undertaken to provide feedback on a Supplier's performance for the previous month’s work, with the objective of improving performance during the contract. They should be used to:

  • Facilitate performance-related discussions that filter out performance issues or barriers to improving performance.
  • Give objective and real-time feedback.
  • Aid the resolution of differences.

Interim evaluation scores may fluctuate from month to month, and only represent the performance of the Supplier in the previous month. However, they will be used as an input into the final evaluation. If a Supplier’s overall grading is less than 40 or more than 80, additional specific comments are required to be entered in the overall comment box on the evaluation form, to support the evaluation grading.

2.4.2 Final evaluations

The final evaluation should be completed shortly after contract completion or after the defects notification period, and should be an assessment of Supplier’s overall performance for a particular contract, not an average of all interim evaluation scores.

All final evaluations require comments to be entered into the comments box. Such comments shall amplify features of the Supplier’s performance, and any significant issues that arose during the contract period.

The appropriate GM will sign off the final evaluation via the system through ‘accepting’, which ensures that a greater degree of consistency is achieved in the rating of final evaluations.
Where applicable, a Supplier may enter their final evaluation score in the AT Tender documents under Track Record non-price attribute.

2.5 Grading process

The AT Project Manager completes the evaluation by selecting an appropriate performance rating for each performance criteria.  By choosing one of the numbers alongside each of the performance measures that matches that grading, an overall grading will be calculated. Where ‘not applicable’ has been selected the overall grading is averaged out.

For any performance measures where a rate of less than 2 or more than 4 has been given, brief comments are required to clarify the rating. On the ATPACE evaluation forms midpoint has been highlighted green. 

Rating:

  • Rate number 0: Unacceptable - significant issues requiring immediate attention.
  • Rate number 1: Serious reservations - some deficiencies where improvement is needed.
  • Rate number 2: Minor reservations - some minor concerns as noted.
  • Rate number 3: Good (business as usual) - supplier meets all contract requirements.
  • Rate number 4: Very good - supplier exceeds some requirements at some levels.
  • Rate number 5: Excellent - supplier exceeds all requirements at all levels.

2.6 Supplier meeting

This should ideally be a face to face meeting. The ATPACE System has the ability to send a draft evaluation to the Supplier Representative. During the meeting discussion, the initial scores can be adjusted by the Project Manager. Once agreed, this evaluation can then be resubmitted to the Supplier Representative to enable them via the ATPACE System to ‘Accept/Disagree’. Or the Project Manager has the ability to Accept on behalf of the Supplier Representative if appropriate.

If the agreed document is re-submitted to the Supplier Representative (as per the process) and they do not acknowledge by either ‘Accept/ Disagree’ within 7 Working Days, the scores on that evaluation will be automatically accepted.

2.7 Escalation process

The Supplier has the right to challenge any evaluation, and the right to elevate the issue if it cannot be resolved at the initial level i.e. the face to face meeting.
 
Despite this, completion of the interim evaluation form without the ‘Acceptance’ of the Supplier, does not invalidate the evaluation. The resulting evaluation form shall still be considered a bona fide performance evaluation.

In the event that an ATPACE evaluation requires a rating to be moderated, the ATPACE evaluation form will be re-assessed by the applicable AT Project Manager, Supplier’s

Representative, the AT GM of the applicable Department and the relevant AT Procurement Manager.

In the event of any disagreement, the rating assessed by the GM and the Procurement Manager will be the final grading.

2.8 Overall performance grade

Performance grades are split into 6 different categories. ‘Business as Usual’ should be considered as a level where the Supplier fully meets contract requirements without the need for AT intervention.

The overall rating is considered in accordance with the following:

Grade Description Rating
Less than 5 Inability to deliver contract requirements.  Unacceptable
5 - 20 Contract requirement only met through the intervention of Auckland Transport.  Serious reservations
21 - 40 Contract requirement met with minor reservations which may or may not require Auckland Transport intervention.  Minor reservations
41 - 60 Contract requirements fully met without the need of Auckland Transport intervention.  Good 
61 - 80 Contract requirements exceeded with potential added value features identified.  Very good
81 - 100 Contract requirements are consistently exceeded with significant added value features identified and delivered.  Excellent

Business as Usual is the benchmark from which all Suppliers are measured.

  • The expectation is that a Supplier graded ‘Good’ meets the minimum requirements specified in our contract documents in all areas without the need of AT intervention.
  • ‘Minor reservations’ means there are areas of weakness or suboptimal performance in one or more categories, and therefore will result in a score lower than Good. A number of weaknesses, particularly in areas that are identified as critical, will result in a ‘Serious Reservations’, or ‘Unacceptable’ grade.
  • Achieving ‘Very Good’ or ‘Excellent’ grades is possible if the Supplier is delivering better than the specified contract standard, and doing so without AT intervention.
  • ‘Excellence’ requires not only consistency, but value added features such as proactiveness, flexibility, identifying and pursuing innovation, and other factors that would lead you to describe the Supplier as being at the leading edge of the industry.

3. Use of ATPACE information in tender evaluation

In tender evaluations, a Supplier’s previous performance is evaluated under the Track Record non-price attribute. Track Record is an assessment of the Supplier’s record of delivering works or services to the quality standards required, on time and within budget. Suppliers that wish to participate in AT Tenders will be required to include Final ATPACE ratings for contracts nominated under Track Record, as a part of their non-price attribute tender submission.

For reference contracts not undertaken for AT, or for legacy AT contracts, Tender Evaluation Panel will undertake reference checking using ATPACE methodology to asses ATPACE score for that contract.

Where the Track Record attribute is weighted, the average of the nominated contracts ATPACE scores will contribute to 80% of the Track Record score. The other 20% will be determined by using the average ATPACE score for ALL contracts completed by the Supplier in the last 5 years.

Track record in tender evaluation (100%) Contribution
Supplier has previously been assessed by ATPACE

80% - Nominated contracts from RFP response.

20% Supplier's overall ATPACE score for contract completed in the last 5 years.

New supplier with no previous record of AT project delivery or no ATPACE scores. 100% Trace record score on nominated contracts in RFP response.


4. Prequalification performance reviews

An average ATPACE score that falls below 40 should be considered as an early warning sign to flag that a performance review might be necessary. The relevant AT Procurement Manager, in discussion with the appropriate AT GM will make a decision on whether a performance review is necessary.

A performance review will be undertaken on a Prequalified Supplier when the average ATPACE score falls below 20.

As a result of the review the Supplier could be removed from the Prequalification Register for a stand down period of 3 months.


For more information or advice

Contact Auckland Transport