|
rev:110127
During our FSTD Evaluator courses and in communication with many of our graduates around the world, we have recieved a number of questions which relate to the new ICAO Document 9625 issue 3. Although different people have their own questions that require answering, invariably they are concerning the adoption of 9625 issue 3 and how that may effect their own situation. In addition, some National Regulatory Authorities (NAA) are wanting to apply the new criteria in order to take advantage of the work that has been done by the members of the IWG.
The partners and colleagues of Training Technology International decided that it may be beneficial to all in our industry if we published an overview of what we have been asked, along with some simple explanations and a suggested plan to apply the criteria to an authorities existing qualification process.
In addition, we have added a number of FAQ's (Frequently Asked Questions) with the answers. You can jump to these directly by clicking here...
We would most appreciate your feedback on the following with any suggestions, or additional questions, so that we can continue to improve this document. To contact us, please click here...
ICAO DOCUMENT 9625 - MANUAL OF CRITERIA FOR THE QUALIFICATION OF FSTD
"recommendations for its use in formulating a regulatory document"
Introduction
Late in 2011, ICAO will issue Volume II of the Document 9625 Issue 3. This Volume, covering the qualification criteria for helicopter FSTD., follows on from the publication in 2009 of Volume I, that for fixed wing FSTD. The Volume II draft, approved by an RAeS international conference in November 2010, is to be passed to ICAO in February 2011, where it will be reviewed by their Editorial Department to ensure that it meets all the ICAO requirements. This latter exercise is purely one of formatting and it will not change any of the technical criteria established by the International Working Group which, over the past three years, has established what type of devices are required to meet the unique helicopter training needs for the use of FSTD.
The introduction of Volume I (and, it is likely to be repeated when Volume II appears), resulted in several National Regulatory Authorities (NAA) wanting to apply the new criteria in order to take advantage, not only of new technology, but also to ensure that the devices they are to qualify, meet the training need. What some of these NAA have not appreciated is that Document 9625 is not intended to be a Regulatory Document or Rule. It is meant as guidance for the criteria to be incorporated into a Regulatory Document issued by a NAA.
In formulating Document 9625., its writers were careful not to write it as a Regulatory Document because:-
- To do so would be usurping one of the roles of the NAA
- The administrative procedures for the adoption and use of a Regulatory Rule differ from NAA to NAA.
As a result, Document 9625 does not contain any administrative information, it being intended that each NAA will adopt the criteria and use them, as they wish, in Rules written to meet their own organizational requirements. This is clearly stated in the Document under para 2.12 of Part II. In this manner each NAA can use the criteria so that they are common to all other NAA, thereby improving “reciprocity” of qualifications but, at the same time reflecting any particular administrative needs of their own Nation.
This has presented some NAA with a problem because they wish to use Document 9625 criteria and take advantage of the new types of FSTD., but they do not have a readily available National Rule to facilitate this. Large NAA such as EASA., FAA., Transport Canada and the Australia CASA are thought to be working on new rules based upon Document 9625 but it is not a task likely to be completed quickly. EASA are already engaged upon translating JAR-FSTD A & H into EASA CS., the FAA are updating Part 60 to the latest technical standards and are also reviewing the implications of adopting Document 9625 criteria. Both Australia and Canada are doing similar exercises but writing new legal rules to be adopted by their various governments, is a time consuming task.
Recommendation
In the following, we in TTI., are suggesting how the new criteria might be adopted by amending current Rules using the route of an ACCEPTABLE MEANS OF COMPLIANCE (AMC). The recommendations are simply one way in which the task may be approached but we believe it will provide intending NAAs with some of the considerations of which to take account
First it should be clearly understood that simply taking an existing set of regulations and adding to them a statement that Document 9625 can be used as an AMC., will not work. We cannot stress this too strongly. To simply try to add an AMC., of this type, omits defining which part of Document 9625 is to be applied as an alternate means of compliance ? The whole of Document 9625 will only address some areas of the existing Rules and miss out major portions of what is required in new legislation. In some areas there will be direct conflict between the provisions of the new Document and those of existing ones. Finally the concept of Document 9625 is firmly based upon a training Needs Analysis, whereas existing regulations were framed purely to take advantage of the most up to date technology available at the time they were introduced. No consideration had been given to what the instructor really required to accomplish the training for which the FSTD was to be used. Additionally, in Document 9625 issue 3., a new type of qualification is proposed whereby a device may be defined, using Part III., by matching simulator features to the training task. This is sometimes called the “a la carte” method because the resulting device, whilst capable of being qualified, may not match any of the defined Types listed in Part II. There is no parallel process in any existing Rule so an additional component will always need to be added to them if a match to Document 9625 issue 3 is intended.
A Way Forward
As a starting point, we suggest that reference be made to one of the current major Regulations such as JAR-FSTD A & H and to amend it as appropriate. It wold be equally possible to adopt the FAA’s Part 60 or any other existing set and the general principles to be followed would be the same. Only the details would vary according to the NAA concerned.
In our example we shall address JAR-FSTD A.
| JAR-FSTD A.001 Applicability: |
No change necessary |
| JAR-FSTD A.005 Terminology: |
A review of the lists of terminology, definitions and abbreviations included in Document 9625 is necessary and any differences should be incorporated into this JAR. Most existing definitions will not require any change but new ones may need to be added. |
| JAR-FSTD A.010 Date of Introduction: |
No longer applicable for JAR-FSTD A & H but might be appropriate for Rules written for other NAA. |
| JAR-FSTD A.015 Application for Qualification: |
No change necessary except to add the new TYPEs and also to cater for FSTD covered in Part III |
| JAR-FSTD A.020 Validity of Qualification: |
Can be amended to reflect the specific requirements of the NAA but no change is necessary because of Document 9625. |
| JAR-FSTD A.025 Quality Systems: |
No change necessary |
| JAR-FSTD A.030 Requirements: |
Reference to Document 9625 Issue 3 must be added to include its provisions as an Alternate Means of Compliance. |
| JAR-FSTD A.031 to 034 |
These JAR only have relevance to JAR-FSTD qualified FSTD so are unlikely to need any change as a result of the addressing of Document 9625 iss 3. |
| JAR-FSTD A.035. Grandfather Rights: |
Are awarded at the discretion of the NAA and are not specifically addressed in Doc 9625 issue 3 because of this. The provision of Grandfather Rights in current Rules is necessary to protect the position of FSTD qualified under any previous Rule. It might, therefore, be equally applicable to FSTD qualified in accordance with the criteria in Document 9625 Issue 3. However, in this Doc there is, additionally, the possibility of using its Part III to match the technical qualities of any older device to the Fidelity Levels of Document 9625 Issue 3., and from these define whether the old device is capable of being qualified as one of the document’s new types. Once such a compatibility is established, the QTG testing necessary and the training for which the device may be used can also be extracted from the matrix. If the Part III path is chosen then the old device will simply be re-qualified under one of the new Types or as a “one off” and grandfather rights will not be necessary. |
JAR-FSTD A.036 FSTD's qualified before 1 July 2000
|
No changes necessary. |
JAR-FSTD A.037 FNPT's qualified before 1 July 1999
|
No longer applicable. |
| JAR-FSTD A.040 Changes to Qualified FSTD: |
Reference will need to be made to identify the possibility of Upgrading or Updating the FSTD in compliance with Document 9625 Issue 3. and, thereby, changing from a Qualification Level to a Qualification Type. |
| JAR-FSTD A.045 Interim Qualification. |
No changes necessary. |
| JAR-FSTD A.050 Transferability: |
No changes necessary although other NAA may have different rules covering this aspect. |
..
From all of the above, it may be seen that as far as the Rules themselves are concerned i.e. Part 1 of the JAR-FSTD., few changes are necessary to include the possibility of compliance with the criteria laid down in Document 9625 issue.3. The real kernel of the required changes falls into the Appendices and the Advisory Circulars Joint (ACJ) in Part 2. Whilst these are not necessarily legally binding, if they are considered as the only specified acceptable means of compliance, then it is almost the same as being a legal requirement. Further, without the amplifications included in these appendices and ACJ it will prove extremely difficult to comply with the legal requirements of the Rule.
| Appendix 1 to JAR-FSTD A.030 |
This addresses the FSTD Standards for those devices falling into the categories of Level A / B / C & D FFS and the various levels of FTD and FNPT. Different standards will apply to FSTD categorized under the Type 1 through 7 provisions of Document 9625 Issue 3 and these will either be necessary to be added to the table or another table generated representing the seven Types. |
| JAR-FSTD A.005 Terminology: |
A review of the lists of terminology, definitions and abbreviations included in Document 9625 is necessary and any differences should be incorporated into this JAR. Most existing definitions will not require any change but new ones may need to be added. |
| ACJ to JAR-FSTD 005 |
Additional terminology & abbreviations can be added from the Document 9625. As in the case of JAR-STD A.005 few differences will be found. |
| ACJ No 1 to JAR-FSTD A.015. Letter of Application: |
Most of the layout of the letter is equally applicable to a device to Document 9625 Issue 3. Changes will be necessary in the table of type of FSTD to reflect the new seven Types. It might be considered preferable to add a second table for this because the descriptions now in the existing table do not readily fit with the new types. |
| ACJ No 2 to JAR-FSTD A.015. FSTD Evaluations: |
The current content is generally still applicable to the processes of evaluation but to include the provisions of Document 9625 issue 3 it will be necessary to make reference to and include the text of Chapter 2 through 9 of Volume 1., with particular attention being paid to Chapter 3 which describes the new Determination Process. It follows from this that any new regulatory document being produced will have to either include all 88 pages of the Appendix A or refer to and have access to the matrix in a software version. Such a software program is being produced for users of Document 9625 issue 3 but is not yet in a “user friendly” format. In practice, it is unlikely that an evaluator or manufacturer will be easily able to use the matrix without this software “front end”.
In this ACJ it will also be necessary to differentiate between the process of evaluating using Part II of Document 9625 that would lead to a qualification against one of the new seven Types of FSTD or, by reference to Part III determine the qualification to be an “a la carte” one based upon a unique set of criteria meeting a specified training requirement; or in the case of a previously qualified FSTD be re-qualified, not against its original regulatory but the new criteria, so as to determine whether it can be classified as one of the seven defined types or not.
It follows that this ACJ will be one of the most important and will have to include a great deal of information.
This is one of the major reasons why we have said in our opening paragraphs of this briefing that it is not possible to simply “adopt “ Document 9625 issue 3 as a new Rule without a substantial amount of additional content. Additionally, it is not thought likely that ICAO will include a software copy of the matrices with the publication of Document 9625. issue 3 because of matters of cost and availability so any new Rule offered by an NAA will have to address how access to the matrices will be enabled.
|
| ACJ No 1 to JAR-FSTD A.025. Quality Systems |
The only reference to a Quality System that appears in Document 9625 issue 3 is in Part 2 (page II-2-13). This calls for a Quality System to be established that is acceptable to the NAA. No details are supplied so the adoption of the requirements specified in JAR-FSTD A or H would be a quick and effective method of meeting this requirement. |
| ACJ No 3 to JAR-FSTD A.025.: Installations. |
The provisions of this ACJ are equally applicable to a device qualified under the provisions of Document 9625. |
| ACJ No 1 to JAR-FSTD A.030: |
Already starts with a reference to the ICAO Document, as amended, so requires little change apart from adding the new Tables of Validation for the new Types of FSTD included in the issue 3. However, this will be no small task if it is intended to add these Tables in a printed format. The optimal solution will be to include the matrices in digital format and, when they are produced, use the ‘front end” software to access them. |
Appendices 1, 2, 3, 4, 5, 6, 7 & 8 to ACJ No 1 to JAR-FSTD A.030. are also included as requirements in Document 9625 issue 3 but appear as ATTACHMENTS in Part II. The attachments contain very similar information as is included in the Appendices but add further advice to the effect that there are Attachments A to O covering additional areas. Prime amongst these is Attachment F that addresses new types of Motion Testing that is aimed at being more objective than are the tests in JAR-FSTD A & H. The full extent of these new tests is yet to be defined but will, probably, be defined during 2011 and introduced into Document 9625 issue 3 as an amendment from ICAO.
The remaining ACJ have little applicability to any FSTD which is to be qualified against the criteria contained in the Document 9625 issue 3. but some of the material is also included in the Document as advice.
What reference to the new document will quickly disclose is that it contains a great deal more information than is the case for JAR-FSTD A & H or CFR Part 60 or similar NAA Regulatory documents. This is not an accident because the Document 9625 issue 3 is meant to be more descriptive and more exacting as well as introducing the concept that the definition of the device is grounded in its training role rather than it being defined by the current state of available technology as were devices in older Rules. It follows that any new NAA Regulatory Document will also have to be much larger, if it is based upon these new criteria, than the existing ones. Any new Rule cannot, in our view, be issued simply with a reference to Document 9625 as its basis. The provisions of the Document should form an integral part of the new Rule.
This information advice is not intended to be exhaustive nor totally comprehensive. We believe that it is simply an indication of the amount of work that will be necessary should any NAA decide to introduce a new Rule encompassing the criteria of Document 9625 issue 3 and shows some of the major areas to be considered in the re-write. Some of these issues are appended herewith in the form of FAQs and more may be added if they are received from our readers.
TTI will be happy to discuss these matters with any NAA considering taking on the task of writing a new Rule and will offer its expertise to those who might wish to contract out such a task.
Brian Hampson.
January 2011.
ADOPTING ICAO DOCUMENT 9625 ISS. 3 - FREQUENTLY ASKED QUESTIONS
(return to top of page...)
Q1. You mention that 9625 cannot be used as a stand alone document because it does not embody the administrative and testing processes of JAR FSTD A. This then makes me ask - who, or which, entity is responsible for closing this loop?
A1. As indicated, the Doc 9625 is an advisory document issued by ICAO who expect all signatory States to implement the criteria in the document. The responsibility for doing this, including all the administrative actions necessary to implement the recommendations will vary from State to State and the manner in which legislation is formulated by that State. Generally speaking, the drafting of the Regulatory document will fall to the National Aviation Authority who will then pass it to the legislature who will pass it into law.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Q2. Does the adoption of ICAO Document 9625 iss. 3 mean that there will be significant changes in how we evaluate and qualify FSTD's ?
A2. This is highly unlikely. The current process of evaluating a FSTD (how we evaluate a FSTD) works very well, and it is not foreseen to require change when a NAA adopts ICAO 9625 issue 3.
To be more specific, evaluating and qualifying a FSTD to the specified standards in ICAO Document 9625 iss. 3 will remain essentially the same as we use for JAR FSTD - A, CFR Part 60 or other regulations. However, the tests will be against new criteria, in many areas and with revised tolerances in some cases. Obviously the few new requirements identified in ICAO Document 9625 iss. 3, such as ATC simulation, will have their own standards, but how we evaluate and qualify the devices will remain the same.
The major change is in the fact that Doc. 9625 issue 3 determines, from training and checking requirements, the unique evaluation criteria for that collection of unique training or checking tasks. These collections become the "Type" of device. The document then permits a device to be evaluated against a specified Type's criteria, as defined in Part II for a specified Type of device or, by using Part III for a unique Type of device not matching the criteria nor training objectives found in the FSTD Types defined in Part II..
The criteria chosen are those deemed to be the minimum to ensure that the FSTD can meet the training requirement for that particular aspect of the device.
This is the first time that Doc 9625 makes this differentiation. The training requirement, in most cases, recognizes the difference between Training and Training to Proficiency, often permitting the former to be carried out on a different Type.
It must be pointed out that the various Types are not hierarchical, meaning that a Type 6 is not necessarily better nor more complex than a Type 5. It is simply that they are designed to meet the specific training needs. This is a different perspective to the current Levels defined in JAR-STD and CFR Part 60.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Q3. If 9625 is an approved and accepted ICAO document then am I correct in understanding that any ICAO member State is free to use this document as a method to test, qualify and approve training usage of FSTDs (FFS and FTDs in the 9625/3 context) in their State but it may not be recognized internationally (reciprocity)?
A3. Not only is every State free to use the provisions of the Doc 9625 but ICAO expects that they will do so. However, it could be that in formulating the Regulation (Rule) in any particular State some administrative procedure might be formulated in a manner that is not acceptable to all other NAAs. For example, the particular State may increase the criteria in some manner. Remember that the Doc. 9625 details minimal acceptable criteria. If this were to be the case the additional items might be unacceptable to all other States but it would still be compliant with the Doc 9625. Similarly, unless the Regulation includes the necessary administrative procedures for the criteria’s implementation another NAA may refuse to recognize that State’s Regulations. Reciprocity can only be achieved if individual NAAs agree to the provisions contained in each other’s Regulations.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Q4. Are any FSTDs currently in build or recently approved/qualified under 9625/3 ?
A4. Several FSTD are being contracted to meet the criteria of Doc 9625. Most of these are also required to meet the current regulations in addition. We know of no FSTD that have yet been Qualified against Regulations that are compliant with Doc 9625 because to our knowledge no State has yet produced new Regulations that would meet the criteria and also be acceptable to other NAAs, internationally.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Q5. When I look at Doc. 9625 and at the latest version of the Volume I RAeS Handbook,(Issue 4) it gives the impression that QTG could be produced to comply with Part II, Appendix B. even though some test parameters and requirements differ slightly (i.e 100 vs. 150 ms response time from JAR FSTD A). I believe this is what is being referred to when it is said a manufacturer will produce a QTG., and if the QTG meets all the requirements of Part II Appendix B why should an NAA not grant Qualification and Approval?
A5. The Volume I of the RAeS Handbook (Issue 4) was created to reflect the provisions of Doc. 9625., as they were known at the time of its publication, it is certainly possible for a FSTD to be built to meet the criteria of the most demanding parts of both Doc. 9625 and any other Regulations such as JAR-FSTD A or CFR Part 60. In this manner the FSTD would meet or exceed the minimum requirements of both. Unless, however, the State has published a new set of Regulations it cannot be said to be Qualified against Doc 9625 because ithe State’s current Regulation will have no mention of the new Types of device nor will such items as the Application Letter include them. In addition there will need to be additional items included as AMCs (or something similar) – see Q5 - where the existing Regulation’s’ might be in conflict with Doc 9625. Partial compliance creates bad law and might prevent reciprocity with other States.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Q6. If Doc. 9625 is an approved ICAO document and expected to be used by a Member State, as it states that a Quality System has to be implemented, then the assumption must be the NAA has to set up a program to assure continuing fidelity of the devices under it's jurisdiction. Then it must be mandated by that NAA.?
A6. Doc 9625 does not describe what Quality System needs to be implemented although it does make a suggestion as to where to go to find such information and as mentioned in A4 above the Regulation must be written to explain the provisions of the Quality Plan. In some existing State’s Regulations the Quality System described therein might also be acceptable against the Doc 9625 provisions. Other States have not included that provision in their existing regulations and this will require some additional information, probably in the form of explanatory material.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Q7. In the "Adopting 9625" draft you mention some items, such as motion testing, that are yet to be included in 9625/3. Is completion of these items a task of the IWG and when it is completed will the information be treated as a simple revision to 9625 or will it involve a more formal and perhaps lengthy approval process?
A7. There are several items in Doc 9625 that are considered desireable for the future FSTD to be Qualified under the new criteria. Motion objective testing is one such and Air Traffic Control another. Current Regulations define motion tests that are almost entirely subjective in so far as motion performance is concerned. It is frequently left to subjective “tuning” to establish an acceptable level of performance. For high level devices this is, obviously unsatisfactory so the International Working Group mandated that some research be carried out to see if a measure of objectivity could be introduced. It was decided that Frequency Domain tests and Time Domain tests might prove a practical solution to this requirement. (Attachment F) At the time that Vol I. of Doc 9625 was published the full results of this research had not been reached so provision has been made to insert these tests at a later date (probably coincident with the publication of Vol II). The method of doing so is described in Doc 9625 as Appendix D. The practical method of meeting these changes will be by a simple page amendment issued by ICAO the amendment being recommended by an International Committee (ICFQ) under the auspices of the RAeS.
Motion testing is yet to be defined but will probably be defined during 2011. Air Traffic Control provisions will follow when suitable systems have been developed and found to be practical and acceptable.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Q8. Will 9625/3 supersede the original Technical Standards produced by the first International Working Groups that werre published by ICAO and used by JAA and FAA to produce Part 60 and JAR FSTD?
A8. Yes, the new Document replaces the earlier ones (Issue 1 and Issue 2) but, of course, Regulations compliant with these earlier criteria will still exist and will be applicable to older FSTD qualified under the earlier Regulations (i.e. Grandfather Rights). However, Document 9625 does not address the issue of Grandfather Rights because those are entirely the remit of the NAA. So it is conceivable that some NAAs may not accept the principal of Grandfather Rights or may introduce limitations on them. Again, this is not something that ICAO will decide.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Q9. Today we know that the major NAAs will not be adopting the content of 9625 directly or indirectly for some time. That may or may not change but in the meantime it could put the early adopters of Doc 9625 issue 3 at a disadvantage by using terminology and criteria that have no apparent linkage with today's standards?
A9. The timeline for the introduction of Regulations genuinely compliant with Doc 9625 is unknown. The legislative process as well as the writing of such Regulations is a time consuming process. However, not all of Document 9625 FSTD are completely different to existing devices. We can stress a clear equivalence between Type VII and Level D. It may be possible to extend this to Type V and the EASA CS FSTD FTD Level 2 and FAA CFR Part 60 FTD Level 6. In doing so, the early adopters could, if they wished, request a dual Qualification by using both the new and the old in parallel if their NAAs will permit, as would seem reasonable.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Q10. Are there any other implications of adopting Qualification Regulations based upon Doc 9625 issue 3 criteria ?
A10. There may be many, depending upon the current NAA Regulations but one area that needs to be reviewed are the Licensing and Approval Regulations of each NAA. As Doc 9625 issue 3 relates the training events for each Type of device and, indeed for the technical attributes of the FSTD Features, there will be some cases where the Approvals might change between a FSTD qualified under a new Rule and those qualified under existing Rules. This may have a “knock on” effect to the Licensing provisions. Of course, as Doc 9625 issue 3 reflects, for the first time, the MPL ( Multi-crew Pilot Licence ) training which is not included in the Regulations covering existing devices, then these must also be reflected in the Approvals / Licence Rules. This will be especially important if the suggestions in A8 regarding dual Qualification are adopted.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Q11. As compared to the previous issue of 9625, issue 3 is very large and somewhat difficult to navigate when you don't use it every day. Is there any intention of producing a "smart" electronic document or software tool to aid in finding topics and it's use in general ?
A11. Document 9625 includes a set of complex matrices defining the criteria to be used for the Qualification, testing and training approval for all FSTD. These are published in hard copy format but are also available as a download from the RAeS FSG website. Even so they are not thought to be “user friendly” for manufacturers, operator or regulators. Accordingly, some effort is being made to provide “front end” software that will ease the task of using Document 9625. It is in production and, hopefully, will be available in 2011. Again the software will be made available after approval by the ICFQ.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(return to top of page...) |
