Internal Auditing of Laboratory Quality Management Systems [PDF]

Zitiervorschau

PREAMBLE This document provides information about the processes involved in Conducting Internal Quality System Audits for Laboratories. A wide range of references were consulted in preparing this guide, therefore it presents a global approach to the conduct of laboratory audits against the ISO/IEC 17025 Standard. Laboratories are however advised that they need not restrict themselves or their staff only to what is included in this guide. This guidance manual is not to be used for commercial purposes. The development of this manual on “Internal Quality System Audits for Laboratories” was supported by the European Union through the 9th EDF Funded Caribbean Laboratory Accreditation Services Project (CLAS), implemented by the CARICOM Regional Organisation for Standards and Quality (CROSQ).

For further information, please contact: CARICOM Regional Organisation for Standards and Quality (CROSQ) 2nd Floor Nicholas House 29-30 Broad Street Bridgetown Barbados, W.I. www.crosq.org [email protected]

Authors

Giselle Guevara, BSc., MPhil, MBA Project Coordinator EDF Funded Caribbean Laboratory Accreditation Services Project CROSQ Barbados Ing. Manfred Kindler Accreditation Expert and Consultant Kindler International Division Berlin, Germany [email protected]

Page 2 of 64

Acknowledgements The Authors are most grateful to the several persons who provided support, very useful input, feedback and gave generously of their time in the development and production of this document. We especially wish to thank those colleagues who reviewed the Manual and paused in their busy schedule to provide incisive comments, helpful tips and stimulating input. Thank you for everyone who provided examples and ideas for the content of this manual, sometimes without even knowing you were helping. We are truly thankful and wish to recognise the contributions of Valerie Wilson and Wendy Kitson-Piggott as Technical Editors, Michael Seepersaud, CROSQ-IDB Project Manager for assisting with the manual publication. Thanks to Manfred Kindler who provided useful technical information, invaluable feedback and assistance, in the production and compilation of this material. We wish to thank CROSQ, the CARICOM Secretariat and the European Union funded CISP Project for their contribution to the production of this guide. Preparation and Editing of this manual was a long and precise task we therefore wish to extend our special gratitude to all who assisted and facilitated the process.

Produced and Printed with support from the European Union under the 9th EDF Fund

Page 4 of 64

INTERNAL AUDITING OF LABORATORY QUALITY MANAGEMENT SYSTEMS Table of Contents CHAPTER 1: QUALITY MANAGEMENT SYSTEMS FUNDAMENTALS .................................. 7 1.1 INTRODUCTION ............................................................................................................. 7 1.2 QUALITY TERMS AND DEFINITIONS............................................................................ 8 1.3 ACCREDITATION vs. CERTIFICATION ........................................................................10 CHAPTER 2: INTRODUCTION TO INTERNAL QMS AUDITING PROCESS ..........................12 2.1 INITIATING THE PROCESS...........................................................................................12 2.2 USING THE PDCA MODEL IN THE AUDIT PROCESS.................................................13 2.3 AUDITING ROLES AND RESPONSIBILITIES ...............................................................15 2.4 HORIZONTAL AND VERTICAL AUDITS .......................................................................15 CHAPTER 3: PLANNING FOR INTERNAL AUDITS .................................................................21 3.1 PREPARING THE AUDIT PLAN ....................................................................................21 3.2 DEVELOPING THE AUDIT SCHEDULE ........................................................................22 3.3 AUDITOR PREPARATION .............................................................................................23 3.4 AUDITOR COMPETENCIES ..........................................................................................26 CHAPTER 4: CONDUCTING THE INTERNAL AUDIT ..............................................................27 4.1 INITIATING THE AUDIT PROCESS ...............................................................................27 4.2 CONDUCTING THE AUDIT ............................................................................................28 CHAPTER 5: MANAGING AUDIT OUTCOMES ......................................................................30 5.1 NON-CONFORMITIES....................................................................................................30 5.2 RECORDING and REPORTING AUDIT FINDINGS .......................................................32 5.3 ROOT CAUSE INVESTIGATION ...................................................................................35 5.4 TAKING CORRECTIVE ACTION ...................................................................................36 5.5 MANAGEMENT REVIEW ...............................................................................................37 CHAPTER 6: AUDIT TOOLS ....................................................................................................38 6.1 AUDIT CHECKLIST ........................................................................................................38 6.2 AUDITOR PREPARATION .............................................................................................41 6.3 PROCEDURE FOR CONDUCTING INTERNAL AUDITS ..............................................43 6.4 CORRECTIVE ACTION RECORDING FORMAT ...........................................................46 CHAPTER 7: FAQs on LABORATORY QUALITY .............................................................47 APPENDICES ............................................................................................................................52

Page 5 of 64

LIST OF FIGURES Figure 1 Summary of the Accreditation Application process ...................................................11 Figure 2 Overview of the Internal Audit Process ........................................................................13 Figure 3 Document Control Checklist .........................................................................................16 Figure 4 Example of a Vertical Audit checklist ...........................................................................18 Figure 5 Example of an Audit Schedule ......................................................................................22 Figure 6 Example Twenty Milestones Audit Checklist ...............................................................23 Figure 7 Competencies required for effective auditing .............................................................26 Figure 8 Example of Audit Elements ...........................................................................................28 Figure 9 Example of a Non Conformity report form ...................................................................30 Figure 10 Summary Audit report form.........................................................................................34 Figure 11 Example of a Root Cause Analysis Report ................................................................35 Figure 12 Example Audit checklist ..............................................................................................38 Figure 13 Example of a Corrective Action Form ........................................................................46

LIST OF APPENDICES APPENDIX 1: SUMMARY OF CLAUSES IN ISO/IEC 17025 .........................................................52 APPENDIX 2: LIST OF POLICIES IN ISO/IEC 17025 ....................................................................54 APPENDIX 3: LIST OF PROCEDURES IN ISO/IEC 17025 ...........................................................55 APPENDIX 4: LIST OF “SHALLS” IN ISO/IEC 17025 ..................................................................57 APPENDIX 5: SAMPLE ACCREDITATION ACTION PLAN .........................................................62 APPENDIX 6: EXAMPLE OF A VERTICAL ASSESSMENT FORM (ISO/IEC 17025) ...................63 APPENDIX 7: EXAMPLE OF A FORM FOR THE WITNESSING OF TESTING ...........................63 APPENDIX 8 REFERENCE DOCUMENTS ....................................................................................64

Page 6 of 64

CHAPTER 1: QUALITY MANAGEMENT SYSTEMS FUNDAMENTALS 1.1

INTRODUCTION

The International Standards „ISO/IEC 17025 - General Requirements for the Competence of Testing and Calibration Laboratories’ and „ISO 15189 – Medical Laboratories – Particular Requirements for Quality and Competence’ both respectively state that a laboratory shall establish, implement and maintain a quality system appropriate to the scope of its activities including the type, range and volume of testing and/or calibration activities it undertakes. Laboratory Accreditation is the formal recognition of a laboratory‟s technical competency to perform specific tests, types of tests or calibrations. ISO/IEC 17025 and ISO 15189 1 accreditation requirements are recognised internationally as defining and incorporating essential elements of a quality management system for laboratories (ref: Appendix 1). The requirements for each stage of the quality improvement process are described in the various clauses of the ISO/IEC 17025 and ISO 151892. Achieving compliance with the ISO/IEC 17025 and/or ISO 15189 Standard can be achieved in a stepwise manner that ensures efficient use of resources. Essential components of these standards include the following: 

A laboratory that operates within a Quality Management System (QMS) will have a single QUALITY MANUAL hereafter referred to as „the Manual‟, that contains the documented policies for the laboratory and either includes or refers to the processes, procedures and forms used by laboratory staff to implement the QMS throughout all the disciplines in the laboratory



A laboratory shall periodically and in accordance with a predetermined schedule and procedure, conduct internal audits of its activities to verify that its operations continue to comply with the requirements of the quality system and the Standard (ISO 17025/15189 clause 4.14)

Laboratory standards mandate that the laboratory‟s internal audit programme must address ALL aspects of the quality management system and ALL requirements of the agreed Standard. The Standard also mandates that audits be coordinated by an authorised and competent person specifically tasked with the responsibility for overseeing the laboratory‟s quality improvement initiative. Evaluations of regional laboratories have identified the need for the strengthening of auditing skills as the thrust towards accreditation becomes more widespread. The complexity of the auditing process depends on the size, scope and organizational structure of the laboratory. Audits can be scheduled and conducted in a manner that involves and motivates staff. Many of the auditing strategies outlined in this publication can be implemented in a simplified 1

This document references the ISO/IEC 17025:2005 –General Requirements for the competence of Testing and Calibration Laboratories 2 Please note that wherever “Clause” is mentioned in this document, it refers to a Clause of the ISO/IEC 17025 or ISO 15189 Standard

Page 7 of 64

manner. This Manual is intended to serve as a guide and training manual for laboratory staff and managers. Note Periodic audits facilitate early identification of quality gaps and allow timely corrective interventions to be taken that reduce or prevent the occurrence of laboratory errors.

1.2 QUALITY TERMS AND DEFINITIONS In order to master the process of auditing a laboratory‟s Quality Management System (QMS) some key quality concepts, terms and definitions must be understood. It is important that laboratory staff and management understand how important audits are to both the QMS development process and the accreditation process. Key quality terms, concepts and definitions include the following: a. Quality Management System

A program developed to support efficient, effective, high quality and appropriate laboratory services (e.g. accurate and precise results, appropriate test selection, timely reporting, and correct interpretation of results, clinical usefulness, and recommendations for further investigations). The QMS facilitates a comprehensive and coordinated laboratory effort (policies, processes and procedures), designed to meet quality objectives and to direct and control an organization with regard to quality. The QMS is comprised of the Quality Manual (Quality Policy, General policies, etc), Administrative Procedures, Technical Procedures and Technical and Quality Records.

b. Policy

A brief written statement describing the laboratory‟s intended action with respect to the specific requirement of the Standard.

c. Process

A series of inter-related steps involved in an activity or examination that uses resources to transform inputs into outputs. Processes may often be linked since the output of one process is often the input to the next. Rarely is there a process that is not linked to some other.

d. Procedure

Written work instructions that specify the way to carry out a step in the process, an activity or an examination. Procedures describe in detail exactly how one individual should perform an activity.

e. Record

Any information that provides evidence (e.g. requisitions, examination results and reports, instrument printouts, laboratory workbooks and worksheets, accession records, calibration records, quality control records, records of audits, complaints and action taken, external quality assessment records, instrument maintenance records, incident/accident reports, staff training and competency records, personnel records).

f.

Document

Page 8 of 64

Any documented information that provides direction (e.g. instructions including policy statements, textbooks, reference intervals and their origins, procedures, specifications, calibration tables, charts, posters, notices, memoranda, plans, software, drawings, regulations and standards).

g. Document Control

A system to regulate the handling and management of documents (including archiving, storing and destruction), containing information that communicates policies, processes and procedures and records. Usually refers to documents that are part of the quality management system.

h. Audit

A systematic and independent examination to determine whether quality activities and related results comply with an agreed standard. An audit can also determine whether the standard being used is appropriate for achieving the organisation‟s objectives and is being implemented effectively Note: In this document the term “Internal Audit” is used to emphasize that the audit is done by the organization itself

i.

Audit scope

j.

Audit plan

The extent and boundaries of an audit that are usually established before the audit process begins A description of the activities and arrangements for the conduct of an audit

k. Management review

A formal evaluation by top management of the status and adequacy of the quality system in relation to the organisation‟s quality policy and objectives

l.

Quality management

That aspect of the overall management function that determines and implements the quality policy

m. Quality manager

The staff member (by whatever title) who has responsibility for the laboratory‟s quality system and its implementation and who, in this capacity, reports directly to top management

n. Quality auditor

Person qualified to perform quality audits

o. Observation

A statement of fact made during an audit, substantiated by objective evidence

p. Objective evidence

Qualitative or quantitative information, records or statements of actions related to the quality of a quality system activity, element or service which is based on observation, measurement or test and which can be verified

q. Audit findings

Documented audit evidence generated by evaluation of the QMS against specified audit criteria

r. Nonconformity

Page 9 of 64

The non-fulfillment of specified requirements of the standard

s. Corrective Action

An action taken to eliminate the causes of an existing nonconformity, defect or other undesirable situation in order to prevent a recurrence

t.

Preventive Action

An action taken to eliminate the cause of a potentially undesirable result

1.3

ACCREDITATION vs. CERTIFICATION

Laboratory accreditation requires the application of assessment criteria specifically developed to determine technical competence. These criteria are based on the international standards ISO/IEC 17025 and ISO 15189, for Testing/Calibration and Medical Laboratories respectively. Accreditation bodies, through specialist technical assessors, aim to ensure that your laboratory‟s test data are accurate and reliable and use these standards to conduct a thorough evaluation of all factors relevant to the laboratory‟s competence. The ISO 9001 standard is widely used in manufacturing and service organizations to evaluate and certify their management systems for controlling the quality of their products or services. The ISO 9001, however, does NOT assess the technical competence of an organisation‟s operations and thus cannot be used for accrediting laboratories. Note While laboratories may be certified to ISO 9001, such certification does not make any statement about the technical competence of a laboratory as this standard does not contain technical requirements for laboratory personnel and operations3.

Why Is a Laboratory’s Technical Competence Critical? Because it is important for laboratories to: Minimise Risk In the world today, customers seek reassurance that the products, materials or services they produce or purchase meet their expectations or conform to specific requirements. This often means the product is sent to a laboratory to determine its characteristics against a standard or a specification. For the manufacturer or supplier, choosing a competent laboratory minimizes the risk of producing or supplying a faulty product. Avoid Expensive Retesting Testing of products and materials can be expensive and time consuming even when they are done correctly the first time. If not done correctly, then the cost and time involved can be even higher if the product has failed to meet specifications or expectations and re-testing is required. This can negatively impact the reputation of a supplier or manufacturer. Additionally, the supplier can also 3

See ISO CASCO Communiqué on Accreditation vs. Certification and ILAC Publication on Laboratory Accreditation or ISO 9001 Certification (www.ilac.org.tt)

Page 10 of 64

be held liable for any failure of the product, particularly if it involves public safety or financial loss to a client. Choosing a competent laboratory thus minimises the chance of retesting being required, increase in operational costs and or exposure to legal action. How does a Laboratory Apply for Accreditation? Figure 1 summarises the steps in the accreditation process from the pre-audit visit to the final award of the accreditation certificate. Figure 1 Summary of the Accreditation Application process

Accreditation Body’s Pre-Audit

Pre-Assessment visit: initial external audit by experts from Accreditation Body Corrective actions completed

Application to Accreditation Body Complete application process for accreditation to ISO/IEC 17025 or ISO 15189

Accreditation Body Document Review On-site Visit

Review of documents forwarded by laboratory previously On site assessment by assessor team follows

Laboratory Corrective Actions

Cause Analysis – identify the source of non conformances and confirm corrective actions

Accreditation

Accreditation Body -vote on accreditation decision Laboratory - receipt of accreditation certificate

Page 11 of 64

CHAPTER 2: INTRODUCTION TO INTERNAL QMS AUDITING PROCESS 2.1

INITIATING THE PROCESS

On behalf of laboratory management, the Quality Manager is responsible for ensuring that all areas of the laboratory‟s activities and responsibilities covered by ISO/IEC 17025 or ISO 15189 and the Quality Manual are audited periodically. Audits must be planned so that all aspects of the laboratory operational procedures, including the performance of calibrations and tests, are covered within a specified period of time. Where a laboratory is accredited for calibration, sampling or testing at satellite sites located away from its permanent premises, these activities must be included in the audit program. The Quality Manager may delegate the task of performing audits provided that the person used: 1) has technical or scientific training 2) is familiar with the laboratory‟s quality system and accreditation requirements 3) has been trained in the techniques of auditing. 4) is, where possible, independent of the activity (i.e. auditors cannot audit their own activities or activities for which they have direct responsibility) Audits carried out by parties such as customers or accreditation body assessors cannot be considered to be a substitute for, or to override the laboratory‟s responsibility for conducting its own internal audit. Internal Audits must be planned to establish that: 1 management objectives, as defined in the quality system, are met in all respects 2 all personnel, at all levels, are satisfactorily carrying out their assigned duties and responsibilities 3 The procedures detailed in the quality system are being followed Audits should be designed to reveal the extent of compliance with requirements in all of the elements examined and should serve to check on the competence of personnel at all levels Note Where a laboratory has accreditation for calibration, sampling or testing on sites away from its permanent premises, these activities must be included in the audit program.

Page 12 of 64

2.2

USING THE PDCA MODEL IN THE AUDIT PROCESS

A simple approach to developing an audit system for the laboratory is to use the Plan-Do-CheckAct (PDCA) model to elucidate the steps which must be taken to ensure a thorough process as in Figure 2 below. Figure 2 Overview of the Internal Audit Process

ESTABLISH THE AUDIT PROGRAMME Objectives and extent Responsibilities Resources Procedures

PLAN INITIATE THE AUDIT

Appoint the audit team leader Define the audit objectives, scope and criteria Determine the feasibility of the audit Establish initial contact with the auditee

CONDUCT DOCUMENT REVIEW

Review relevant management system documents and include records Determine document adequacy with respect to audit criteria

PREPARE FOR ON- SITE ACTIVITIES

Prepare the audit plan Assign work to the audit team Prepare work documents

Page 13 of 64

DO

CONDUCT ON SITE ACTIVITIES

Conduct opening meeting Ensure effective communication during the audit Define roles and responsibilities Collect and verify information Generate audit findings Prepare audit conclusions Conduct a closing meeting

CHECK

PREPARE, APPROVE AND DISTRIBUTE AUDIT REPORT Prepare the audit report Approve and distribute the audit report

COMPLETE THE AUDIT

Identify needs for corrective and preventive actions and implement Identify opportunities for improvement and implement

ACT AUDIT FOLLOW UP Monitor and review

Page 14 of 64

2.3 AUDITING ROLES AND RESPONSIBILITIES ROLE OF THE QUALITY MANAGER IN AUDITING At a minimum, the Quality Manager is responsible for the following: a. To identify and select laboratory staff who have demonstrated the potential to be effective auditors to be trained as auditors b. To identify facilitators to conduct audit training and to provide appropriate briefing for facilitators c. To evaluate training outcomes to ensure that staff are competent and to update training records d. To plan, record and organize audits and ensure that deficiencies are corrected rapidly and effectively. The Quality Manager should have sufficient authority in the organization to command respect and secure any changes or actions that are necessary to maintain a satisfactory quality system. The Quality Manager must have direct access to the highest levels of management at which final decisions are taken on matters relating to quality e.g., Head of the Laboratory or Managing Director, e. Where, in exceptional circumstances, the laboratory appoints an external audit officer, the Quality Manager is responsible for ensuring that the person selected is trained in the techniques of auditing and is fully familiar with accreditation requirements, the laboratory‟s quality manual, and all relevant procedures. In a small laboratory, audits are usually carried out by the Quality Manager. In large laboratories with a very wide range of testing activities, the Quality Manager may need to appoint several audit officers or deputies to cover specific areas or activities. Staff should not audit their own activities and should be independent of the specific activities being audited and thus should be able to contribute fresh perspectives. Staff required to carry out auditing must be trained in the techniques of auditing and for these activities must report to the Quality Manager who must monitor and control their activities. One strategy for engaging staff in the quality improvement process may be to involve them in the auditing of activities conducted in departments other than their own. ADDITIONAL KEY ROLES AND RESPONSIBILITIES There are several additional responsibilities that must be undertaken or spearheaded by other staff in collaboration with the Quality Manager. These include (ref: Section 6.2):  the training facilitators who develop training material and programmes and conduct relevant training for staff  the human resource officer who schedules, plans and organizes the training events

2.4

HORIZONTAL AND VERTICAL AUDITS

There are two major auditing strategies generally used, both of which should be included in your auditing programme and interchanged as necessary to cover all elements or requirements in the QMS. These auditing strategies are referred to as the „horizontal‟ and the „vertical‟ audit respectively.

Page 15 of 64

Horizontal Audits Horizontal audits involve detailed checking of specifically selected items. For example, all items of equipment that are required to be calibrated would be checked to ensure that: a) they have been calibrated at the appropriate time and to the appropriate level of uncertainty b) they were labelled or otherwise coded to indicate their calibration status c) the calibration certificates are up-to-date d) records of calibration are up-to-date and all documentation is readily available at the designated location. Disadvantage: When the horizontal audit is only carried out over a specific section of the laboratory, it may only reveal some of the weaknesses that may exist in the quality system. Laboratories should therefore carry out horizontal audits across the entire laboratory as often as is feasible. Supplementing an audit programme based on the horizontal auditing approach with vertical audits will strengthen the programme. Figure 3 illustrates a typical example of a checklist which can be used for a horizontal audit of the document control requirements. Figure 3 Document Control Checklist

Instructions - If the laboratory has implemented the action, process or procedure referred to, the auditor will tick YES and state in the Proof/Comments column what was seen to verify or prove implementation. If not, then tick NO and include any relevant and useful comments 1. Is there an up to date SOP for document control? 2. Is the document current and within review date? 3. Is there evidence that the document has been prepared, checked and approved by a suitably qualified member of staff? 4. Does the document have the necessary information such as version number, unique identification, title, review dates, etc.? 5. Is there evidence of cross referencing to other documents? 6. Is there evidence that documents are accessible and secure? 7. Is there evidence of appropriate document distribution? 8. Is there evidence that this is working? 9. Is there evidence of a process for document change? 10. Is there evidence of a backup procedure for the document control system? 11. Is there evidence that previous versions of documents have been archived? 12. Are previous versions of documents available for retrieval? 13. Is there evidence that staff adhere to the document control procedure? 14. Is there evidence that out-of-date documents are being withdrawn from circulation?

Page 16 of 64

Yes

No

Proof/Comments

15. Are out of date documents reviewed? 16. Are documents appropriately formatted etc? 17. Is there evidence of regular document review? 18. When was the last document review? 19. Is staff trained in document control? 20. Do they have documented evidence of the training? 21. Is there adequate back-up of documents? 22. Is staff informed when documents are updated and available?

Vertical Audits In a vertical audit, a number of calibration or test items or samples are selected at random. Each operation associated with these items or samples is checked including: a) b) c) d) e) f) g)

the competency and actual practices of relevant reception, calibration and/or testing staff the adequacy of maintenance records for equipment used the appropriateness of the calibration or test methods and procedures used the appropriateness of the quality control methodologies in use environmental conditions during calibration or testing the reliability of results recording and reporting, the adequacy of storage and disposal practices

If vertical audits are conducted more frequently than horizontal audits, it is important to vary the point of entry into the system, e.g., by selecting items on different occasions through a random selection of reports or certificates, item numbers, technician records or retained test/calibration items. Disadvantage: Vertical audits intervene at a specific „point in time‟ and do not check a specific element across the entire laboratory operation. For example, a vertical audit will not include a check on ALL calibrations or tests being performed in the laboratory but would be confined to only the calibrations or tests associated with the particular report, certificate, item or record randomly selected. Vertical audits should therefore be viewed as complementary to horizontal audits and should not replace the full horizontal audit exercise. Where it is not feasible to carry out a complete audit program covering all aspects of the quality system in one session, laboratories must make sure that all areas, including the witnessing (observation) of the performance of tests, are monitored over a 12 month period. A plan must be prepared to ensure that horizontal and vertical audits are carried out at the appropriate frequency. This plan can detail aspects of the laboratory operation to be addressed each month so that complete coverage can be achieved over a period of 12 months. It may be necessary for laboratories to carry out additional unscheduled audits whenever there is reason to doubt the effectiveness of the quality system. For example, activities must be audited promptly when a laboratory has received a complaint about its calibration/testing Page 17 of 64

activities that raises some doubt about the laboratory‟s compliance with its stated policies, procedures or standards. Management should ensure that someone is given the task of auditing the Quality Manager‟s activities to ensure that the quality function is carried out satisfactorily. Figure 4 Example of a Vertical Audit checklist Lab Name Report number

Audit date

Department Title Name of auditor

Date and signature of auditor

Instructions 1. 2.

Select laboratory sample number(s) either from the computer record or from work sheets or logbook etc. Audit from receipt of sample to issuing of report.

Audit area Chain of Custody form Is the chain of custody form available for the sample?

Records/methods checked and procedures witnessed

Is there sufficient information to allow unique identification of:  the client  the sample Are details relating to the client and sample available:  date and time of collection  type of sample  tests requested  date and time of receipt  environmental conditions  client information Has transcription of the tests requested been done accurately? Check: - logbook or computer entry with original chain of custody form. Sample Reception Are there procedures for sample receipt and logging in? Are there procedures for storage and handling of samples? Are there criteria for rejection of samples?

Page 18 of 64

Observations

Non Conformity (Yes/No)

Audit area Are there procedures for ensuring staff safety? If the sample or sub sample was referred to another laboratory, are records in place of tests sent away?

Records/methods checked and procedures witnessed

Test methods/procedures Are there documented standard operating procedures readily available and are they detailed enough to allow a consistency of application? List those checked Is there evidence of document control procedures in place? State last revision date. Is there evidence of review of procedures? State date of last review. Were procedures followed in accordance with documented method? List technicians checked. Is there evidence of raw data and calculations performed? Have calculations been checked for accuracy? Quality Control Are there documented standard operating procedures for conducting the QC process? Are records available for each stage of the process? Are they detailed enough to allow determination of the result, batch number, status of QC material? Are QC results monitored for accuracy and precision of testing results? Are records available for each stage of the procedure such that there is a complete audit trail of staff, equipment, reagents and quality control procedures involved? List all records examined

Page 19 of 64

Observations

Non Conformity (Yes/No)

Audit area Reporting results Is a copy of test report able to be generated? Are there any transcription errors between workbooks and the test report?

Records/methods checked and procedures witnessed

Does the report contain sufficient information as required by ISO 17025  Laboratory name  Unique identifiers  Dates of sample receipt, testing & reporting  Sample description etc.  Reference intervals and interpretative comments as appropriate  Pagination e.g. “Page 1 of 1” included where appropriate Check report Is there evidence of the report being delivered to the client? Does the laboratory have procedures for reporting of results? Have results been authorized by approved staff? Is the turnaround time acceptable? Does the laboratory monitor turnaround time routinely? Can the completed report be traced back to the original sample information?

Page 20 of 64

Observations

Non Conformity (Yes/No)

CHAPTER 3: PLANNING FOR INTERNAL AUDITS 3.1 PREPARING THE AUDIT PLAN An audit plan including the audit scope, the audit criteria, the audit schedule, reference documents (such as the organisation‟s quality manual and audit procedure) and the names of audit team members, should be developed and documented by the Quality Manager. An audit timetable should be developed by each auditor in conjunction with the auditee (person being audited) to ensure the smooth and systematic progress of the audit. It is a requirement of ISO/IEC 17025 that audits be carried out in the context of a pre-determined programme such that each aspect of the quality system is examined at least once per year. Where the laboratory holds a separate accreditation for site calibration or testing, or for sampling, audit programs must exist for those areas. Each auditor should be assigned specific QMS elements or functional departments to audit. These assignments should be made by the person designated as the Lead Auditor in consultation with the full auditing team. It is expected that assigned auditors should have some technical knowledge of the departments they are required to audit. Depending on the size and scope of a laboratory‟s operation and in order to ensure that the laboratory‟s operations are audited effectively, the frequency of audits may vary considerably among laboratories. It is important to observe the requirement that each aspect of the quality system is examined at least once per year. To ensure that the laboratory complies with its auditing requirement, it is essential that: (a) the laboratory has planned for audits in its quality system (b) the laboratory has nominated a person, or persons, to be responsible for conducting audits (c) the procedures for auditing are documented (d) the audits are actually carried out (e) the audits are conducted in accordance with a pre-planned program (f) the results of audits are recorded (g) remedial action is initiated immediately by the laboratory to correct all noncompliances identified in its quality system within a reasonable timescale and (h) the remedial action is effective and completed promptly Prior to the actual audit, a review of documents, manuals, previous audit reports and records should be carried out to check for conformity with the quality management system requirements and any previously recorded quality gaps as well as to develop a checklist of key issues to be audited. Key Documents for the Conduct of Audits Working documents required to facilitate the auditor's investigations and to document and report results may include: a) criteria documents such as ISO/IEC 17025 and any supplementary laboratory manuals and documents Page 21 of 64

b) checklists used for evaluating quality management system elements (normally prepared by the auditor assigned to audit that specific element) c) forms for reporting audit observations, such as a non-conformance form or corrective action request form. These permit the recording of the nature of the nonconformity, the agreed corrective action, and the eventual confirmation that the action has been taken effectively. Examples of checklists and forms are included in Chapter 6 (Audit Tools). Note Audit cchecklists should include all requirements of the standard for specific elements of the laboratory operation. Several generic checklists for the auditing of laboratory QMS systems exist.

3.2 DEVELOPING THE AUDIT SCHEDULE To ensure proper planning for the Audit is conducted and all staff are aware of which sections of the QMS are to be audited at what time, a detailed Audit schedule must be prepared in advance. This audit schedule should outline which sections of the QMS and which departments will be audited whether by vertical or horizontal audits. The audits also assist the trained auditors to properly arrange their workload to accommodate the audits. Figure 5 Example of an Audit Schedule Clause Document control Review of contracts External services and supplies Subcontracting Resolution of complaints Identification of non conformations/ Corrective Actions Preventive actions

Jan

Feb

Mar

Apr

May

Test methods Equipment Measurement traceability Sampling Handling of test and calibration methods Reporting of results Service to the customer Personnel Resolution of complaints Accommodation and environment Management review Improvement

Page 22 of 64

Jun

Jul

Aug

Sep

Oct

Nov

Dec

3.3 AUDITOR PREPARATION Audits are conducted by observing activities taking place, asking questions, evaluating responses and examining items such as records, equipment and reports. Auditors are unlikely to be effective in these activities unless they have prepared a clear plan of exactly what they intend to look at during the audit and while they are looking at those items, specifically what they will be looking for. The written list of the items the auditor intends to look at and for during the audit is called the audit checklist. To avoid omission of specific areas, this checklist can be designed to include detailed headings for each of the aspects to be audited. Auditors should arrive for the audit with a very clear understanding of what items they intend to look at and while they are looking at those items, specifically what they intend to look for. Checklists can be generated in two ways: 1) The Audit Manager generates the checklist for the auditor. Typically, the Audit Manager maintains a master list containing a very large number of checklist items covering all aspects of the QMS. This has the advantage of facilitating the Audit Manager‟s responsibility to ensure comprehensive audit coverage of the laboratory‟s activities. 2) The Auditors generate their own checklists for each audit they undertake. Typically the

auditors read those parts of the quality system documentation that relate to the activity to be audited and decide what items to look at and for based on that reading. The Audit Manager can advise the auditor on topics/activities/QMS elements that should be covered.

Note Unless the auditors have a clear idea of what they intend to look for, they cannot realistically estimate how long each checklist item will require and hence whether or not the overall checklist (audit plan) is reasonable in the time allocated.

Figure 6 is a checklist developed using the 20 Milestone approach. This checklist can be used to perform a quick overview of the Quality Management system when the laboratory staff are in the process of developing and initiating the Internal Audit system. Once audits have been well established and the QMS is up and running smoothly, the laboratory staff can then move to a more complex checklist which includes details elements from the standard. Figure 6 Example Twenty Milestones Audit Checklist Standard Requirements MS 1: Quality Policy Have the Objectives of the quality management system and the laboratory's commitment to quality been defined in a quality policy statement? MS 2: Management Is there an organizational plan for the laboratory? MS 3: Scope Has the laboratory documented its scope? MS 4: Quality System Does the laboratory have a written document control system? Is there a Quality manual of policies and reference to procedures readily available to all personnel? Has Laboratory management established, documented,

Look for

Page 23 of 64

Look at

Documents/Actions

Standard Requirements implemented, and maintained a quality management system? Is there an updated document distribution master list? Is there a plan for continuous quality improvement? MS 5: Procedures Is there a documented procedure describing methods for sample collection and labelling, preservation, and conditions for transportation, and storage before testing, consistent with good laboratory practice? Is there a complete procedure manual reflecting current procedures? Does the laboratory have a system documenting that all personnel are knowledgeable about the contents of procedure manuals (including changes) relative to their activities? Are there guidelines for the release of results? Is there a program for management and retention of records? MS 6: Test Instructions Is there documentation that all personnel performing sample collection have been trained in the proper selection and use of equipment/supplies, and collection techniques? Are new procedures validated initially and periodically verified? Are there instructions for use of Instruments available? Are up to date procedures documented and available to staff as necessary? MS 7: Sampling Is the sampling manual distributed to all sample collecting areas within the lab AND to areas outside the main laboratory (such as client sites or other laboratories)? Are chain of custody forms completed with the required information? Are samples safely disposed of according to established policy? Are QC values evaluated for suitability? Are results verified (double checked) before release? MS 8: Review contracts Are requests properly reviewed before work is accepted? MS 9: Equipment Does the quality control program clearly define goals for monitoring analytic performance, procedures, policies, tolerance limits, corrective action and related information? Are control samples tested in the same manner and by the same personnel as client samples? Is there evidence of active review of results of controls, instrument maintenance and function, temperature, etc. for routine procedures? Is there documentation of corrective action taken when controls, etc. fall outside of control limits? Does the system provide for the timely correction of errors? MS10: Validation and Quality Control For quantitative tests, has a statistically valid acceptable range been established for each lot of control material by repetitive analysis in runs that include previously tested control materials? MS 11: Work conditions Does the laboratory have adequate conveniently located space so that the quality of work, quality control procedures, safety of

Look for

Page 24 of 64

Look at

Documents/Actions

Standard Requirements personnel, and clients are not compromised? Is there a Safety manual of policies and procedures readily available to all personnel? MS 12: Literature Have computer applications associated with testing and laboratory information been validated and safe guarded? Are there explicit documented policies that specify who may use the computer system to enter or access data, change results, or alter programs? MS 13: Traceability (Calibration) Are all reagents checked when received and labeled as to content, date of receipt, date of opening and date of expiry? Is there a documented schedule, procedure and active review for regular maintenance of Equipment either by maintenance contracts or documented in-house procedures? MS 14: Uncertainty Have uncertainty budgets been estimated and calculated for each type of test area or method? Have tolerence limits for testing properties been established? MS 15: Training Is there a functional continuing education program adequate to meet the needs of the laboratory personnel? Are personnel records maintained on all employees? Has the competency of each person to perform his/her assigned duties been assessed? MS 16: Monitoring Is the Proficiency testing program sufficient for the extent and complexity of testing done in the laboratory? Is there appropriate evidence of review of problems, and their solutions, as identified by PT? For areas where proficiency testing is not available, are other procedures used to validate performance on a regular basis? Does the quality improvement (QI) program follow a documented operational plan? Are key indicators of quality monitored and evaluated for improvement opportunities? MS 17: Internal Auditing Are internal audits planned and conducted as scheduled? MS 18: Complaints Have the clients' and staff satisfaction with the laboratory service been measured within the past 2 years? MS 19: Review Documents Are quality control records available at the workstation? Are control data organized and presented so they can be evaluated daily by the technical staff to detect problems, trends, etc? Are manual and automated result entries verified before final acceptance and reporting by the computer? Is there a documented system to ensure that all revised reports for previously reported incorrect (erroneous) results are identified as revised, corrected, or amended on all forms of reports (paper, video displays, etc.)? MS20: Management review Have review meetings been conducted? Have corrective actions been addressed & recorded?

Look for

Page 25 of 64

Look at

Documents/Actions

3.4

AUDITOR COMPETENCIES

As stated above, auditors have to effectively observe and examine, ask pertinent questions, evaluate responses, critically analyse the information they receive while comfortably interacting with auditees, laboratory management and the Quality Manager. A successful auditor must therefore exhibit a mix of the competencies detailed in Figure 7 below. Figure 7 Competencies required for effective auditing Competencies (based on ISO/IEC 19011:2002) An Auditor must: Be open minded and mature Be willing to consider alternative ideas or points of view Possess sound judgment & analytical skills Be ethical – fair, truthful, sincere, honest, and discreet Be observant – actively aware of physical environment and activities therein Be decisive – reach timely conclusions based on logical reasoning and analysis Be self reliant – acts and functions independently while interacting effectively with others Be tenacious – persistent, focused on achieving objectives Obtain and assess objective evidence fairly Remain true to the purpose of the assessment Interact with personnel in such a way that will best yield the assessment objective Constantly evaluate the effect of assessment observations and personal interactions during an assessment Commit full attention and support to the assessment process React effectively in stressful situations Perform the assessment process without deviating due to distractions Arrive at generally acceptable conclusions based on assessment observations ASSESSOR KNOWLEDGE AND SKILLS Understand the criteria in the relevant accreditation standard (i.e., ISO/IEC 17025) Understand the accreditation body‟s policies applicable to the desired scope of accreditation Be able to accurately interpret and apply the criteria to the actual assessment situation Demonstrate a track record of appropriate knowledge and skills applied in a laboratory environment EDUCATION At a minimum, post secondary education in a relevant scientific/technological discipline Attestation from technical peers to confirm the relevance of education and knowledge credentials

Page 26 of 64

Lead Auditor

Auditor

Technical Auditor

CHAPTER 4: CONDUCTING THE INTERNAL AUDIT 4.1 INITIATING THE AUDIT PROCESS Taking the First Steps Internal auditors who develop their own checklists usually do so after reading the available documentation about the activities to be audited. Documentation will include relevant sections of the quality manual as well as relevant procedures and test methods but may also include: 1) sampling plans 2) QC plans 3) contracts 4) standards and 5) other pertinent documentation. During the review of documentation the auditor may find some items that appear to be noncompliant. For example, what is stated in a specific procedure may not appear to correspond to the related policy outlined in the Quality Manual or to the related client contract. A further example may be that what is stated in a test method may appear not to be compatible with the requirements of the standard. When auditors find such apparent discordances, they include them in their checklists so that, on the day of the audit, the apparent non-compliance can be confirmed or refuted. Audits will address QMS elements such as document control, sample reception and identification, equipment control, labelling of reagents, record keeping, etc. However, they should encompass, to the extent permitted by the scope and the technical background of the auditors, the technical correctness of the laboratory‟s activities. These more technical aspects will usually include staff competencies, equipment, QC and method reliability, environmental appropriateness and sample management. Audits based on checklists are generally compliance and implementation focused. During the preparatory stage the compliance of the documented system is assessed and on the day of the audit the correlation between documentation and actual practice is determined. Since audits are also intended to provide information on the effectiveness of the QMS, that is, the extent to which the laboratory‟s quality objectives are being achieved, it is important to ensure that these quality objectives are written in specific measurable terms. Note While a laboratory‟s documentation may be satisfactory, the auditor MUST confirm that the documented description corresponds to the actual practice.

Page 27 of 64

Figure 8 Example of Audit Elements

   

Staff qualifications, training, experience basis for authorization to conduct tests/use equipment ongoing proficiency checks knowledge, understanding and performance of test methods

     

Equipment initial verification for compliance with test methods fitness for purpose on-going maintenance, calibration, intermediate checks, etc. status indication and limitations on use availability of operating instructions, in house calibration appropriate environment for use and storage

             

4.2

Methods suitability/availability of documented methods, correctly followed appropriate interpretation (where required) preparation of reagents current and updated methods sampling Environment complies with requirements of test methods recorded and monitored (to extent necessary) appropriate housekeeping Sample Handling sampling techniques and sample ID conditioning and preparation of samples retention and disposal of samples Quality Control control of reference standards and materials appropriate methodology QC checks correctly analysed and interpreted action taken on results when appropriate

CONDUCTING THE AUDIT

The time committed to the audit process is largely spent observing activities taking place, asking questions, interviewing the technicians and other staff, listening to responses, formulating follow-up questions, and seeking clarification. Thus conducting the audit at the location where the activity is taking place is essential. It is important for internal auditors to have contacted the relevant Manager prior to the audit to confirm the arrangements, including what they wish to witness and who they would like to talk to. A brief meeting or other contact with the Manager just prior to the audit to reconfirm the arrangements should be undertaken before initiating the audit process.

Page 28 of 64

Questioning Styles The types of questions asked and the way in which questions are asked are critical to the responses and information received. Different types of questioning styles exist and each can prove to be effective in specific circumstances. These include: a) Closed questions: Closed questions invite Yes/No responses but may miss significant bits of information. b) Open questions: Open questions may be prefaced by the words how, what, why, where, and who and are likely to obtain a richer body of information than closed questions. When the response to an open question is not what the auditor is expecting it to be, or it is not clear, rephrasing the question is usually worthwhile to obtain clarification or confirmation. c) Hypothetical questions: Hypothetical or „what if‟ questions can be helpful but must be used with caution. If the auditors consider a particular circumstance to have the potential to lead to quality problems but can find no evidence to show that this has yet happened and are unable to persuade the auditee of the potential for problems, they can hypothesize the circumstance under which they think the problem will arise by asking „what if‟ questions. The circumstances hypothesized must, of course, not be so improbable that it is unreasonable to expect the QMS to be able to accommodate them. Questions based on unstated assumptions should, similarly, only be used with care since they can very easily lead to miscommunication. The internal auditor may not have technical expertise in the actual test being conducted, but can look at the control procedures and evaluate their implementation and effectiveness. If, having looked at an item on the checklist, no problems are found, the auditor should move on to the next item on the checklist. From time to time the auditor may find what appears to be a non-conformance, or failure to implement effectively or some other condition likely to adversely impact quality. When this happens the auditor should investigate the problem to the depth necessary to confirm or disprove its existence and to understand the nature and magnitude of the problem. At an early stage in this process the auditor should explain to the auditee the circumstances they think they have found and why it is considered to be a problem. This gives the auditee an early opportunity to correct any misunderstandings or miscommunication and if it is a problem, the auditee can assist the auditor to understand the circumstances in which it arose. When the auditor has clarified the circumstance, all the information needed for presenting the report and explaining the problem to management should be recorded. Auditees must be aware of what the auditor will be reporting before the auditor proceeds to the next checklist item. Note Most problems that occur in labs arise from simple and preventable circumstances such as: misidentified and mixed up samples, procedures not completely followed, results incorrectly recorded or transcribed. The QMS should have control procedures established to minimize and/or prevent these and to identify them if they do occur.

Page 29 of 64

CHAPTER 5: MANAGING AUDIT OUTCOMES 5.1

NON-CONFORMITIES

A nonconformity is simply defined as “nonfulfilment of a specified requirement”. When auditors find objective evidence of a nonconformity they are required to report this by writing a nonconformity statement on a nonconformity report form (NCR), also called a corrective action request (CAR). Often the objective evidence is physical, such as an unauthorized change to a controlled document, a nonconforming product, or an uncalibrated gauge being used. What the auditee states of others or claims happened elsewhere cannot be recorded as objective evidence. Objective evidence is FACTUAL, not based on opinion, judgment, or prejudice but on validated information. Figure 9 Example of a Non Conformity report form Lab Name Audit Date Number

1 2 3 4 5 6 7

Department Description of Non conformity

Reference to ISO 17025

Section in Quality Manual/Procedure

Auditor

When the auditor has objective evidence of failure to comply with a specified requirement they may raise a nonconformity. In a laboratory, the primary source of specified requirements is the laboratory‟s quality manual, procedures, test methods and the remainder of the quality system documentation. What is stated in its quality system documentation must be done. If the laboratory is accredited or is seeking accreditation then the relevant requirements of the accreditation body are also specified requirements and there may, as well, be other regulatory requirements in specific fields of testing. Types of Non Conformities Nonconformities fall into several groups. These include a) failures of intent b) implementation failures c) effectiveness failures ‘Intent’ Nonconformities Intent nonconformities arise when the QMS, as it is intended to be operated, is non-compliant with external requirements. Examples of these are as follows: a) a legal requirement may be that results of pesticide testing be retained for 15 years but the laboratory‟s QMS permits disposal of quality records after 5 years Page 30 of 64

b) a labelling requirement to which the laboratory is committed to comply requires that the estimated uncertainty of measurement be included on all test certificates but the laboratory‟s practice is to include this only if requested by the client c) An accreditation body requires that work be sub-contracted only to laboratories that they have accredited for that testing activity but the laboratory‟s QMS authorizes them to subcontract to any laboratory they deem competent. In all of these examples, the system, operating the way it was planned and intended to operate, is non-compliant with a specified requirement. Hence such nonconformities are often referred to as “system nonconformities” or “intent nonconformities” or “structural nonconformities”. ‘Implementation’ Nonconformities Failures of implementation occur when one or more of the requirements of the laboratory‟s QMS are not complied with. Examples of this are as follows: a) the laboratory‟s policy, as stated in the quality manual, is that raw data and original observations are recorded in the technician‟s notebook and then transferred as derived results onto the worksheets. The auditor found, however, that one technician actually records the raw data on the rear of the worksheet not in the notebook b) the laboratory procedure requires that environmental conditions be monitored every day, but it is found that the technician actually records environmental conditions every 3 days. While both of these findings are implementation nonconformities, the first is relatively minor since raw data is not destroyed. The corrective action for this may simply be to change the QMS to permit this practice. However in the second instance the nonconformity is more serious because of the departure from the laboratory‟s QMS that requires the monitoring of environmental data which may affect results. ‘Effectiveness’ Nonconformities The auditors will find circumstances where the QMS is both compliant and being implemented as intended, but nevertheless there is still a quality problem. Examples of this are as follows: a) the auditor may find that a scale, purchased from The Equipment Co. and certified as meeting the requirements OIML 123, did not in fact comply with those requirements. b) a technician is authorized to perform a test or operate an item of equipment on the basis of training received, but is then found not to be fully competent to do so. The auditors establish that the laboratory‟s procedures for evaluating and approving suppliers are fully compliant with external requirements and recommended good practices and had been followed. Additionally the laboratory‟s follows its training procedures; however it does not verify the effectiveness of the training. It is evident in both situations that those procedures were not effective at least in this case. Classifying Nonconformities by Importance (Potential Impact) Most internal audit systems require auditors to classify the nonconformities they raise according to their importance. The most commonly used classifications are „Major‟ or „Minor‟ nonconformities. This often leads to differences of opinion between auditor and auditee, especially if the classification criteria are not clear or the decisions are subjective. Page 31 of 64

A major nonconformity is usually defined as the absence or complete failure of a required quality system element and a minor nonconformity as an isolated failure to comply with a quality system element. Thus intent nonconformities, because they will lead to ongoing non-compliance perhaps throughout the laboratory, are more likely to be classed as major than will implementation or effectiveness nonconformities. Major Nonconformities Major nonconformities can include for example when the auditor finds: a) that the laboratory‟s QMS did not include a requirement for the monitoring and control of amendments to documents or b) that while the QMS does have a requirement for control of amendments there are 3 documented test methods with unauthorized amendments or c) two cases of technicians working with old versions of SOPs and a worksheet that carries no revision status indication or d) that a test was incorrectly carried out because the documented procedure could not be found and the technician relied on his memory The combined effect of the above findings suggest that the laboratory‟s document control system is not working effectively. The consequences of the nonconformity also have a bearing on the classification. A nonconformity which seriously jeopardises the validity of test results received by the client, or has the potential to do so, would be a major nonconformity. Specific circumstances may thus dictate if a nonconformity will be recorded as major or minor by the auditor. For example: a) if a nonconformity would have been detected by the laboratory‟s control mechanisms prior to release of results and the laboratory could recover the situation by retesting, most auditors would classify this as minor b) If retesting was not possible (perhaps a destructive test) or retesting would take too long to meet the agreed date for results to be provided, this is more serious since the laboratory has failed to meet its client agreement and depending on the circumstances this might be classified as major c) If the nonconformity would be unlikely, had the auditor not found it, to have been detected by the laboratory‟s control mechanisms and thus invalid test results would probably have been supplied to the client, this is even more serious. Most auditors would seriously consider recording a major nonconformity in this situation d) Finally, if the auditors found that this had in fact happened and that, over an extended period, invalid results had been given to clients, this is almost as serious as it can get. This would certainly be classified as a major nonconformity

5.2

RECORDING and REPORTING AUDIT FINDINGS

Recording Audit Findings Laboratories are required to maintain detailed records of all audits, which are clear and readily accessible. The audit records must indicate and document what activities the auditors observed Page 32 of 64

(such as test methods, equipment, records, and documents), who was involved, and what was found. This is necessary so that future audits of the same department can complement the audit, not just repeat it, and can also achieve more comprehensive coverage. If the auditors consider they have found nonconformity then it is very important to ensure that all relevant details are recorded so it can be understood later, such as WHAT they found and WHY it is a nonconformity. Depending on the circumstances it might also be necessary to record WHERE and WHEN the events occurred and WHO was involved. Documented information on nonconformities must also include the location of the specified requirement that has not been complied with (e.g. 4.6.2 of Quality Manual, section 5 of test method ASTM 1234) and, why this is an actual or potential quality problem. Non-conformances must be fully recorded, with details of the corrective actions suggested by the auditee, how and by whom they are to be implemented, and including an agreed timescale for their achievement, which is based on the seriousness of the non-compliances and decided in consultation with the Quality Manager. Audit summary forms are useful in maintaining a record of the audit findings and corrective actions. This summary will help to highlight areas of weakness and deterioration of the QMS but it may also include positive statements about satisfactory performance. This should be signed by both the Quality Manager and Head of Laboratory/Managing Director. Reporting Audit Findings Each audit report should include the following: a. name of auditor b. date of audit c. areas (departments) audited d. details of aspects examined including sample numbers, equipment identification etc, where appropriate (even where no non-conformances were observed) e. any non-conformances observed; (category e.g., major or minor; if necessary) f. corrective action agreed, responsibility for and time period for completion, confirmed date for completion g. signature of the Quality Manager confirming that the corrective action process has been completed. Most systems require the auditor to produce a written report summarizing the audit findings. This should outline what was audited and what was found. Both conformance and nonconformances can be addressed in the report. Nonconformities need not be repeated but can simply be referred to. A summary report form can be provided to minimize the auditors work. Note Where the audit reveals serious non-compliances, consideration should be given to auditing the relevant areas again in the near future to check that the actions taken are still effective.

Page 33 of 64

Figure 10 Summary Audit report form

Instructions to Auditor: Prepare the narrative report by using and attaching separate sheets. The report should summarise what was seen and the findings of the Audit. The total number of pages of the report should also be identified on each page. LABORATORY NAME: QUALITY MANAGER: REPORT DATE: AUDITORS (PRINTED NAME / SIGNATURE/DATE): _________________________________/_______________________________/____________ _________________________________/_______________________________/____________ Names of Laboratory Staff Attending the Closing Meeting __________________________ ___________________________ __________________________ The auditor(s) discussed the contents of this report and the nonconformity report with relevant laboratory staff. SIGNATURE OF LABORATORY OFFICIAL:____________________________ DATE:__________

________________________________________________________________________

________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________

Writing Nonconformity Statements When auditors raise a nonconformity they are usually required to write statements of nonconformity for anyone not involved in the audit to understand. Clear statements, for example, on what was found and why they were considered to be nonconformities need to be documented. Thus the Auditor, personnel with responsibility for proposing corrective action, persons with authority to approve the proposed corrective action and those responsible for verifying that the corrective action has been effective, will need to understand the nature of the problem. The statement of nonconformity must provide that understanding and thus it is important that the statement is factual, concise and complete.

Page 34 of 64

5.3

ROOT CAUSE INVESTIGATION

Identifying the Root Cause of Nonconformities When a problem is identified, the lab‟s corrective action procedures should ensure that action is taken at three levels: I. First: the problem has to be fixed e.g., retest, notify the client, repair malfunctioning equipment, notify staff concerned, redraft worksheets, amend procedures, etc. II. Second: the scale and nature of the problem needs to be determined - Is it a oneoff occurrence or not? Has it happened before? Are similar things happening in other sections? III. Third: the underlying reason(s) or root cause for the problem need to be determined and appropriate action taken such that the problem is unlikely to recur. Auditors can maximize the benefit of the audit if they can steer the manager to what they consider to be the root cause(s). This can be achieved in various ways. Examples are as follows: a) By wording the nonconformity statement appropriately and identifying the clause within the document/standard against which the nonconformity is assigned b) By ensuring that the problem is clearly articulated to the auditee at the time of the audit. If the proposed corrective action appears to be superficial, fixing the problem but not the root cause, the auditor can ask for more appropriate action. Usually this is best done indirectly by asking „what if‟ questions. Avoid direct advice to facilitate “ownership‟ of the solution by the auditee. Root Cause Investigation A „Root Cause Analysis‟ is a process for identifying the basic or causal factors that underlie variation in performance such as, for example, the occurrence of an adverse event, a near miss, or a non conformity. The focus is primarily on systems and processes and not individual performance. Figure 11 outlines the types of questions that may be posed in a root cause analysis process.

Figure 11 Example of a Root Cause Analysis Report Level of Analysis

Questions

What happened?

Adverse Event/ Near Miss

What are the details of the event: (Brief description) When did the event occur? (Date, day of the week, time) What area was impacted?

Why did it happen?

The process or activity in which the event occurred

Reference the affected process or SOP What steps were involved in (contributed to) the event?

Page 35 of 64

Findings

Root Cause? Yes/no

Ask “Why”? Yes/no

Take Action? Yes/no

Level of Analysis

Equipment factors Environmental factors

Other

Why did this happen? What systems underlie those factors?

5.4

Human Resource issues

Questions To what degree is necessary information available when needed? Is it accurate, complete, and unambiguous? How did the equipment performance possibly affect the outcome?

Findings

Root Cause? Yes/no

Ask “Why”? Yes/no

Take Action? Yes/no

What factors may have directly affected the outcome? To what degree was the physical environment (or changes) a factor in the event and did it affect the processes being carried out? Are there other factors that may have possibly influenced this outcome? Were there any uncontrollable factors involved in the event? What was the role of staff that may have contributed to the outcome? How can orientation and training be improved?

TAKING CORRECTIVE ACTION

Acting on Audit Findings It is important that the Quality Manager and the laboratory‟s internal auditors monitor progress on non-compliance actions to ensure that agreed timescales are met. Corrective action request forms (CARs), on which auditors report the non conformities found, are useful. In certain situations the laboratory may have to stop calibration/testing in a specific area until satisfactory remedial action has been implemented. It may also be necessary to take action with regard to work already completed which may now be considered suspect. Thus, whenever a non-conformity that may jeopardize the result of a calibration or test is discovered, the corresponding activity should be halted until the appropriate corrective action has been taken and has been shown to lead to satisfactory results. In addition, results that may have been affected by the non-conformity should be investigated and customers informed if the validity of corresponding calibration, test or inspection certificates/reports is in doubt. Responsibilities The corrective action procedure will need to be followed to reveal the root causes of some problems and to implement effective corrective and preventive actions. The implementation of the agreed corrective action is the responsibility of the auditee. The auditor should check the effectiveness of corrective actions as soon as possible after the agreed time frame has elapsed. The Quality manager should have the ultimate responsibility for confirming the clearance of nonconformities and then closing them out. Page 36 of 64

The laboratory must have procedures to ensure that appropriate corrective action is proposed, authorized, completed and verified for all nonconformities raised by the auditors. Responsibility and authority for each stage must be defined and documented. Usually the authority to accept the proposed corrective action lies with the Lead Auditor, Quality Manager (if different), or Lab Manager. The auditor may be asked to verify that the approved corrective action has been taken and has been effective. For internal audits this is usually most readily accomplished by returning to look at the evidence „first hand‟. In other cases the evidence may be sent by email.

5.5 MANAGEMENT REVIEW The prime purpose of the internal audit system is to provide information that will enable laboratory management to make an informed judgement as to the extent to which the overall QMS is appropriate, compliant and effectively implemented. The entire cycle of audits which cover all areas of the QMS is intended to verify continued compliance of processes. The mechanism in the QMS for reviewing the effectiveness of the System and decision-making on the improvement process is the Management Review, usually conducted annually. The Management Review provides the opportunity for review and discussion of the combined audit findings, examination of patterns and trends demonstrated by the audit data as well as providing the opportunity to highlight particular areas of laboratory strength and weakness and to discuss strategies for quality systems strengthening.

Page 37 of 64

CHAPTER 6: AUDIT TOOLS 6.1

AUDIT CHECKLIST

Instructions - If the laboratory has implemented the action, process or procedure referred to, the auditor will tick YES and state in the Proof/Comments column what was seen to verify such. If not, then tick NO and make any comment which might be necessary for follow up at a later date. Figure 12 Example Audit checklist

4.1 ORGANIZATION AND MANAGEMENT Does the laboratory have a documented description of the medical laboratory that includes its current licensure and/or other legally required recognition to function as a medical laboratory? Does the laboratory have a documented description of the services it provides for patient care? Does the description include services it provides for patient care? Does the description include services that address the needs of clinical services? Does the laboratory have a written policy that declares commitment to the standard, regardless of the site or facility where work is performed? Does the laboratory have a written policy that addresses the declaration of conflicts of interest that could influence the credibility of laboratory activities? 4.15 MANAGEMENT REVIEW Are there dated documents signed by the laboratory director indicating review of the laboratory quality management system? Do the reviews indicate the suitability and effectiveness of the system? Is there a currently dated and signed documented management plan indicating quality goals, objectives, and action plans? 5.1 PERSONNEL Are there documentation and records to show that the requirements have been met? Are the organizational chart and personnel records relating to qualifications, continuing education and training, competency evaluations, and job descriptions available? Are the roles, responsibilities, and authority of the laboratory director included either in the job description for the position or in a formal letter of appointment? Is there documentation on how the laboratory director fulfils the responsibilities regarding the professional, scientific, consultative, organizational, administrative, and educational matters? Is there a process of appointing and monitoring delegates for assigned responsibilities? Is a process for monitoring of all work performed in place? Are quality improvement strategies and activities documented? Has a quality management system been implemented?

Page 38 of 64

Yes

No

Proof/Comments

Has the laboratory shown that the resources are adequate to meet the demands of the volume and scope of services provided by the laboratory? Are there policies, processes, procedures and records dealing with the competency assessment of staff and their task assignment, training and continuing education? Are appropriately qualified and experienced personnel making relevant professional judgments on the examinations carried out? Have policies been developed and implemented related to electronic data entry, storage, security, access, modification, and retrieval? 5.2 ACCOMMODATION AND ENVIRONMENTAL CONDITIONS Have safety precautions been implemented in the laboratory? Do personnel have adequate bench space and/or room? Are staff and patients protected from unwarranted hazards? Do ambient temperature control, electrical supply, and water supply meet the requirements of the in-vitro diagnostic systems and computer systems? Are the laboratory design and environment suitable for the volume and scope of services provided? Are environmental conditions monitored that could adversely affect the quality of the measurements or other results? Is there segregation of incompatible sections of the laboratory? Is access to the laboratory or parts of it controlled? Are processes and procedures in place to store samples, slides, blocks, and documents, including records and results? Are work areas clean and well maintained? 5.3 LABORATORY EQUIPMENT Does the laboratory have processes, procedures, and results for the verification of all instruments and analytical systems? Have the laboratory results obtained from instrument systems been validated prior to release? Are there results of the routine monitoring of instruments, reagents, and analytical systems? Is the required calibration of instrument systems carried out and documented? Are there documented processes, procedures, and records of preventive maintenance? Are preventive maintenance records reviewed by management? Is each piece of equipment uniquely identified? Is there an effective inventory control system for reagents? Is the detailed acquisition, use, maintenance, repair and replacement information available for each piece of equipment? Are hazardous materials correctly identified and stored? Are the required hazardous material data sheets readily available to all staff? Is there evidence that personnel are trained and authorized to use items of equipment, and are records available? Are equipment manuals and/or instructions for use available at the work site?

Page 39 of 64

Yes

No

Proof/Comments

Is equipment correctly installed and inspected? Are personnel aware of the regulations, processes and procedures for the safe disposal of chemical, radioactive and biological materials? Are documents available to show how defective equipment is identified and decommissioned? Is the medical significance of defective equipment performance determined, and is there appropriate notification of the health care professional? Are safe practices followed for the servicing of defective equipment? Is a record of instrument/equipment calibration available at the work site? Is calibration or verification carried out on equipment that has been repaired or otherwise has been outside of direct control by the laboratory? Does the laboratory apply correction factors and, if so, are they correctly developed and applied? Have computer systems been shown to be suitable for intended use in the laboratory? Are electronic data secured? 5.4 PRE-EXAMINATION PROCEDURES Does the request form seek information detailed in the standard? Are there processes and procedures to seek information missing from a request form? Are monitoring and feedback available on completion of request forms? Is a primary sample collection manual available in paper or electronic format? Does the manual meet the requirements of the standard? Are users aware of the manual, and do they follow its directions? Are laboratory personnel familiar with the contents of the manual? Does the primary sample collection manual document control meet the requirements for such a document? Is there active monitoring of sample transport? Are there processes and procedures in place to monitor the integrity of samples and conformity to safety regulations during transport? Is traceability of the primary sample to an identifiable patient a requirement for accepting a sample? Are there procedures, processes, and facilities for accepting, examining and reporting on urgent samples? Is there evidence of traceability of secondary samples to the primary one and to the requisition? Are verbal requests for laboratory services accepted and, if so, is there documentation and traceability of the request? Are samples stored under appropriate conditions, at least until issue of the final report?

Page 40 of 64

Yes

No

Proof/Comments

6.2

AUDITOR PREPARATION

Sample Procedure for Training Internal Auditors

Sample Procedure for training Internal Quality System Auditors ACME Practice Laboratory Ltd Page 1 of 2

Approved by Quality Manager

Issued by General Manager

Issue Date: 9th June 2009 Document No. Revision No. 1 P010 Revision date : 9th June 2010

AMENDMENT SHEET Version

Page No.

Description of Change

Approved by

Effective Date

PURPOSE This procedure provides guidelines for the management of Internal Quality Auditor training for the Staff within the Laboratory. It explains the steps by which Auditor training needs should be identified, implemented, recorded, audited and reviewed. SCOPE This procedure applies to the theoretical and practical training of Laboratory staff as Internal Auditors for the purpose of implementing the Internal Audit schedule and program. DEFINITIONS Quality Manual – a document stating the Quality policy, quality Management system, and quality practices of an organization. Test Methods – Technical Procedures for conducting testing of samples in the Laboratory SOP – Standard Operating Procedures

Page 41 of 64

PROCEDURE START

Responsibilities

Documents Generated

Identify persons to train as auditors

List of Trainee Internal Auditors

Quality Manager

Identify facilitator and establish objectives

Training Proposal

Trainer

Develop training materials

Quality Manager

Human Resources

Training Manual

Training Schedule

Schedule and organize training

Trainer

Attendance Register Evaluation Forms

Conduct training

Quality Manager

Is the training effective?

No

Conduct investigation & document findings

YES

Quality Manager

Update training records Quality Manager Plan and execute Audits

Competency Evaluation

END

Investigation Report

RELATED DOCUMENTS

Personnel Records

Training Matrix Employee training files

Audit plan and schedule

REFERENCES

Quality Manual Human Resources Records

Page 42 of 64

6.3

PROCEDURE FOR CONDUCTING INTERNAL AUDITS

Example Procedure for Conducting Internal Audits ACME PRACTICE LABORATORY BARBADOS Standard Operating Procedure

Issue Date:

Date Revised:

Effective Date:

Revision 1

LAB-SOP xyz – Procedure for conducting Internal Quality Audits Prepared By: Reviewed By: Approved By:

Page #

Date: Date: Date

Purpose

To verify that our Quality System is in compliance with ISO/IEC 17025 (ISO 15189) and that procedures are in compliance with our documented Quality System. To ensure the requirements of the ISO Standard are met and that all employees are following procedures of the Quality System.

Scope

Audits to the Quality System are completed once a year (more if necessary) in order to ensure the system is working and also to implement corrective or preventative actions if there are any problems

Definitions and Acronyms Quality System: Organizational structure, procedures, processes and resources needed to implement and maintain quality management. Auditor: An employee who has the qualifications to perform quality audits. Lead Auditor: An auditor designated to manage a quality audit. Observation: A statement of fact made during an audit and substantiated by objective evidence. Objective Evidence: Records or statements of fact, which can be verified. Nonconformance: The non-fulfillment of specified requirements.

Responsibilities

The Internal Audit Team is responsible for conducting all Internal Audits for the Quality System. Management will review all findings and actions related to Internal Audits.

Procedure

Quality System functions are audited once a year and more frequently as required based on status and importance. The schedule of audits will be completed by the Quality Manager and kept in the Quality System records. Audits are conducted by individuals who have the following qualifications:  General knowledge of company operations  Prior audit experience  Formal ISO training and/or on-the-job training

Page 43 of 64

Internal audits have a lead auditor to document the entire audit and the audit team to assist in the process. Internal Audits will be done in accordance with the written procedures and checklists which are found in the Quality System database. Audits should be led by an individual not directly related to the function being audited. Auditors will review revisions to the ISO Standard to assure the Quality Manual and related procedures have been updated to comply with the change. The following documents are used in the audit process:  Previous audit findings  A copy of the procedure (s)  Internal Assessment checklist  Previous Corrective Actions (Reports from past audits) Audit Preparation An experienced auditor is designated as the Lead Auditor. The Lead Auditor has final decisions regarding the conduct of the audit and any observations. The Lead Auditor is responsible for: a) Preparing and maintaining the Audit Schedule, auditee notification at least one week prior to an audit, b) Preparing the Audit Plan, conducting opening and closing meetings, c) Selecting the Audit Team, ensuring that Auditors are independent of the activity they have been assigned to audit, d) Briefing the Auditors, defining the requirements of each audit assignment and assisting in the preparation of audit checklists as needed, e) Assisting Auditors in preparing the Audit Reports, f) Reporting on Audit Results at Management Review Meetings. Opening Meeting Objectives a) Introduce the members of the audit team to the auditees and establish communication links with activities being audited. b) Review the scope and objectives of the audit and provide a short summary of the methods and procedures to be used to conduct the audit. c) Clarify the details of the audit plan and confirm the time and date for the closing meeting and any interim meetings. Conducting the Audit All areas of the QMS will be audited at least once per year. In addition to annually scheduled audits, the Lead Auditor may select certain activities for follow-up for more frequent auditing as their status and past compliance history changes. General Housekeeping, work environment, knowledge of the Quality System and Policy, traceability, document and data control and control of quality records are observed in all audits. If not reviewed, the reason must be noted in the on the Audit Report. The Auditors will collect evidence through interviews, examination of documents and quality records, and observation of activities and conditions in areas of concern. Information gathered through interviews should be tested or investigated by acquiring the same information from other independent sources, such as physical observation, measurements and records. All audit observations are documented, even if not originally covered on the checklist. Area management will be constantly informed of findings, there are to be no surprises at the closing meeting.

Page 44 of 64

During the audit, the Lead Auditor may make changes to the Auditors work assignments or the audit plan, if necessary to achieve the audit objectives. Closing Meeting Objectives a) Present audit observations to auditee‟s management, taking into account their perceived significance. b) Present the audit teams‟ conclusions regarding the effectiveness of the quality system in the form of audit reports and quality issue forms. c) Findings of nonconformance will be resolved through the Quality Issue process, keeping in mind that corrective actions need to be to a degree appropriate to the magnitude of the problem and commensurate with the risks encountered. NOTE: For internal audit quality issues, the Lead Auditor is authorized to accept or reject corrective actions unless problem solving is required. Follow-Up Audit Activities Necessary follow-up activities are determined by the Lead Auditor as corrective actions are submitted for approval. The Lead Auditor will note required follow-up on the audit schedule and determine an appropriate length of time needed to implement the corrective action and judge its effectiveness. Follow-up activities may be assigned to an auditor or performed by the Lead Auditor. Findings are noted on the original Quality Issue and, if satisfactory, the issue is approved by the Lead Auditor and closed. Internal Audits are reviewed in the annual Management Review Meeting. The corrective actions and the report (summary) are reviewed.

Documents Generated   

Audit checklists Audit reports Opening and closing meeting attendance list

Related Documents     

Corrective and Preventive Actions Procedure Auditor Training Procedure Audit Schedule Audit Plan Form Audit Report Form

References

ISO 17025 General Requirements for Quality and Competence in Testing and Calibration Laboratories END OF PROCEDURE

Page 45 of 64

6.4

CORRECTIVE ACTION RECORDING FORMAT

Figure 13 Example of a Corrective Action Form Level of Analysis

For any findings identified as needing an action, indicate the planned action, implementation date, and measure of effectiveness (how can the action be verified for effectiveness).

Corrective action

Who is Responsible

Measures of Effectiveness

Action Item #1

Measure:

Action Item #2

Measure:

Action Item #3

Measure:

Action Item #4

Measure:

Action Item #5

Measure:

Action Item #6

Measure:

Action Item #7

Measure:

Action Item #8

Measure:

Action Item #9

Measure:

Cite any references that were considered in developing this analysis and action plan:

Page 46 of 64

CHAPTER 7: FAQs on LABORATORY QUALITY What is calibration? Calibration is the process of comparing a measuring instrument against an appropriate standard to establish the relationship between the values indicated by the instrument and those of the standard (the standard provides a known and widely accepted reference point). Calibrating an instrument is intended to eliminate or reduce bias when the instrument is used to measure. Do all measuring instruments need to be calibrated? All measuring devices usually need to be calibrated - whether they are simple devices or state-of-the art systems - because certain instrument characteristics change with time. There are some situations where an instrument need not be calibrated, for example where its readings are „for information only‟ and their accuracy has little or no impact on the test, product or service being provided. How often should an instrument be calibrated? Calibration frequency can be established using the manufacturer‟s recommendations. In some cases, commercial calibration laboratories can suggest intervals for particular instruments. There are two main considerations when determining the calibration frequency. Firstly, the instrument‟s reproducibility should be taken into account. New devices should be calibrated frequently in order to establish their reproducibility. Secondly, the required uncertainty of measurement should be assessed. If the instrument‟s reproducibility is shown by successive calibrations to be substantially better than the potential uncertainty then the interval between calibrations can be extended. At the other extreme, where the instrument‟s reproducibility approaches the limits of uncertainty, the calibration intervals should be much shorter. In order to be confident that our calibrations are accurate we must be able to trace or link these results back to a calibration standard, held by a national measurement institute, through an unbroken chain of comparisons and where each link in the chain has stated measurement uncertainties. This is referred to as „traceability‟. In addition, it is important to ensure that appropriate equipment and procedures are used in the calibration process by trained and authorized personnel. Why perform calibrations? • To improve the measurement accuracy and reliability of the instrument • To reduce the number of end-products rejected because they are outside acceptable tolerances • To provide products with reduced and more competitive tolerances, better reliability, and access to quality-conscious markets • To minimize unnecessary rework and delays, resulting in customer dissatisfaction • To eliminate or minimise health and safety, legal and regulatory issues What is traceability? Traceability is the ability of a measured result to be linked or related or traced to appropriate measurement standards - generally international or national standards - through an unbroken chain of comparisons (traceability chain) in which all uncertainties are indicated. It should be noted that the instrument itself is not traceable, but the result produced by the instrument is! Traceability applies to both physical and chemical measurements. The standard referred to can be a material measure, measuring instrument, reference material or measuring system that defines, realizes, conserves or reproduces a unit or one or more values of a quantity that serves as a reference.

Page 47 of 64

What is the meaning of „traceability to international standards‟? Representatives of seventeen nations signed the „Metre Convention Treaty‟ which provided the foundation for the establishment of the International System (SI) Units. Since then, national standards laboratories have cooperated in the development of measurement standards that are traceable to the SI. Units of measurement with the highest accuracy are realized by international measurement standards. The value of the international standard is usually determined by comparison of national standards of the highest quality or in the case of the kilogram by the mass of the International Prototype. National measurement standards, maintained in a national metrology institute (NMI) are compared with these international standards before they are used as a reference for calibration of standards of lower precision. Therefore it is possible to demonstrate an unbroken chain of comparisons that ends at a national standards body. What is the traceability chain? It is a series of comparisons between the devices under test to a reference standard. The final comparison in the chain is made using the International System (SI) units as the reference. Each comparison is a link in the chain and the uncertainty of each comparison (link) must be known and documented. NMIs provide the ultimate measurement references for their countries. The intent of all NMIs is to match the SI units as closely as possible so that traceability to the SI can be established by comparing to an NMI that in turn compares its reference standards to the SI.

SI UNITS

Primary National Standard NMI Secondary Standards NMI Working Standards NMI or other Calibration ofinstrument

What are the benefits of traceability?  Allows you to be competitive and operate in the global environment thus reducing technical barriers to trade  It provides another way of guaranteeing a measurement‟s accuracy.  It is an essential element of a quality control system  It links people, organizations, documents, techniques and measurements within a large and diverse measurement community What is measurement uncertainty? Test or calibration measurements are subjected to imperfections in each step of a process – from sampling to final measurement. Measurement uncertainty (also referred to as “uncertainty of a measurement”) is the term used to describe this imperfection, and is defined as the parameter associated with the result of a

Page 48 of 64

measurement that defines the range of the values that could reasonably be attributed to the measured quantity. This indicates the interval within which the true value of the quantity being measured is expected to lie with a stated level of confidence. How is measurement uncertainty evaluated? Measurement uncertainty is evaluated by quantifying and combining a number of uncertainty components, which can be:  Random Effects – e.g., fluctuations in temperature, humidity, air-pressure, variability in performance of the measurement, sampling.  Systemic Effects – e.g., offset of measuring instruments, drift in its characteristics between calibrations, personal bias in reading an analogue scale or uncertainty of the value of a reference standard. The components are quantified either by evaluation of the results of several repeated measurements, or by estimation based on data from records, previous measurements, knowledge of the equipment and experience of the measurement. The evaluation from repeated measurements is done by applying a mathematical formula derived from statistical theory. The parameter determined is the standard uncertainty expressed as a standard deviation. Measurement uncertainty is usually written as an expanded uncertainty and provides an interval within which the value of the measurand (measured substance) is believed to lie with a higher level of confidence. It is obtained by multiplying the combined standard uncertainty by a coverage factor (k); where k is based on the level of confidence desired. For a level of confidence of 95%, k is 2. Why is measurement uncertainty important? • It provides laboratories and customers with valuable information about the accuracy and reliability of test or calibration data. • It tells how well the results represent the value of the quantity being measured. • It helps to maintain confidence in the comparability of results (which helps to reduce barriers to trade). • It shows whether a result is within the acceptable limits or outside of it. • It is a requirement of ISO 17025:2005 (clause 5.4.6) and ISO 15189 (clause 5.6) What is laboratory quality control (QC)? Quality control (QC) is an integral part of a laboratory‟s operation. It is designed to monitor, detect, reduce, and correct deficiencies in a laboratory's internal analytical process prior to the release of patient results and improve the quality of the results reported by the laboratory. Quality control is a measure of precision or how well the measurement system reproduces the same result over time and under varying operating conditions. The control can be a substance, sample or procedure intended to verify the performance characteristics of a test system. QC should not be confused with quality assurance (QA), which encompasses everything a laboratory does to assure high quality services to its customers. Why establish a QC program? Your QC program should be a monitoring system that provides immediate information about the acceptability of results, as well as a method for evaluating data over time. Most control material is accompanied by the manufacturer‟s approximate target values for the mean and standard deviations (S.D.). These stated values should only be used as a guide since the laboratory must always establish its own mean and standard deviation for each analytic procedure using a statistically significant number of control determinations (repetitions of 20 or more runs). Control charts, developed using the laboratory‟s calculated mean and S.D, are used to plot QC control results and to determine acceptability of a test run. Westgard rules may be used to analyze the QC data. Additionally, policies and procedures for remedial action for QC failures should be documented and applied as needed to maintain accuracy and reliability.

Page 49 of 64

What do we look for on QC charts? 1. Excessive scatter – points are widely scattered above and below the mean. This is evidence of poor reproducibility of results. Common cause is operator variability in pipetting or failure to strictly observe all steps in the procedure. 2. Bias – majority of points fall either above or below the mean. The two most common causes are an error in calibration or an error in reconstituting a control material. 3. Drift or trend – QC results move away from the mean in a particular direction over an extended period of time. Causes are outdated reagents, deterioration in reagents or instrument is losing its calibration. 4. Shift – control values suddenly change from the normal distribution above and below the mean to results that remain either above the mean or below the mean. Common causes are recent recalibration interventions, change in a reagent lot, sudden deterioration of reagents (e.g. possibly not refrigerated overnight), new technician or a major change in instrument performance. What are the benefits of conducting QC?  Monitors the stability of test methods or systems.  Assesses the accuracy and precision of test results and test methods.  Alerts the analyst if there is a problem with the reagent, test method, or instrument.  Demonstrates competency of laboratory personnel  Allows the laboratory to track the overall performance of a test procedure over time.  Detects random and systemic errors. What is method validation? Method validation is defined as the confirmation, through the provision of objective evidence, that the requirements for a specific test method or application have been fulfilled. The aim is to establish the operational limits and performance characteristics of a new, modified or otherwise inadequately characterised test method. When is method validation performed? Test methods should be validated when:  a new test method is being developed  an established test method is modified  QC indicates that an established method is changing with time  demonstrating the equivalence between two methods, e.g. a new method and a standard method. How is method validation performed? The studies used to determine method performance characteristics should be conducted with equipment that is within specification, working correctly and adequately calibrated. The technician conducting the study must be competent and capable of making appropriate decisions from the data produced during the study. The following are common characteristics tested during method validation studies:  Specificity/selectivity – ability to accurately measure the analyte in the presence of interferences.  Linearity - ability of an analytical method to produce test results that vary in a manner that is directly proportional to the changes in concentration of analyte in a sample - within a given concentration range.  Accuracy – closeness of the measured value to the true value for the sample.  Range – the concentration interval over which acceptable accuracy, linearity and precision are obtained.  Precision (Repeatability & Reproducibility) – the amount of scatter in the results obtained from multiple analyses of a homogeneous sample.

Page 50 of 64

   

Detection Limit – lowest concentration of the analyte that can be confidently detected by the method. Quantization Limit – strictly the lowest concentration that can be determined with an acceptable level of repeatability, precision and trueness. Robustness – ability of the test method to remain unaffected by small but deliberate changes, e.g. temperature. Recovery – assesses the efficiency of the method in detecting all of the analyte present.

Why is method validation necessary?  To provide objective evidence that the results are accurate and reliable  To demonstrate that a test method is “fit for implementation”  Validation data can be used to design QC programmes What is verification? Test method verification is a simplified validation process employed to check or verify a test method‟s performance characteristics. It typically includes a subset of the parameters evaluated when a complete validation is performed, and asks the question “Does this new test method perform to its specification in my laboratory?” When is verification conducted? It should be done before a new externally validated test method is employed by the laboratory to report results. How is verification conducted? The laboratory should:  Develop a clear, detailed verification procedure that defines the parameters to be evaluated.  Define and approve the acceptance criteria (e.g., manufacturer‟s package insert) to be used in analyzing the results.  Compare evaluation study results to the previously established performance characteristics.  Based on the results, accept or reject the test method.  Summarize the data collected from the verification study in a final report. The following is a listing of the most common subset of characteristics used in verification studies: Characteristics Analyzed  Accuracy  Precision  Reportable range of test results for the test method  Manufacturer‟s reference intervals (medical laboratories)

Types of Studies Performed  Comparison of methods to estimate inaccuracy or bias  Replication experiment to estimate imprecision  Linearity type experiment to estimate imprecision and to determine reportable range  Collecting reference values to verify the established reference range

Why is verification necessary?  To provide objective evidence that your laboratory has the ability to achieve acceptable results for a given test method.  To prove that an externally validated test method is acceptable for its intended use in your laboratory. END OF DOCUMENT

Page 51 of 64

APPENDICES APPENDIX 1: SUMMARY OF CLAUSES IN ISO/IEC 17025 MANAGEMENT REQUIREMENTS 4.1

Organization Laboratory management shall have responsibility for the design, implementation, maintenance and improvement of the quality management system.

4.2

Management system (QMS) The QMS shall have all its procedures documented. It shall include internal quality control and participation in interlaboratory comparisons. A quality policy statement and a documented quality manual are needed.

4.3

Document control Define, document and maintain procedures to control all documents and information. Define retention periods and archive documents for later reference.

4.4

Review of request, tenders and contracts Ensure that all parties understand and abide by the provisions made in contracts. Keep records of all reviews, including significant changes and maintain these records.

4.5

Subcontracting of tests and calibrations Have procedures for evaluation and selection of subcontracted laboratories and consultants. Maintain a register of all subcontractors used.

4.6

Purchasing external services and supplies Ensure that external services, equipment and consumable supplies consistently meet the laboratory‟s standards. Have an inventory control system and quality records of external commodities purchased.

4.7

Service to the client Clients shall be afforded consideration to verify that their samples are appropriately handled and analysed. Confidentiality shall be maintained.

4.8

Complaints Keep records of any complaints or feedback and of investigations or corrective action taken by the laboratory.

4.9

Identification and control of non-conformities When a non-conformity is detected: a) Personnel must be designated for problem resolution; b) Corrective action must be defined and taken immediately; c) The deviation must be documented, and reviewed by staff.

4.10 Improvement Review all procedures at regular intervals to identify any potential source of non-conformance. Action plans must be developed, documented and implemented as appropriate. 4.11 Corrective Action Carry out investigations to determine the root causes of problems. Document and implement any changes made to procedure. Review all procedures regularly and take action to improve the system. 4.12 Preventive Action

Page 52 of 64

Identify any necessary improvements and areas of non-conformities, either technical or concerning the management system. Develop action plans and procedures to reduce the likelihood of potential nonconformities. 4.13 Control of records Establish procedures for Identification, Collecting, indexing, access, Maintenance, Safe disposal of quality and technical records 4.14 Internal Audits Formally and periodically carry out internal audits to ensure that the QMS is maintained. 4.15 Management review Laboratory management must periodically review the QMS to maintain and improve the system. Findings must be recorded and any corrective action needed must be taken immediately. TECHNICAL REQUIREMENTS 5.1 General 5.2 Personnel Define qualifications and duties for all personnel. Implement the Quality Management System (QMS) and procedures. 5.3

Accommodation and environmental conditions Ensure the facilities for examination allow for correct performance of tests and reliability of results.

5.4

Test and calibration methods and method validation Use only relevant, validated, documented procedures which meet the needs of the users of the laboratory‟s services. Record any deviations or changes in procedure.

5.5

Equipment Provide the laboratory with relevant, functioning and calibrated equipment. Ensure authorized personnel operate equipment in a safe working environment.

5.6

Measurement traceability The laboratory must have documented evidence that measurements made by the laboratory can be traced to the International System of Units (SI).

5.7

Sampling The laboratory must be sure that there is a good understanding and agreement between the customer and the laboratory with respect to the sampling process to be carried out.

5.8

Handling of test and calibration items Authorized personnel must review results and determine reliability before releasing results. Disposal of samples must be in accordance with local standards.

5.9

Assuring the quality of test and calibration results Establish quality control methods to verify attainment of quality results. Document, record and retain records of results. Take action if results are suspected to be incorrect.

5.10 Reporting results Results must be reported legibly, unambiguously and in an identifiable manner. Use an audit log if the present reporting system cannot capture changes or alterations

Page 53 of 64

APPENDIX 2: LIST OF POLICIES IN ISO/IEC 17025

ISO 17025 Clause Number

Policy Name (Description)

For ensuring the protection of confidential information

4.1.5c

For avoiding involvement in activities that would diminish confidence in the laboratory‟s competence, impartiality, judgement or operational integrity

4.1.5 d

Quality Policy Statement

4.2.2

For conducting a review of requests, tenders and contracts

4.4.1

For the selection and purchasing of external services and supplies, equipment and consumable supplies that affect the quality of tests and calibrations

4.6.1

For the resolution of complaints

4.8

To identify and control nonconformities when they occur

4.9.1

To designate appropriate authorities for implementing corrective actions

4.11.1

For identifying training needs and providing personnel training

5.2.2

Page 54 of 64

Section in Quality Manual

DONE

NOT DONE

APPENDIX 3: LIST OF PROCEDURES IN ISO/IEC 17025 ISO 17025 Clause Number

Procedure Name (Description)

For ensuring the protection of confidential information

4.1.5c

For avoiding involvement in any activities that would diminish confidence in its competence, impartiality, judgement, or operational integrity

4.1.5d

To control all documents

4.3.1

For approval and issue of documents

4.3.2.2

For hand amendments of documents pending re-issue (if allowed)

4.3.3.3

To describe how changes in documents maintained in the computerised systems are made and controlled

4.3.3.4

For review of requests, tenders and contracts

4.4.1

For the selection and purchasing of external services and supplies that affect the quality of tests and calibrations

4.6.1

For the resolution of complaints

4.8

To identify and control nonconformities when they arise

4.9.1

To designate appropriate authorities for implementing corrective actions

4.11.1

To identify, document and eliminate the root cause(s) of nonconformities

4.11.2

For initiating and monitoring preventive actions

4.12.2

For identification, collection, indexing, access, storage, maintenance and safe disposal of records

4.13.1.1

To protect and back up records stored electronically

4.13.1.4

For internal audits which include the types of audit, frequencies, methodologies and required documentation

4.14.1

For conducting management reviews

4.15.1

For checking that the environment does not adversely affect the performance of specimen collection and equipment

5.2.2

To ensure good housekeeping

5.3.5

To describe test and calibrations methods

5.4.1

To describe validation methods

5.4.5.2 5.4.6.1

For estimating uncertainty of measurement for calibration labs For estimating uncertainty of measurement for testing Labs performing calibrations

5.4.6.2

For protecting the integrity of (electronic/computer) data at all times

5.4.7.2b

To provide up-to-date instructions on the use and maintenance of equipment

5.5.3

Page 55 of 64

Section in Quality/Procedure Manual

DONE

NOT DONE

Procedure Name (Description)

For safe handling, transport, storage and use of equipment

ISO 17025 Clause Number 5.5.6

For performing intermediate checks to maintain confidence in the calibration status of equipment

5.5.10

For ensuring correction factors are correctly updated

5.5.11

For calibration of equipment

5.6.1

For calibration of reference standards

5.6.3.1

For intermediate checks to maintain confidence in the calibration status of reference, primary, transfer or working standards and reference materials

5.6.3.3

For safe handling, transport, storage and use of reference standards

5.6.3.4

For sampling performed by the laboratory

5.7.1

For recording relevant data and operations relating to sampling

5.7.3

For transportation, receipt, handling, protection, storage, retention and/or disposal of test/calibration items

5.8.1

For avoiding deterioration, loss or damage to the test or calibration item during storage, handling and preparation.

5.8.4

For quality control procedures that monitor the validity of tests and calibrations undertaken

5.9.1

Page 56 of 64

Section in Quality/Procedure Manual

DONE

NOT DONE

APPENDIX 4: LIST OF “SHALLS” IN ISO/IEC 17025 Requirement

Met

Not Met

N/A

Clause

The laboratory shall be an entity that can be held legally responsible.

4.1.1

The laboratory management system shall cover all work carried out in permanent and mobile facilities

4.1.3

If personnel do perform activities other than testing, e.g. consulting, the organisation shall be defined in order to identify conflicts of interest

4.1.4

The laboratory shall have personnel and resources needed to carry out its duties according to its scope of accreditation

4.1.5.a

The laboratory shall have arrangements to avoid their personnel being influenced by internal or external pressure that could influence the quality of their work, e.g. paid by the number of tests performed

4.1.5.b

The laboratory shall have policy and procedures to avoid involvement in anything that could diminish its confidence

4.1.5.d

The laboratory shall define its organisation and management structure and the relationships in the organisation, e.g. by using an organisation chart.

4.1.5.e

The laboratory shall specify the responsibility, authority and interrelations of personnel that could influence the test results.

4.1.5.f

The laboratory shall provide supervision of testing staff, including trainees, by competent personnel.

4.1.5.g

The laboratory shall have technical management with the competence and resources needed for the overall responsibility for quality.

4.1.5.h

The laboratory shall appoint a Quality Manager.

4.1.5.i

The laboratory shall appoint deputies for key personnel.

4.1.5.j

The laboratory shall establish, implement and maintain an appropriate quality system for its activities.

4.2.1

The laboratory‟s quality system and policies shall be defined in a named Quality Manual.

4.2.2

The quality policy statement shall be signed by the management.

4.2.2.a-e

The Quality Manual shall make references to supporting document systems and outline the structure of the system.

4.2.3

The roles and responsibilities of the Technical Management and Quality Manager shall be defined.

4.2.4

Page 57 of 64

Requirement

Met

The laboratory shall include or make reference to external and internal documents in the quality system. All documents issued as part of the quality system shall be reviewed and approved. A master list (or equivalent) shall outline the documented quality system and be used to control the documents.

Not Met

N/A

Clause 4.3.1 4.3.2.1

The procedures adopted shall ensure that: -

Authorised documents are available where needed

-

Documents are periodically reviewed and if necessary

4.3.2.2

Replaced -

Invalid/obsolete documents are removed and marked as such

Quality system documents generated by the lab shall have an ID, page numbering, information pertaining to their validity and be authorised.

4.3.2.3

Changes to documents shall be reviewed and approved by the same functionary that approved the original.

4.3.3.1

Altered text in documents shall, if possible, be identified in the revised document.

4.3.3.2

If the laboratory allows for changes in documents to be made by hand, that procedure must be documented.

4.3.3.3

If changes to “electronic documents” are allowed, that procedure must be documented.

4.3.3.4

The laboratory shall have procedures for the review of requests, tenders and contracts. Records of reviews (including subcontractors) shall be maintained.

4.4 4.4.2 / 4.4.3

When the laboratory is going to use a subcontractor, it shall use a competent one.

4.5.1

The laboratory shall inform the client if the laboratory uses subcontractors

4.5.2

The laboratory shall maintain a register with its approved subcontractors and their competence.

4.5.4

The laboratory shall have a policy and procedure for the selection of purchasing of services and supplies and a procedure for the approval of bought goods.

4.6.1 / 4.6.2

The laboratory shall offer its clients a service.

4.7

The laboratory shall have a policy and procedure for complaints and a list of received complaints.

4.8

Page 58 of 64

Requirement

Met

Not Met

N/A

Clause

The laboratory shall have a policy and procedure to control non conforming work.

4.9.1

The laboratory shall monitor the effect of any corrective action.

4.10.4

When there is a doubt that the laboratory does not follow its own procedures, the laboratory shall perform an internal audit:

4.10.5

If there is the need or opportunity to perform a preventive action the laboratory shall do that.

4.11

The laboratory shall have a system for archiving of its documents.

4.12

Each observation etc. shall be documented in a way that makes it possible to identify the responsible person, and if of importance, the time.

4.12.2.1

Mistakes shall not be “tippexed” or erased, the wrong note shall be over-lined, dated and signed together with the correct result.

4.12.2.3

The laboratory shall have a procedure and a predetermined schedule for internal audits covering the whole quality system.

4.13

The management shall periodically, according to a schedule, perform management reviews.

4.14

The laboratory shall ensure the competence of its personnel. That competence shall be documented.

5.2.1

The laboratory shall have a policy about training and competence and a procedure to detect the need for training.

5.2.2

The laboratory shall have up-dated documented job descriptions.

5.2.4

The laboratory shall have documented authorisations for personnel to perform specific tasks, e.g. signing of test reports.

5.2.5

The laboratory must have facilities and an environment that does not affect the test result.

5.3.1

The laboratory shall, if necessary, monitor, control and record environmental conditions.

5.3.2

The laboratory shall use appropriate test methods and laboratory specific SOPs where needed.

5.4.1

The laboratory shall inform the client of the appropriate method to meet the clients‟ needs. The lab shall choose international, regional or national methods or methods published in reputable organizational or technical journals or methods proposed by the manufacturer of the equipment used.

5.4.2

Non standard methods shall be validated before use.

5.4.4 / 5.4.5.2

When calibrating, the uncertainty shall be estimated.

5.4.6.1

Page 59 of 64

Requirement

Met

Not Met

N/A

Clause

Testing laboratories shall have procedures to estimate its uncertainty.

5.4.6.2

Calculations and data transfers shall be checked periodically.

5.4.7.1

The laboratory shall validate its own developed software and have procedures to protect data stored electronically, e.g. log in / user name and back ups.

5.4.7.2

The laboratory shall have the equipment needed to perform the tests.

5.5.1

Used equipment shall have the quality needed and shall be appropriately calibrated.

5.5.2

The equipment shall be operated by competent staff; instructions shall be available where needed.

5.5.3

Each item of equipment shall be identified and properly documented.

5.5.4 / 5.5.5

Equipment suspected to function incorrectly shall be prevented from being used

5.5.7

When intermediate checks (verifications) are needed to guarantee the function of a piece of equipment, those checks shall be performed according to a defined plan and procedure.

5.5.10

The laboratory shall have a programme for calibration of its equipment.

5.6.1

When necessary the testing laboratory shall have traceability to SI units for the physical factors in question.

5.6.2.2

The laboratory shall have a programme and procedures for the calibration of its reference standards.

5.6.3.1

Reference material shall be traceable to SI units or to certified reference material where possible.

5.6.3.2

The laboratory shall have procedures to ensure the calibration status of its standards and reference material.

5.6.3.3

The laboratory shall have procedures for the handling of its standards and reference materials.

5.6.3.4

If the laboratory carries out sampling, it shall have a plan and procedure for that.

5.7

The laboratory shall have procedures for receipt, transport, pre-treatment labelling and identification etc. of test items. If any abnormality is detected upon receipt of a test item, this shall be recorded and noted in the test report.

Page 60 of 64

5.8.1 / 5.8.2 5.8.3 / 5.10.3.1.a / 5.10.3.2.e

Requirement

Met

Not Met

N/A

Clause

The laboratory shall have plans and procedures for monitoring the validity of its results. Systems shall be used in order to detect trends, e.g. the use of statistical evaluations.

5.9

The result shall be presented in a calibration certificate or test report fulfilling the requirements in ISO 17025.

5.10

If opinions and interpretations are included in the report, the basis of that must be explained and clearly separated from the result.

5.10.5

If subcontractors have been used that must be clear from the report.

5.10.6

In the case of electronic transmission of results, this must be agreed (in writing) with the client and the requirements in ISO 17025 must be met.

5.10.7

Additions or corrections to a test report can only be made by a new document with an explanation.

5.10.9

Page 61 of 64

APPENDIX 5: SAMPLE ACCREDITATION ACTION PLAN ACTION PLAN FOR THE PERIOD: JANUARY 2011 TO DECEMBER 2011 GOAL: ORGANISATION NAME: Objectives:

Date prepared: Prepared by: Activities

1) 2) 3)

Responsible

Indicator of success

Deadline

Quality manager and Lab management

Approved available copies of the quality manual and the quality policy statement

15-1- 2011

Area of Improvement Organisation and Management Clause 4.1

e.g. Outcome 1 –

Revision of the Quality Manual and the quality policy statement and have them approved by the lab director

1.1 1.2 1.3 Outcome 2 2.1 2.2 2.3 Outcome 3 3.1 3.2 3.3 Outcome 4 4.1 4.2 4.3 4.4 

Add more rows as necessary

Page 62 of 64

Schedule (weeks or months)

APPENDIX 6 EXAMPLE OF A FORM FOR THE WITNESSING OF TESTING Name of Laboratory: Department Name: Auditor name:

Date of Audit:

(PLEASE COMPLETE ONE FORM PER TEST WITNESS) 1. Type of sample 2. Name of method, calibration process or operation being evaluated and brief description. 3. Name of person observed 4. Qualifications/ Experience 5. Detailed comments, observations and/or reference. Indicate what was demonstrated and/or talked through as applicable. (Use reverse if necessary)

Additional / General Comments This space may also be used to expand on comments in specific sections

5.1 Validation data for method / technique / procedure

6. Non conformities / conclusions

6.1 Overall impression of conduct of method / technique / procedure

Signature Assessor/Auditor Signature of person observed

Date

Page 63 of 64

APPENDIX 7 REFERENCE DOCUMENTS Preparing For Accreditation: Guidance on Implementation of Quality Management Systems (QMS) For Medical Laboratories. A QMS “How To” Guide. Edition 1 December 2006 Strengthening Of Medical Laboratory Services In The Caribbean. A CARIFORUM Project Funded by the European Union (CAREC EU Project) Estimation of uncertainty of measurement in Medical Laboratories, 12th September 2006. European Diagnostic Manufacturers Association Position paper. www.edma-ivd.be M3003 The Expression of Uncertainty and Confidence in Measurement, Edition 2 August 2006 United Kingdom Accreditation Service www.ukas.com Uncertainty of Measurement in Clinical www.ifcc.org/ejifcc/vol13no4/130401006

Microbiology.

Fuentes-Arderiu

X,

eJIFCC

vol

13

no.

International Laboratory Accreditation Cooperation (ILAC) Documents www.ilac.org ILAC G2 Traceability of Measurement ILAC G9 Guidelines for the Selection and Use of Certified Reference Materials ILAC G13 Guidelines for the requirements for the Competence of Providers of Proficiency Testing Schemes Quality Management Standards ISO/IEC 17025:2005 General requirements for the competence of testing and calibration laboratories ISO/IEC 17043

Conformity Assessment – General Requirements for Proficiency testing

ISO/IEC Guide 2:2004

Standardisation and related activities – General Vocabulary

ISO/IEC 17000

Conformity Assessment - General Vocabulary

ISO 19011:2002

Guidelines for quality and/or environmental management system auditing

ISO/TR 10013:2001

Guidelines for Quality Management System documentation

ISO 10002: 2004

Quality Management – Customer Satisfaction – Guidelines for complaints handling in organisations

ISO 9000

Quality Management Systems – Fundamentals and vocabulary

ISO 9001

Quality Management Systems – Requirements

Page 64 of 64

4.