by Jerry Everhart – Vice President, Quality and Measurement Technologies
JTI Systems, Inc.
This article was printed in CAL LAB Magazine, issue January/February 1997.
A high degree of competitiveness in the national and international manufacturing markets has created a demand for process controls that build quality into products, rather than relying on inspection to sort out costly rejects. This article describes a process measurement assurance program (PMAP) that determines and controls measurement errors (uncertainties) as the product is produced. The results of this program coupled with production statistical process control (SPC) determine the product values with a known certainty. When the measurement error is determined along with the product variation, the product error is known and further inspection activities are greatly minimized or eliminated.
Need for Process Measurement Assurance Program (PMAP)Competition for markets has caused manufacturers to question their reliance on inspections and reinspections as a means of achieving quality in the products they build.1 Increasingly, manufacturers want to produce quality in the products as they are manufactured, rather than relying on inspections to eliminate lesser quality products. Manufacturers have used Statistical Process Control (SPC) to determine and control the variations in products during the manufacturing process. This allows continued improvement of processes and products. However, the SPC of products alone does not ensure that the measurement of the product is correct. In fact, measurements taken on products have hidden errors. These measurement errors are “instrument errors” (including standards or master errors), operator errors, and environmental influences on measurements. (Fig. 1)Often these measurement errors are not defined or measured, resulting in false accounts of product variability, and influencing final products. Many of these measurement errors can produce a bias or systematic error in product measurements. Rather than identifying and controlling these errors, the manufacturer often adjusts processes to meet design specifications. This can ultimately result in product deviations from design specifications. Because these errors cause unknown variations, confidence in values is lost, creating the need for reinspection of products.Typically, the product is inspected by the manufacturing personnel, then reinspected by the quality department. The quality department usually checks the product with a different measuring instrument, different person, and in a different area.Many times a quality audit is performed after final inspection. These repetitive inspections result from lack of knowledge and confidence in the initial measurement. In view of this, it is of major importance that these measurement errors (instrument, standards, operator, procedure, and environment) are evaluated, recognized, and controlled as the products are produced. This concept of parallel control of products, and control of the measurement systems (through PMAP) provides total and continuous control of the final product, and results in higher product quality at lower cost.Another approach commonly used is a repeatability and reproduceability (R&R) study. R&R studies attempt to determine measurement errors by making repeated measurements of product parts on a specific measurement device and comparing the results to the product variation to determine measurement influence.
Figure 1. Measurement errors, to some degree, are in product values.
R&R studies also require comparison of those measurements with measurements of the same group of parts by other operators to determine operator influence. This approach is time-consuming, costly and provides limited results. Confidence in the results is affected by the quality of the parts chosen. Measurement uncertainty can not be determined without standards or parts that have a known uncertainty. Also, the results are derived from a periodic (noncontinual) approach and must be repeated when possible change in measurement error is to be evaluated.Traditional Calibrations Measurement error in a production measurement system is rectified only in a limited way by the calibration laboratory, as the gage or measurement instrument and its associated working standards are periodically returned to the standards and calibration laboratory for recalibration. Calibrations are usually performed in a controlled environment using calibration procedures and specifications that are written by the instrument manufacturer for specific conditions. After calibration, the instrument is returned to the production floor and used to check product under conditions that may differ considerably from the calibration laboratory. This measurement procedure is often a major source of measurement error. Operators, who may not be as well trained as those in the calibration laboratory, set up and use the instrument, introducing additional measurement errors.Another possible source of measurement error has been introduced with the use of computers and software programs for operating measurement equipment and determining results. Generally, software quality assurance programs (SQAs) are in place to assure software quality; however, these programs only validate that no changes have occurred in the software code.These conventional approaches to calibration of measurement systems are in an appraisal mode of operation, not a preventive mode. This approach may be inadequate in a competitive high-technology production environment where measurements are made by state-of-the-art equipment.
Furthermore, it is important to address what happens when an instrument becomes defective or drifts out of its predetermined uncertainties during its calibration period. The products measured prior to the instrument’s recalibration are potentially out of specification. Investigation of product measurements and related errors that influence product quality are required. If the actual process measurement error is to be determined, all calibration errors and uncertainties must be calculated, including the operator and environmental influences.
These errors are usually difficult to determine, and pinpointing the actual time of measurement system failure is often impossible. It is no longer acceptable to rely on post production inspection of product to segregate costly defects caused by the measurement process.
PMAP As a Continuous Calibration Method
A Process Measurement Assurance Program is a continual calibration concept for measuring equipment. This approach of establishing and controlling measurement uncertainties eliminates the need of issuing discrepancy reports on measurement equipment. In addition, this approach eliminates the investigation of product that is potentially out of specification because the performance of the measurement system has changed between calibration periods. One objective of PMAP is to determine, monitor, continually control, and improve the measurement capability of the measuring system in its operating environment.
Initially, the capability of the measurement system is established by standards laboratory personnel or measurement experts in the production environment after proper calibration adjustments have been made on the equipment. This determination provides the expected baseline performance level (reference limits) for the measurement system. Using this baseline, PMAP can evaluate the production operator’s influence on measurement capability.This is a powerful tool which can be used to measure training needs for individual operators. Both the systematic error and random errors of the total measurement system are established, monitored, and controlled by PMAP, while the product is manufactured. Control of the measurement system, as it is being used, provides control of the error in the product values resulting from the measurement system. By comparing measurement performance to pre-established statistical reference limits, manufacturers can detect shifts in calibration of the measurement system prior to a manufacturing step. This methodology is applicable to, and should be used in, the calibration processes of calibration laboratories to assure the correct calibration values and uncertainties.As errors in the product measurements are established and statistically examined, the error in the product values can be established with a determined uncertainty. This PMAP calibration method provides confidence in the total measurement system and product values, thus allowing quality to be built into the product and eliminating or greatly reducing the need to reinspect the product. This concept of calibration bridges the gap between the standards (metrology) laboratory and the production operations.
It is important to realize that SPC methods determine only product variation, but do not determine the errors in product values. PMAP is used to determine systematic and random errors in the values assigned to the product by the measurement system and to maintain the calibration status of the measurement system on a continual basis.
Concept of Operation
The concept of PMAP is based on the use of a control standard referred to by the National Institute of Standards and Technology (NIST) as a check standard,2 to statistically examine the capability of the measurement system. The control standard is chosen or manufactured to represent the product, or a specific feature of the product, for the determination of the systematic error and random variations of the measurement process.
Selection of the control standard value requires consideration of the range of the measurement instrument used in the measurement process. Proper selection can evaluate and assure calibrated values of the standards used. Considerations are given to the stability of the control standard value and the amount of uncertainty of the control standard’s calibrated value. Measurement errors are determined by making measurements on the control standard with the measurement system, using the same procedures that are used when measuring the product. Initial measurements are made by standards laboratory personnel to establish confidence limits (reference limits) that can be used as a baseline to assure that the calibration is maintained. Establishing these limits often leads to immediate improvement of the measurement process.
The calibration laboratory or quality department personnel (measurement experts), will periodically make control measurements with the control standard using the same procedure that is used to measure the product. The frequency of these control measurements is based on the measurement system’s stability, which also affects calibration periods in the traditional calibration method. If the traditional calibration period, for example, should be 12 months, then enough standard reference measurements would be required to re-establish the standard reference limits for the next calibration period. It should be noted that actual physical re-calibration or adjustments may not be required using PMAP. If 20 data points are sufficient to re-establish the reference limits over the 52 weeks (12 months), then standard control measurements should be performed at least once every two weeks in order to predict calibration limits for the next calibration period.
After initial reference limits are established and the control measurement interval for future reference measurements is determined, the control standard is measured by production personnel using the same procedures and operations that will be used to measure the product. This control measurement is made and recorded prior to manufacturing and measuring the product, for example, at the beginning of the workday. The total measurement system is used for the control measurement. This may include a computer and its software. The measurement is checked against the pre-established confidence limits (reference limits) to assure that the system is still in calibration prior to measuring the product. A control measurement of the standard is repeated at the end of a production period (for example half of a workday) or anytime the product variations indicate a possible measurement problem. This measurement closes the loop on the measurement system’s performance before the product leaves that stage of manufacturing.
The PMAP control measurements are recorded and stored according to the data and time of each measurement. Statistical determination of the upper and lower production control limits (confidence limits) is used to determine the error in the assigned values of the product for that specific production time period. Although PMAP measurements and calculations can be hand charted and calculated, computer assistance can achieve a considerable time and cost savings. This PMAP concept of continual calibration, in addition to preventing re-inspection of the product, allows for extending or often eliminating routine calibration periods. PMAP tracks and assures the calibration of the measuring instrument, master (span adjustment) standards, check standards (used to minimize error linearity), and the control standard. Changes in measurement performance because of equipment, standards, or environmental influence on the measurement system are reflected by a change in the control measurement results, as are operator influences on the measurement process. Additionally, error limits in product values are determined from PMAP calculations: thus product is made and measured by production personnel with a known uncertainty in the product values. This additional confidence in product value allows the product to be moved to its next stage of manufacturing without time-consuming, expensive re-inspection.
PMAP Control Charting
PMAP control charts, unlike typical SPC control charts, are designed to determine more than the random variations of a measurement system. PMAP determines the systematic error as well as the random error. The utilization of the control standard in the PMAP control chart becomes the reference by which the systematic and random errors are established.Systematic Error
Systematic error of a measurement system can be caused by nonlinearity in measurement equipment, errors in associated standards, or the measurement process. This systematic error produces a bias in measurement results which can be determined using a control standard in the measurement system. PMAP determines this systematic error of a measurement system by subtracting the certified value of the control standard from the mean (x_) of the control measurements. Because traditional measurement processes ignore this systematic error, product re-inspections are common. In many cases, the variability of the product may be acceptable and in control, but the product values may be out of product specifications because of the systematic error of the measurement system (Figure 2).
Figure 2. the systematic error of a measurement system as shown by PMAP control chart.
Random Variability
Random variability of a measurement system is the inability of the measurement system to repeat the same measurement results. The random variability of a measurement system can result from any or all of the previous sources of error, with the exception of a non-linearity error, which results in a systematic error. The random variability of a measurement system is determined by analyzing the PMAP control standard measurements to establish the variability around the mean (x_) of the measurements. The variability is described by the conventional standard deviation method. The control limits can be expressed by a coverage factor (k) which is usually 2 or 3.For the purpose of illustration we will chose 3 standard deviations where 99.7% of the measurements are expected to fall within the three sigma probability. A change in the measurement system’s ability to repeat a measurement within limits is
reflected each time the standard deviation is recalculated at recalibration intervals.PMAP continually checks for any significant change in the production measurement system’s ability to repeat a measurement by immediately plotting each control standard measurement against the pre-established 3s reference confidence limits before the product is manufactured (Fig. 3).
PMAP Results
Measurement System Error
PMAP defines the total measurement system error as the result of all systematic and random influences on the measurement system’s ability to determine the control standard’s certified value. The error is determined by subtracting the certified value of the control standard along with its affiliated uncertainty from the upper and from the lower 3s reference limits. This error is calculated from the control measurements made by calibration laboratory, quality personnel or measurement experts.
Figure 3. The random variation of a measurement system as shown by PMAP control chart. The system’s performance is continually evaluated by plotting each control measurement before manufacturing the product.
Measurement system error limits are defined as follows:Positive Margin of Error = U3s – CS (4)Negative Margin of Error = L3s – CS (5)where:U3s = upper 3s reference limitsL3s = lower 3s reference limitsCS = PMAP control standard certified valueThese error determinations are based on the control standard measurements made by standards laboratory or quality personnel and determine the measurement system’s calibration status. At recalibration intervals, the systematic error and random variability are re-calculated to determine new reference limits until the next calibration interval. The “F” test is used to determine if the random variability has significantly changed.3,4The “t” test is used to determine if the systematic error is within expected range of the previous error. The F test and t test are defined as follows:Note: The appropriate degrees of freedom (based on number of readings) are applied to both the F and t test formulas.
If these tests reveal significant changes in the measurement systems performance, re-adjustment or recalibration may be required. If both tests are in control, these reference limits are used for the next calibration interval (Figure 4).
Figure 4. The total measurement system error is a result of all systematic and random influences as shown by PMAP control chart.
Error in Product Values
The confidence in product values comes from establishing the error in the product values. This confidence is expressed as possible error or uncertainty that may be present in each product value. This product uncertainty is established by analyzing the production personnel’s PMAP measurements of the control standard. The production control standard measurements are analyzed over the specific time period required to manufacture and measure the product. The PMAP control standard is subtracted from the upper and the lower production control 3s limits to determine product error limits. The certified uncertainty of the control standard is applied to the error limits to establish the possible error or possible uncertainty in the product values for that specific lot of product. Refer to equations 8 and 9.
Product error limits are defined as follows:
Upper Product Error Limit = U3s – CS + Un (8)
Lower Product Error Limit = L3s – CS – Un (9)
where:
U3s = upper PMAP production 3s limits
L3s = lower PMAP production 3s limits
CS = PMAP control standard certified value
Un = Control standard uncertainty
Uncertainty in Product Values
The upper and lower product error limits (equations 8 and 9) are a valuable tool that allow production personnel to adjust the process specification limits to guarantee that the product is within design specifications. The product specification limits can be shifted by subtracting the lower error limit (9) from the upper product design specification and by adding the upper error limit (8) to the lower product design specification. The results of these new process specification limits compensate for total measurement error and guarantee acceptable product without re-inspection. With knowledge of random uncertainties and systematic uncertainties, the appropriate square root of the sum of squares for random and algebraic calculations of systematic uncertainties can be performed.
Summary
The Process Measurement Assurance Program (PMAP) is a method of continual calibration that puts quality process controls in the calibration of measurement systems. The application of PMAP is important to the production environment but it should be noted that these concepts apply to any process where the results are determined by a measurement system. PMAP provides a system calibration that evaluates the performance of the total measurement system in the manner in which it is used. The benefits of this calibration method are preventive, not appraisal. Any out of control situations due to equipment, environment, personnel, or standards are detected immediately. These detection’s reduce production rejects.
It is also important to realize that the implementation of a PMAP requires a thorough knowledge of the specific measuring equipment and the direction of a metrologist who can select control standards and procedures that meet the requirements described in this report.
The results of PMAP implementation provide for the manufacturing of a product with values of a known certainty without repeated inspection. This statistical approach to calibration provides continual calibration and continued improvements to measurement processes.
References
Deming, W. E., 1982, “Quality, Productivity, and Competitive Position”, Massachusetts Institute of Technology, Center for Advanced Engineering Study, p 19-22
Croarkin, Carroll 1977, “Measurement Assurance Programs Part II: Development and Implementation” Special Publication 676-II, National Bureau of Standards, p 19-20.
Rickmers, A. D. and Todd, H. N., 1967, “Statistics An Introduction”, McGraw-Hill, Inc , p 21-22, 66-68
Varner, R. N. and Raybold, R. C.. 1980, “National Bureau of Standards Mass Calibration Computer Software” Technical Note 1127, National Bureau of Standards, p 14-23.
About the Author
Jerry Everhart is a metrologist with more than 25 years experience in standards and calibration technology. He has conducted seminars on process measurement assurance programs for the American Society of Quality Control, National Conference of Standards Laboratories, Ohio Quality and Productivity, Westinghouse, EG&G, Rockwell International, General Electric, Mason & Hanger-Silas Mason, Inc., Martin Marietta, Allied Signal, Sandia National Laboratories, 3M Corporation, the National Institute of Standards and Technology, and the Department of Energy.
To Contact:
Jerry L. Everhart
JTI Systems, Inc.
P.O. Box 45536
Rio Rancho, NM 87174
Acknowledgements
A big Thanks to Carol Singer and CAL LAB Magazine for their assistance. To CAL LAB Mag
FREE ARTICLE!
We’ll gladly mail the published version of this article, click here. Be sure to include your name, mailing address, and the name of the article.
To learn more about how JTI can help your business, contact us at 505-710-4999