|
|
ValidityThe content validity is a central criterion for indicators, which in any case must be evaluated in an assessment. An unsatisfactory validity can lead to a systematic distortion of the health care service assessment, so that the quality indicator is no longer meaningful. An important prerequisite for a high validity is a good reliability. With the determination of what the quality indicator claims to measure, one has to refer exclusively to the health care aspect which is specifically depicted in the quality indicator. A comprehensive statement about the health care quality of all diagnostic and therapeutic processes of a provider using only one quality indicator is desirable, but may not be practical. Therefore, quality indicators always refer to individual and concrete health care aspects which should always correspond to the relevance criteria (see criterion “meaning for patients and health care delivery system of the quality characteristic captured by the quality indicator”). In common usage, frequently the statement “the indicator is valid” is equated with a generally high methodological quality. Also the term “face validity” is likely an expression of the fact that the indicator appears to be of high quality on first impression. However, an objective measure for a generally high methodological quality does not exist. The methodological quality can only be examined step by step through an application of the individual criteria. Therefore, in the QUALIFY instrument the term validity is strictly used in a narrow and clearly defined and practically verifiable way as mentioned in the definition. The causes for unsatisfactory validity are many. As an example, a BQS quality indicator from the measurement area “breast surgery” with “discovered malignancies with open biopsies” (procedural year 2006) shall be mentioned: this indicator is supposed to capture the number of open biopsies for diagnostic reasons in the denominator. In reality, the quality indicator captures all open procedures without previous histology, because open biopsies for diagnostic reasons cannot be identified through German procedure codes or BQS documentation. However, not every open procedure without previous histology is a diagnostic procedure and even open procedures with previous histology can in individual cases be of a diagnostic nature (e.g. histological findings of the first operation in discrepancy to a mammography finding). This means specifically that the indicator does not measure exactly what it is supposed to measure and therefore has a limited content validity. This poor validity was resolved in BQS procedural year 2007 by capturing open biopsies for diagnostic reasons in BQS documentation. Besides the content validity, further aspects of validity are specified in the literature: construct validity and criterion-related validity. Construct validity is defined by the Agency for Healthcare Research and Quality (AHRQ) as: “the indicator should be related to other indicators or measures intended to measure the same or related aspects of quality.“ (www.ahrq.gov/clinic/epcsums/hcupqisum.htm). Criterion-related validity is present when the measurements are highly correlated with another construct valid measurement (the criterion) (http://arbeitsblaetter.stangl-taller.at/FORSCHUNGSMETHODEN/Guetekriterien.shtml
Overall, it is to be noted that in the literature, especially in regard to the criterion validity, many different and partially contradictory definitions prevail. Therefore the definitions mentioned here, are only an excerpt. Definition Core Statement Information Base for the Assessment Assessment Process A detailed process description can be found in Appendix 1. Assessment Stages
|
). Therefore, for the examination of the construct and criterion-related validity, a gold standard is required, which is not available. Also, internationally the examination of these criteria is abandoned in practice. Thus, the assessment of validity is done in QUALIFY with a focus on the content validity.