|
|
Practical Application of the QUALIFY Instrument – Selection of Quality Indicators for Mandatory Public Reporting of German HospitalsMandate and ObjectivesBQS quality indicators were developed in order to be used for internal quality management and for anonymous external quality comparison. A use for public reporting of hospital results was so far not intended. BQS was commissioned on May 16, 2006, by the Federal Joint Committee (Gemeinsamer Bundesausschuss) to identify suitable indicators to inform patients and the insured. This analysis should enable the Federal Joint Committee to decide, whether and in which form the results of external hospital measurement can be mandatory and uniformly presented in the structured quality reports according to the German Social Code § 137 SGB V. The public reporting of quality measures in medicine and patient care using uniform quality indicators is largely new for the German health care system, but it is the focus of increasing public interest. It is therefore important that quality indicators are chosen, for which the danger of erroneous decisions and false interpretations is as small as possible. BQS Quality Indicators Realization of the Mandate of the Federal Joint Committee Execution of the Analysis and ResultsCustomization of the QUALIFY Instrument for the Mandate 14 criteria were identified which had special relevance for the particular question and for which a suitable background information for the assessment could be provided. (Table 5).
In Table 6 it is shown which criteria were not used for the assessment and the reasons for not using them are displayed. Table 6: Reasons for non-appliance of criteria
Acceptance of the Suitability of an Indicator For the criterion indicator evidence suitability was determined by a classification of “medium” “high” or “very high”. For the other 13 criteria the suitability was determined by an assessment result of “applies” or “rather applies”. Preparation of the Assessment The assessment of the individual criterion was prepared by sending out relevant information before the sessions and during the sessions they were explained and supplemented with special facilitation material.
The BQS specialty groups first identified the most relevant measurement areas,which means those with the largest number of cases. 12 of 24 areas were prioritized in this way for the analysis. Table 7: Prioritized measurement areas
In the next step, indicators were identified in which methodological limitations with individual criteria were recognized in the past (e.g. outcome indicators with limited risk adjustment). Sentinel event indicators were also excluded from the assessment, since due to the rarity of the observed events a limited ability to discriminate was known for these indicators.
Execution and Results of the Assessment The Federal Joint Committee used them as a basis for the resolution to require publication of indicator results in the quality report of the hospitals. For 27 of these indicators which were evaluated as suitable without reservations, the requirement exists for publication in the quality report. For the four quality indicators evaluated as suitable from the measurement area “community acquired pneumonia”, the Federal Joint Committee issued only a recommendation for publication. The BQS specialty group Pneumonia had recommended that outcomes from this measurement area should be reported only beginning with the data for the year 2007. The rationale for this recommendation was that after the introduction of this measurement area in 2005, the experiences and agreements from the Structured Dialogue? in 2006 could not be fully realized during 2006. The Federal Joint Commission followed this recommendation. The results of the assessment of these indicators with the help of the QUALIFY instrument is published for all 14 criteria on the website www.bqs-qualitaetsindikatoren.de Experiences in the Practical Application of the QUALIFY Instrument.Practical Suitability of the QUALIFY Instrument In practice, in the first sessions initial reservations existed among specialty group members because of the high degree of formalization. A discussion time of 2-4 hours per specialty group was necessary to explain the instrument and the application. Afterwards, a familiarity with the assessment procedures developed relatively quickly within the specialty groups. The methodological assessment of quality indicators with the help of the QUALIFY instrument requires 2-5 hours of consulting by the expert committees, as well as substantial effort to compile and compare the background information for the assessment. Overall, the instrument has proven to be time consuming but practically manageable. Benefit of the QUALIFY Instrument This high complexity could be reduced with a clear structure and uniform definitions of the QUALIFY instrument. The arrangement into practically manageable single criteria made a structured discussion and assessment possible. The uniformity of the assessment ensures that the same requirements in quality indicators are applied to all measurement areas (BQS quality indicator sets). All assessment results can be displayed transparently. This enables an open scientific discussion of the instrument as well as the assessment results of the of individual quality indicators. Thus a public discussion as to whether the “right” quality indicators were selected can occur in a structured way with the help of the criteria. Further Development of the QUALIFY InstrumentPractical experiences have shown that in the future refined moderation tools can be used with individual criteria to make complex assessment situations more understandable and therefore facilitate a faster assessment. E.g. in the criterion ability of statistical differentiation the visualization of the three step methodology could be improved after the first experiences. Unavoidable redundancies exist between individual criteria. In individual cases it should be checked whether the criteria cannot be more clearly separated from each other. For example, the criterion importance also captures aspects of utilization, so that the criterion benefit could be more clearly focused on the practical benefit of the indicator. For the criterion reliability, it has been shown that an improved information base should be strived for. In the future, during the development of new indicators an improved information base should be created using specific pilot tests (test-retest and inter-rater reliability). The scientific publication of the QUALIFY instrument and its pilot application will enable a discussion which could be the basis for further methodological advancement of the instrument. |
). In these care, 180 BQS quality indicators were available, whose results were analyzed for all hospitals. These indicators were included in the analysis.