CROWNWeb: Producing data to measure performance

The work on which this publication is based was performed under Contract Number HHSM-500-2011-00157G, titled “CROWNWeb Outreach, Communication, and Training,” funded by the Centers for Medicare & Medicaid Services, Department of Health and Human Services. The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. government. The author assumes full responsibility for the accuracy and completeness of the ideas presented. The author welcomes comments on the ideas presented; please send comments to CRAFT@projectcrownweb.org.

Publication Number: FL-OCT-2013OCTT2-10-1329

 

Since its inception, the Centers for Medicare & Medicaid Services’ Quality Measures Project (formerly known as the End-Stage Renal Disease Clinical Performance Measures Project), has served as one of the renal community’s primary means of assessing and reporting the quality of dialysis services provided under the Medicare program. Data obtained and documented as part of the Quality Measures Project have enabled ESRD stakeholders to gauge patient care efforts and outcomes by allowing a comparison of clinical performance results at the facility, ESRD Network, and national levels. Established as a result of Section 4558(b) of the Balanced Budget Act (BBA) of 1997, the Quality Measures Project captures important patient characteristics and clinical measures—specifically in the areas of adequacy of dialysis, anemia management, and vascular access—that have been used by stakeholders such as the ESRD Networks and their Medical Review Boards to analyze dialysis treatment trends and identify areas for improvement.

Historically, the Quality Measures Project followed a highly manual and at times rigorous paper-based data acquisition process that took CMS upwards of 24 months to obtain and present findings. Determined to modernize the ESRD community’s ability to access clinical performance results, and to obtain a broader overview of patient care efforts, CMS transitioned the method used to collect Quality Measures data elements into an electronic format through the agency’s CROWNWeb data-collection system.

In mid-2013, CMS made strides in its utilization of CROWNWeb as a means to document facilities’ clinical performance results by initiating the clinical data submission closure and analysis process for all reported May 2012 to December 2012 and early 2013 clinical data. The conversion to CROWNWeb includes changes to CMS’ original Quality Measures data attainment procedure, and allows facilities to access results directly from the system’s Reports screen upon availability. However, the Project continues to follow federal guidelines that help capture factors that may impact patient outcomes.

Clinical closures and data extraction

On June 28, CMS closed the submission period for all 2012 clinical data. This decision, which was made after a close evaluation of the clinical submission efforts of thousands of dialysis facilities using CROWNWeb, allowed the agency to accomplish a major milestone in its aim to transition the Quality Measures data management and retrieval efforts to an electronic format.

To assist with the shift of Quality Measures reporting to CROWNWeb and to help ensure the accuracy of the data provided by facilities, CMS and its CROWNWeb data support contractors follow multiple error-checking processes that include the comparison of CROWNWeb’s reported measurements against previous Quality Measures summaries, as well as an examination of all reported data to evaluate if they fall within CMS’ pre-determined clinical range for each field. Data that fall outside of the expected clinical range are reviewed to identify potential error types and sources. Measurements that fall outside of the pre-determined range with no recognizable error source are considered abnormal and set aside for continued analysis. If a measurement is determined to be erroneous (for example, a different data type inadvertently entered in a treatment field), that specific record is eliminated from the findings.

Prior to making the Quality Measures data available via CROWNWeb, CMS and its clinical data analysis contractor follow a multi-step procedure that begins with the closure of the clinical months (see Table 1).

Table 1. CROWNWeb Quality Measures data extraction and reporting process.

Unlike the previous data collection process, which utilized samples of patients, Quality Measures data reporting through CROWNWeb uses the total reported population. After all existing raw clinical data are compiled, the clinical data analysis contractor employs a layered “analysis plan” that includes the study of descriptive statistics (i.e., data reporting frequencies, extreme observations, and acknowledgement of any missing datasets), the grouping of results (i.e., monthly, ESRD Networks, states, and facilities), and the identification of specific quality calculations (i.e., eligibility, and inclusion/exclusion criteria). This “plan” encompasses two phases: Phase 1 focuses on the validity, verification, and accuracy of reported data, and Phase 2 centers on consistency and precision.

CMS allowed one year from the CROWNWeb National Release to begin the initial closure of clinical months and analysis of 2012 data, in order to provide adequate time for users to acclimate to the system as well as time to implement enhancements to CROWNWeb’s performance that supported data reporting requirements. However, CMS remains committed to closing the gap in time between receipt of Quality Measures data and distribution of the findings to the renal community. CMS plans to execute a monthly Quality Measures reporting cycle that allows users to generate summaries that reflect near real-time results, rather than statistics from a year prior. To support this effort, CMS identified submission deadlines for 2013 clinical data—beginning with three-month entry and closure periods for January 2013 to September 2013 clinical data (see Table 2).

Table 2. January 2013 to September 2013 CROWNWeb clinical closure months

 

Data reporting

CROWNWeb data entry occurs year-round. However, reports available in CROWNWeb follow the historic three-month reporting period for the ESRD CPM Hemodialysis Summary Report, six months for the ESRD CPM Peritoneal Dialysis Summary Report, and monthly for the Fistula First Summary Report. Depending on the selected summary, CROWNWeb users can generate Quality Measures reports that contain data for one, three, or six consecutive months, respectively.

Quality Measures reporting through CROWNWeb is in its infancy. Once the CROWNWeb Quality Measures data extraction and analysis process is complete and the data are uploaded into the CROWNWeb system, users can query the system for results—beginning with the July 2012 Quality Measures report for hemodialysis (three month collection window), and October 2012 for peritoneal dialysis (six month collection window). As facilities continue to enter their data into CROWNWeb, CMS’ ultimate plan is to gather the information to make the Quality Measures data results available via CROWNWeb within 90 days from the end of the collection cycle. Once data are available, authorized users can select a reporting month in CROWNWeb and the system will present the percentage of patients used to calculate each measurement based on the reporting period.

Quality measures findings

CMS utilized data submitted for more than 5,200 dialysis facilities during May 2012 to December 2012 to calculate CROWNWeb’s initial Quality Measures. These results included 16 measures originally approved by CMS based on the National Kidney Foundation’s Dialysis Outcome Quality Initiative (K/DOQI) clinical practice guidelines, but also incorporated changes applied to the Quality Measures Project in 2008 by CMS to capture additional factors that may impact patient outcomes. In all, data were analyzed for 26 measurements that focused on:

• Anemia management
• Adequacy (hemodialysis and peritoneal dialysis)
• Mineral metabolism
• Vascular access
• Influenza vaccination
• Patient education, perception of care, and quality of life
• Patient survival

 

Historically, to calculate and compare their outcomes based on a national average, ESRD Networks and facilities manually completed the ESRD Outcome Comparison Tool at the end of each ESRD Clinical Performance Measures Annual Report. With CROWNWeb, the system automatically displays the percentage of patients who fall within the measure threshold, along with a denominator that represents the total number of patients considered. Furthermore, CROWNWeb has built-in logic that allows the system to automatically group results based on patient affiliation with the selected facility, to help ensure that the patient care outcomes display for the correct dialysis provider. CROWNWeb links the findings to a unique clinic identifier known as the CROWN Facility ID. Visit http://m4.healio.com/~/media/nni/medicare/endstagerenaldisease/cpmproject/downloads/esrdphaseiiicpm04012008final.pdffor a complete list and description of the measurements used as part of the Quality Measures.

In nephrology, elements such as dialysis adequacy, anemia management, and mineral metabolism monitoring are key components in determining patients’ overall health and required treatment regimens. Additionally, access-related data assist the community in gauging the success of efforts to maximize or minimize the use of certain vascular access types, as well as in the surveillance of dysfunctions. ESRD Networks and facilities compare their outcomes based on the national average and recommendations, and implement various quality improvement projects that help identify root causes and develop best practices that will aid facilities with bettering their performance.

When reviewing Quality Measures results, reviewers should use the family of measures together to obtain a complete picture of care efforts. -by Jamila Seaton, MPH

More information

For more information on CROWNWeb and the system’s data reporting capabilities, visit the Project CROWNWeb website at www.projectcrownweb.org, or visit the Centers for Medicare & Medicaid Services CROWNWeb website at www.qualitynet.org and click on the ESRD tab. For information regarding the Quality Measures Project visit www.cms.gov/Medicare/End-Stage-Renal-Disease/CPMProject/index.html.