Home
HOME ARCHIVE SEARCH TABLE OF CONTENTS

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Services
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
PubMed
Right arrow PubMed Citation
The Gerontologist 47:480-489 (2007)
© 2007 The Gerontological Society of America

Comparing Staffing Levels in the Online Survey Certification and Reporting (OSCAR) System With the Medicaid Cost Report Data: Are Differences Systematic?

Bita A. Kash, PhD1, Catherine Hawes, PhD1 and Charles D. Phillips, PhD1

Correspondence: Address correspondence to Bita A. Kash, PhD, MBA, Department of Health Policy and Management, School of Rural Public Health, Texas A&M University Health Science Center, TAMU 1266, College Station, TX 77843. E-mail: bakash{at}srph.tamhsc.edu


    Abstract
 TOP
 Abstract
 Methods
 Results
 Discussion
 References
 
Purpose: This study had two goals: (a) to assess the validity of the Online Survey Certification and Reporting (OSCAR) staffing data by comparing them to staffing measures from audited Medicaid Cost Reports and (b) to identify systematic differences between facilities that over-report or underreport staffing in the OSCAR. Design and Methods: We merged the 2002 Texas Nursing Facility Cost Report, the OSCAR for Texas facilities surveyed in 2002, and the 2003 Area Resource File. We eliminated outliers in the OSCAR using three decision rules, resulting in a final sample size of 941 of the total of 1,017 non-hospital-based facilities. We compared OSCAR and Medicaid Cost Report staffing measures for three staff types. We examined differences between facilities that over-reported or underreported staffing levels in the OSCAR by using logistic regression. Results: Average staffing levels were higher in the OSCAR than in the Medicaid Cost Report data. The two sets of measures exhibited correlations ranging between 0.5 and 0.6. For-profit and larger facilities consistently over-reported registered nurse staffing levels. Factors associated with increased odds of over-reporting licensed vocational nursing or certified nursing assistant staffing were lower Medicare or Medicaid censuses and less market competition. Facility characteristics associated with over-reporting were consistent across different levels of over-reporting. Underreporting was much less prevalent. Implications: Certain types of facilities consistently over-report staffing levels. These reporting errors will affect the validity of consumer information systems, regulatory activities, and health services research results, particularly research using OSCAR data to examine the relationship between staffing and quality. Results call for a more accurate reporting system.

Key Words: Nursing home staffing • OSCAR • Medicaid Cost Reports • RN staffing levels


Nursing homes in the United States have a history of inadequate staffing levels, despite ongoing government efforts, such as adopting minimum staffing standards, instituting state-based staffing enhancement programs, and making staffing information available to the public (Harrington, 2005). Despite this long-standing concern about the adequacy of staffing levels, there is doubt about the accuracy of currently available federal staffing data, which is gathered in the annual survey of nursing homes that participate in the Medicare and Medicaid programs (Bostick, Rantz, Flesner, & Riggs, 2006). These staffing data are reported in the Online Survey Certification and Reporting (OSCAR) system, which is the only uniform data source available for all nursing homes that participate in the Medicare or Medicaid programs, nearly 98% of all U.S. facilities (Centers for Medicare & Medicaid Services [CMS], 2001; General Accountability Office, 2002; Mor, 2005; Straker, 1999).

Recently, CMS (2001) published a two-phase report that addressed the impact of staffing levels on quality and also stressed the need for more accurate measures of staffing levels in nursing homes. OSCAR staffing measures cover only the 2 weeks prior to the annual certification survey. This short time span, and the possibility that homes may "staff up" when they believe a survey is imminent, have made the validity of the OSCAR staffing data open to question. Despite these potential reporting and measurement issues, researchers have continued to use staffing measures from the OSCAR and the Medicare/Medicaid Automated Certification System, the precursor of the OSCAR, to examine the relationship between staffing and numerous quality indicators.

The potential problems with the OSCAR staffing data and their use in research might be one explanation for the sometimes inconsistent results regarding the relationship between staffing and various quality indicators, a factor that may have undermined the strength of this relationship (CMS, 2001). Thus, it is both timely and necessary to examine the accuracy of the OSCAR staffing measures, as well as the nature or pattern of any inaccuracies. If there are inaccuracies in the OSCAR staffing data, it is important to determine whether there is a systematic pattern of over- or underreporting of staffing levels, either by facility type or by staffing type.

In this study, we examined both the accuracy of OSCAR staffing data and whether there were systematic differences in reporting. To determine the accuracy of the OSCAR data and to identify any patterns of under- or over-reporting, we used what is widely considered a more accurate source of staffing data: audited Medicaid Cost Reports. The Cost Reports differentiate facility spending on registered nurses (RNs), licensed vocational nurses (LVNs), and certified nursing assistants (CNAs). These data are subjected to audit by the Medicaid agency, and facilities face the risk of stiff penalties for fraud if they over-report and would lose money if they underreport. Thus, we viewed the Cost Report as the best possible data source against which to evaluate the accuracy of OSCAR staffing data.

Despite the acknowledged need for accurate and reliable staffing data, experts have criticized the only uniform national data source available on nursing home staffing. Although information about facility characteristics, quality-of-care indicators, and resident case-mix measures has been widely used and validated for the OSCAR and Minimum Data Set data sources (Fries et al., 1994; Hawes et al., 1995; Zimmerman et al., 1995), the validity of staffing data derived from the OSCAR has been an ongoing concern (CMS, 2000; General Accountability Office, 2002). This accuracy issue is believed to be related to the nature of the reporting and collection process and the lack of an auditing component. OSCAR's staffing levels are based on each facility's self-reported hours of staffing by staff type for the 14 days immediately preceding the annual survey. Facility administrative staff usually report the OSCAR staffing levels, and the data are not typically subjected to any editing or independent validation by the survey teams (CMS, 2001). Despite its concerns about the accuracy of the staffing data from the OSCAR, CMS has been making OSCAR staffing data publicly available for all nursing homes since January 2003 on its Web site, commonly known as Nursing Home Compare (CMS, 2002). Recently, the National Citizens' Coalition for Nursing Home Reform (2004) stressed the importance of nurse staffing levels as the one identifier of a "good nursing home" while recommending that CMS adopt a new, audited data collection system based on payroll and invoices.

Despite the relatively heavy reliance on OSCAR staffing data in the policy and research arenas and despite the ongoing discussion about the validity of OSCAR staffing data, only three studies have actually examined the comparability of the data set with other data sources, such as Medicaid Cost Report, payroll, and survey data. One study compared OSCAR staffing data to those reported to the Ohio Department of Health via an annual survey of nursing homes (Straker, 1999). This study concluded that, in general, the OSCAR showed higher numbers of staff hours than those reported to the Department of Health. The study was based on 1995 OSCAR and 1995 and 1997 Ohio Department of Health data. The study had a limited sample size due to difficulties with matching facilities from the OSCAR with the Ohio Department of Health data, which resulted in the exclusion of more than 400 of the 900 facilities from the study.

A more recent study of OSCAR staffing data focused on RNs and physician extenders, used 1997 data, and compared OSCAR staffing levels with results from another self-reported survey of 327 New York nursing homes. This study concluded that the OSCAR data were useful in exploring relationships between staffing and quality indicators, but that they were not sufficiently reliable for the purpose of developing minimum staffing policy (Feng, Katz, Intrator, Karuza, & Mor, 2005). Many studies that have associated higher staffing with higher quality of care have also mentioned the potential limitations related to the accuracy of the OSCAR staffing measures (Akinci & Krolikowski, 2005; McCue, Mark, & Harless, 2003; Zhang & Grabowski, 2004). Therefore, more studies now choose to restrict the study sample of nursing homes in their analysis to one state in order to apply staffing measures derived from Medicaid Cost Reports (Harrington & Swan, 2003; Rantz et al., 2004; Schnelle et al., 2004).

The third and most comprehensive study was performed by CMS and included three states (New York, Ohio, and Texas). This study compared OSCAR staffing with both payroll data (for Ohio) and Medicaid Cost Report data (for New York and Texas). Results from this study suggested little evidence of widespread staffing up before the OSCAR assessment period (i.e., the date of the survey). The study did find that the lowest staffed facilities had an average reported increase of 15% in staffing hours during the OSCAR assessment period (CMS, 2001). The report concluded that staffing data from the Cost Reports were more reliable than OSCAR staffing data but that the differences were relatively small. This CMS study was based on 1997–1998 data for a relatively large sample of nursing homes. Unfortunately, the study did not further examine the lowest staffed facilities that had over-reported staffing levels during the OSCAR survey. All three studies speculated about the reasons for any observed inaccuracies and slightly higher OSCAR staffing levels. According to these three studies, misreporting of staffing hours in the OSCAR was most likely associated with the nature of the staffing question in the OSCAR (such as the wording of the question about the number of hours worked in the past 14-day period) and the process of data collection during the OSCAR assessment period. However, it is unlikely that errors associated with question wording would produce systematic differences across facilities or staffing types.

Today, more studies use staffing measures reported to the state Medicaid agencies because of the perceived higher validity of these Cost Reports (CMS, 2001). Researchers believe this data source provides more accurate staffing measures, summarizing information from a whole year of payroll data—instead of the last 14 days before the assessment date in the OSCAR. The Medicaid Cost Report is also subject to additional correcting and auditing processes not used in the survey process associated with the OSCAR data. Facilities that are identified as over-reporting staffing levels in the Cost Report are potentially subject to a more involved audit and allegations of Medicaid fraud and related sanctions for misreporting of data (CMS, 2001). Therefore, we assumed that Medicaid Cost Report data on staffing would be more accurate than the OSCAR staffing measures. These data offer the opportunity to examine staffing level differences between the self-reported OSCAR and the corrected and audited Medicaid Cost Report for a state with a large population of nursing homes and a rigorous cost reporting process.

As noted, the purpose of this article is to determine whether there were significant differences between OSCAR and Medicaid Cost Report staffing data for Texas nursing homes. Furthermore, the study attempted to identify specific and consistent characteristics of facilities that over-report or underreport staffing levels for RNs, LVNs, and CNAs in the OSCAR system. This approach allowed us to determine whether inaccurate staff reporting during the OSCAR annual survey period was random or actually a systematic error either among some types of nursing facilities or for some types of staff. Results from this study can inform research and policy about the types of facilities that have a strong tendency for inaccurate reporting of staffing levels in the OSCAR process.


    Methods
 TOP
 Abstract
 Methods
 Results
 Discussion
 References
 
Data Sources
We used three data sources for this study of nursing home staffing: (a) OSCAR data for Texas nursing homes that completed an annual survey in 2002, (b) the 2002 audited and corrected Texas Nursing Facility Medicaid Cost Report, and (c) the 2003 Area Resource File (ARF) for rural and urban information based on 2000 census data.

The OSCAR is a data set maintained by CMS and is a compilation of all of the items collected during the annual survey of nursing facilities that participate in the Medicare or Medicaid programs. The OSCAR data provide information on facility characteristics such as size, occupancy, ownership type, and chain membership. It also provides researchers and the CMS Nursing Home Compare Web site with a count of citations for failing to meet minimum federal standards (deficiencies) and other information that is intended to help consumers make informed decisions (Harrington, O'Meara, Collier, & Schnelle, 2003). However, there are several limitations to the OSCAR data, including (a) the limited number of quality indicators, (b) the questionable validity of staffing data, and (c) the lack of data on staff turnover (Mukamel & Spector, 2003; Straker, 1999).

We compared staffing data from the OSCAR for all non-hospital-based Texas nursing homes to the 2002 Medicaid Cost Report data. Each nursing facility participating in the Medicaid program is required to submit financial and statistical information on a standard Cost Report form to the Texas Department of Aging and Disability Services each year. The Cost Report provides information on more than 500 reported cost and expense items including information about facility type, staffing levels, and staff turnover. Because it is audited and providers are subject to penalties for misreporting, the Cost Report has been identified as providing more accurate information on staffing (IOM, 2004). Providers report staffing data in paid hours and total salaries and wages for the entire reporting year for each direct care staff type (RN, LVN, and CNA), other resident care staff (medication and restorative aides, social workers, and dieticians), as well as administrative staff.

The ARF is a county-level health resources information database. It contains measures of economic activity and socioeconomic characteristics in each county. We used ARF data in the multivariate modeling in this research in order to determine whether there was any effect of location (rurality) on the accuracy of self-reported facility staffing data in the OSCAR.

Data Merging and Cleaning
First we merged the 2002 Medicaid Cost Report, which included the population of non-hospital-based nursing homes in Texas (1,017 facilities), and the 2003 ARF based on county name. Next we merged this new data set with the Medicare provider number and Medicaid contract number, using a crosswalk file supplied by the Texas Department of Aging and Disability Services. Finally, we merged the OSCAR data set for Texas nursing homes with the composite Cost Report and ARF data set based on the crosswalk that allowed us to link the OSCAR provider numbers with the Medicaid contract numbers as facility identifiers. In this process, we dropped all merged observations that did not match perfectly between the source files and the merged files. The final merged data set included a total of 1,003 observations, meaning only 14 facilities were lost in the merging process.

In cleaning the OSCAR data set, we used a combination of the logical decision rules originally developed by Harrington and colleagues and the recommendations from a CMS report to Congress (CMS, 2001; Harrington, Carrillo, & Mercado-Scott, 2005). Whereas Harrington and colleagues recommended excluding all facilities that fall into the lowest 1% and highest 2% distribution in staffing levels, the CMS decision rules did not automatically exclude facilities with very low and very high staffing in the OSCAR. We eliminated observations from the OSCAR data that seemed unreliable based on the following criteria: (a) We dropped any facility with more than 24 hr of total average staffing hours (RNs, LVNs, and CNAs combined); (b) we eliminated facilities with zero licensed staff; and (c) we excluded facilities in the highest 1% and the lowest 1% distribution in OSCAR staffing levels (based on the average total direct care staff hours per resident day). After this process of elimination, the number of nursing homes in the data set decreased from 1,003 to 941, a reduction of 62 observations from the merged data set. These decision rules removed what might have been unreasonable data from the OSCAR database (i.e., data most likely associated with errors in reporting or recording). In essence, this means that the analyses presented here constitute a "best case" of OSCAR data because of the removal of some "obvious" errors in the OSCAR database from the analysis.

Dependent Variables
We examined four direct care staffing variables: (a) RN hours per resident day (hprd), (b) LVN hprd, (c) CNA hprd, and (d) total direct care staff hprd (sum of the preceding three staff types). We calculated staffing levels from the OSCAR data source by dividing the total hours worked on a daily basis by the total number of residents for each staff type, as clearly defined in the OSCAR. We calculated the measures of staffing levels in the Cost Report by dividing the reported paid hours for each staff type, which included employed and contracted staff, by the total days of service for all beds for the reported year. This calculation of hprd followed the formula supplied to us by the Texas Department of Aging and Disability Services.

Because facilities report agency staff in the OSCAR assessment survey, both data sources should include both employed and contracted staff. Therefore, we included both employed (salaried) staff as well as contracted (agency) staff in the staffing hours for each staff type in the Medicaid Cost Report. We used the two sets of four staffing variables to perform comparisons of means and correlations between the OSCAR and Medicaid Cost Report. Next we examined Medicaid Cost Report staffing level distributions and compared them to the OSCAR staffing level for each facility. We developed dummy variables to identify facilities that over-reported or underreported staffing in the OSCAR at 105%, 110%, and 120% above and at 95%, 90%, and 80% below the reported Medicaid staffing levels for RNs, LVNs, CNAs, and total staffing levels. This procedure allowed us to identify facilities that inaccurately reported OSCAR staffing levels, using three different cutoff points for over-reporting and underreporting of staffing levels.

Independent Variables
The independent variables of interest for this study were organizational characteristics and market factors that might have an impact on facilities' reporting of staffing data in the OSCAR. Organizational variables included ownership type (1 = for-profit and 0 = not-for-profit facilities), multifacility system or chain membership (1 = chain facility and 0 = non-chain facility), payer mix (percent Medicare and Medicaid resident days), facility size (number of beds), average occupancy rate, and the average case-mix index score for each facility. The case-mix index is a composite measure of resident acuity at the facility level, based on the average Texas Index of Level of Effort, a case-mix classification system similar to the Resource Utilization Groups (RUGS-III) that the Medicare program uses (Fries et al., 1994). The two market variables included in the logistic regressions predicting the probability of inaccurate reporting were (a) urban influence code, a county-level ARF variable that ranged from 1 (most urban) to 9 (most rural); and (b) the Herfindahl index, a capacity-based measure of market concentration ranging from 0 (least concentrated and most competitive market) to 1 (most concentrated and least competitive market).

Analysis Strategy
First we examined descriptive statistics for all staffing variables from the OSCAR and the Medicaid Cost Report as well as all independent variables. Next we developed the dummy variables identifying facilities that inaccurately reported OSCAR staffing levels in comparison to Medicaid staffing distributions. We created and examined summary statistics for all variables, including the dummy variables.

Next we examined differences in mean level of staffing reported in the OSCAR versus the Medicaid Cost Report and correlation coefficients between the two staffing measures. We calculated and examined both Pearson and Spearman correlation coefficients. Because we expected that the OSCAR staffing levels would be greatly affected by outliers, even after the data-cleaning process, we also calculated the Spearman correlation coefficient (correlation of the ranks of the staffing variables). We produced and examined graphical representations of this expected linear relationship by plotting the OSCAR staffing levels and Medicaid staffing levels, producing a regression line and plotting the estimated error terms around zero.

Furthermore, we examined differences in organizational characteristics and market factors for facilities that reported staffing data inaccurately (over-reported or underreported staffing levels) in the OSCAR using bivariate statistical tests. Finally, we estimated logistic regression models by using six levels of inaccuracy in the OSCAR: three levels of over-reporting and three levels of underreporting. This process of sensitivity analysis helped us (a) determine whether there were consistent over-reporting inaccuracies in the OSCAR staffing levels associated with specific facility characteristics as well as (b) explore the possibility of systematic underreporting. We applied the Huber White sandwich estimator of variances in all of the logistic models in order to correct for the potential selection bias due to grouping of nursing homes at the county level (Wooldridge, 2003).


    Results
 TOP
 Abstract
 Methods
 Results
 Discussion
 References
 
Based on the summary statistics presented in Table 1, mean staffing levels from the OSCAR and the Medicaid Cost Report were quite similar. In general, total staffing levels (RNs, LVNs, and CNAs combined) were about 3% higher in the OSCAR data compared to in the Medicaid Cost Report.


View this table:
[in this window]
[in a new window]

 
Table 1. Descriptive Statistics for Staffing Levels, Facility, and Market Characteristics.

 
Despite the similarity in mean staffing levels, there were significant differences between the OSCAR and Medicaid Cost Report data for specific staff types. The OSCAR staffing reports showed 38% higher mean levels of staffing for RNs (0.36 hprd vs 0.26 hprd) and 4% higher CNA staffing (1.97 hprd vs 1.90 hprd) than the Medicaid cost reports. The average LVN hprd in the OSCAR were about 9% lower than the average LVN hprd in the Medicaid Cost Report. The underreporting of LVN hours would seem to be a complement to the over-reporting of RN hours, because in practice in facilities, these two job categories have some measure of substitutability.

We examined the percentage of facilities that over-reported OSCAR staffing levels at 105%, 110%, and 120% above the Medicaid Cost Report staffing levels. The highest prevalence of over-reporting was associated with RN hours (65%–77% of facilities over-reported RN hours) and CNA hours (29%–40% of facilities over-reported CNA hours). About 7% of facilities over-reported LVN levels at 120% above Medicaid Cost Report, and 19% of facilities over-reported LVN levels at 105% above the Medicaid Cost Report cutoff.

The examination of the percentage of facilities that underreported staffing levels in the OSCAR system revealed that only about 7% of all facilities underreported RN hours, about 27% underreported LVN hours, and only 6% of facilities underreported CNA hours at 80% of the Medicaid Cost Report levels. After moving the cutoff point for underreporting staffing levels closer to the Medicaid Cost Report levels (at 95% of Medicaid Cost Report staffing levels), we observed a somewhat higher prevalence of underreporting in the OSCAR. Even after this reexamination, only 16% of facilities underreported RN staffing levels and only about one third of facilities underreported total staffing levels at 95% of the Medicaid Cost Report cutoff.

About 82% of all nursing homes in this study were for-profit facilities, and 65% belonged to a multifacility chain. The average Medicaid census (proportion of resident days paid by Medicaid) was about 71%, whereas the average Medicare census was about 6%, with the remaining 23% being private pay or some other payer, such as the Veterans Administration or private long-term-care insurance. The average facility bed size was 111, and the average occupancy rate was 75%. The average Herfindahl index for the nursing homes in the sample was at 0.24, indicating relatively moderate to high market concentration, and the average level of urban influence was at 3.38 (relatively urban). We observed a relatively large variability in Medicaid and Medicare census, facility size, case mix, occupancy rates, and market factors.

Table 2 presents the correlation between OSCAR staffing levels and the staffing measures retrieved from the Medicaid Cost Report.


View this table:
[in this window]
[in a new window]

 
Table 2. Correlation Between Medicaid Cost Report and OSCAR Staffing Measures.

 
For total RN hprd, the Pearson correlation coefficient was 0.53, whereas the Spearman correlation coefficient was only 0.48. By contrast, for LVN hprd, the Pearson correlation coefficient was relatively high at 0.63 and even higher when accounting for outliers using the Spearman correlation (0.71). The two CNA staffing measures showed relatively low Pearson correlation at 0.52 and slightly higher Spearman correlation at 0.61. Overall, we found an average correlation of 0.54 to 0.62 for the average direct care staffing levels from the two data sources. All correlation coefficients (both Pearson and Spearman) for all staff types were statistically significant at p <.01.

When compared to the results from a study performed by Feng and colleagues (2005), who reported relatively weak Spearman correlation coefficients (0.57 for RNs, 0.45 for LVNs, 0.29 for CNAs, and 0.27 for total staff), our results indicated relatively higher correlations between the OSCAR and Medicaid Cost Report staffing levels. We only found slightly weaker correlation for RNs (0.48) and much higher correlations for LVNs, CNAs, and total staff (0.71, 0.61, and 0.62, respectively). The relatively high correlation results in our study might have been related to the strict decision rules we used to clean the OSCAR data set by omitting extreme staffing level values.

We further examined the correlation between the two staffing measures graphically by regressing the Medicaid Cost Report staffing measures for each staff type on the corresponding OSCAR staffing measures (see Figures 1 and 2). We examined the nature of the correlation between OSCAR staffing data and Medicaid Cost Report staffing levels by using a series of scatter plots, fitted single regression lines, and residual plots, with OSCAR staffing as the predictor and Medicaid Cost Report staffing as the predicted variable. The graphical analysis of the regressions indicated increasing error variance at higher OSCAR staffing levels.


Figure 01
View larger version (30K):
[in this window]
[in a new window]

 
Figure 1. Predicting Medicaid cost report RN hprd with OSCAR RN hprd. RN = registered nurse; hprd = hours per resident day; OSCAR = Online Survey Certification and Reporting; CR = Cost Report; CI = cost interval

 

Figure 02
View larger version (10K):
[in this window]
[in a new window]

 
Figure 2. Residuals versus fitted values plot for registered nurse hours per resident day regression

 
For example, RN hprd in facilities with lower reported OSCAR staffing levels seemed to be relatively close to the Medicaid Cost Report levels (residuals grouped around zero). However, the confidence interval widened at higher reported OSCAR staffing levels. We also observed similar residual plots an increasing error variance as reported LVN and CNA staffing levels in the OSCAR went up, producing a distinct fan-shaped residual plot.

Table 3 presents bivariate comparisons of facility and market-level characteristics between nursing homes that over-reported staffing in the OSCAR (for total direct care staff at 105% above the Medicaid Cost report staffing levels) and those that did not over-report. We do not report summary statistics for facilities that underreported staffing due to the low prevalence rates of underreporting activity in the OSCAR (generally less than 5% of facilities underreported total staffing levels in the OSCAR) and the consistent results mirroring the result of over-reporting behavior. Therefore, we concentrate on results from the more meaningful analysis of over-reporting facilities.


View this table:
[in this window]
[in a new window]

 
Table 3. Characteristics of Over-Reporting and Not Over-Reporting Facilities (Based on Average Total Staff Levels at the 105% Above Medicaid Cost Report Cut-Off).

 
Statistically significant differences in reported staff hours were mostly associated with CNA hprd. On average, the facilities that over-reported total staffing levels in the OSCAR had actual CNA levels (based on Medicaid Cost Report) that were 0.13 hprd lower than those of facilities that did not over-report. The level of market concentration was about 0.28 for over-reporting facilities compared to 0.22 for those facilities that did not over-report staffing in the OSCAR. Therefore, over-reporting facilities seemed to be located mostly in counties with lower levels of competition. Over-reporters also had lower actual staffing hours (based on the Medicaid Cost Report measures), fewer resident days covered by government payors (Medicaid and Medicare), and lower occupancy rates.

The multivariate logistic regression models predicted the odds for being an over-reporting facility at the three different over-reporting cutoff points for each staff type (presented in Table 4, Models A–C). We performed a similar analysis using logistic regression models to predict the odds of being an underreporting facility at three different levels of underreporting by staff type. Because this analysis did not produce meaningful results, such as significant predictors of underreporting behavior, we do not report results for underreporting models and conclude that underreporting activity was not systematic. The logistic regressions predicting underreporting did, however, produce one insightful result: One independent variable, for-profit status, was consistently associated with 44% to 62% reduced odds of underreporting total staffing levels in the OSCAR.


View this table:
[in this window]
[in a new window]

 
Table 4. Logistic Regression Results.

 
Factors associated with over-reporting RN staffing for all three logistic models at the different cutoff points were for-profit ownership and larger nursing home size (higher number of beds). Results from the Model A logistic regression analysis (OSCAR staffing levels at 105% above Medicaid Cost Report staffing levels) demonstrated that for-profit ownership increased the odds of over-reporting RN staffing levels by 123%. Furthermore, for each additional percentage increase in Medicaid resident days, the odds of over-reporting decreased by 1%. For each additional bed in a nursing home, the odds of over-reporting RN staffing hours increased by 1%. Being located in a more rural county (measured by the urban influence code using a 1–9 scale) decreased the odds of over-reporting RN staffing levels by 8%. The odds of over-reporting RN hours were reduced for nursing homes located in more rural counties only in Models A and B (using the 105% and 110% above Medicaid Cost Report staffing cutoffs). Based on logistic regression Model C (at 120% above the Medicaid Cost report staffing cutoff), the odds of over-reporting RN hours were significantly increased by about 1% for each additional percentage point increase in occupancy rate.

Results from the logistic regressions predicting the likelihood of over-reporting LVN hours in the OSCAR were consistent for all three logistic regressions at the three different cutoff points. Medicare and Medicaid census reduced the odds of over-reporting LVN staffing. Results from the Model A logistic regression analysis (at 105% above Medicaid Cost Report staffing) showed that for each percentage point increase in Medicare days and Medicaid days, the odds of over-reporting LVN staffing levels were reduced by 7% and 2%, respectively. None of the other facility characteristics or market factors contributed significantly to the odds of over-reporting LVN levels. Results from the odds of underreporting LVN staffing models mirrored these findings: Higher Medicare and Medicaid census was usually associated with higher odds of underreporting LVN levels.

The three CNA prediction models produced consistent results for Medicaid census and occupancy rates as significant factors associated with reduced odds of over-reporting. Models A and B also showed significant results for Medicare census (negative association), rural location (negative association), and market concentration (positive association) as significant factors affecting the odds of over-reporting CNA levels. Results from the Model A logistic regression demonstrated that for each percentage point increase in Medicare resident days and Medicaid days, the odds of over-reporting CNA staffing levels were reduced by 4% and 2%, respectively. For each percentage increase in occupancy rate, the odds of over-reporting CNA hours were reduced by 1%. Being located in more rural settings (for one unit increase in the urban influence code) reduced the odds of over-reporting CNA staffing levels by 8%. Being located in more concentrated markets (less competition in the county in which the facility was located) significantly increased the odds of over-reporting CNA staffing levels in Models A and B. Results from the sensitivity analysis confirmed the significant effect of market concentration on CNA staffing level differences between the OSCAR and Medicaid Cost Report. Higher market concentration (less competition) in the market was associated with reduced odds of underreporting CNA levels in the OSCAR in two of the three logistic regression models.

Based on logistic regression Model A, for each percentage point increase in Medicare and Medicaid days, the odds of over-reporting total direct care staffing levels were reduced by 3% and 2%, respectively. Each percentage point increase in occupancy rate decreased the odds of over-reporting total average staffing levels by 1%. Being located in more concentrated markets (less competition in the county) increased the odds of over-reporting total average staffing hours significantly. In general, the total average direct care staff levels were more likely to be over-reported by facilities with lower Medicare and Medicaid census, those with lower occupancy rates, and those located in counties with lower levels of competition. Again, the effect size and significance of Medicare and Medicaid census on the likelihood of over-reporting total overage direct care staffing levels were consistent across all three cutoff points. In the underreporting models, we found consistent results only for for-profit status.


    Discussion
 TOP
 Abstract
 Methods
 Results
 Discussion
 References
 
Based on the results of this analysis of nursing home staffing measures, we believe that staffing levels in the OSCAR are not as reliable as staffing measures retrieved from the Medicaid Cost Reports. As expected, the OSCAR data provided as part of the survey process showed higher levels of staffing, especially for RNs and CNAs. In this study we were able to identify important facility and market-level characteristics that seemed to contribute to facilities' tendency to over-report staffing levels in the OSCAR. The factors contributing to the likelihood of over-reporting were low Medicare and Medicaid census, high market concentration, and for-profit ownership.

The RN results for ownership and facility size tell a relatively simple story. Both for-profits and larger homes tend to have lower hours per resident day of care from RNs than they report in the OSCAR and may artificially inflate their report of staffing levels for survey periods in order to avoid being cited for deficiencies in RN staffing. The meaning of the findings concerning the impact of the Medicaid and Medicare censuses is somewhat more opaque. It may simply be that those homes taking more of the public's dollars may expect higher levels of scrutiny from surveyors and that this expectation deters some up-staffing or over-reporting. Alternatively, it may be that facilities with higher proportions of private-pay residents feel pressure to report higher staffing levels in order to look better to these consumers. It will obviously take further research to clarify the dynamics underlying these relationships.

These results have implications for both policy and research. The results strongly imply that one of the most frequently used indicators in studies of nursing home quality contains systematic error or bias. The effect of this bias is most easily illustrated when one thinks of two nursing homes that generate identical outcomes. They also appear to have identical staffing levels. However, one home incorrectly reports its staffing as higher than is the case. This reporting bias distorts any observed relationship between these reported staffing levels and quality indicators. This implies that quality studies using OSCAR data may systematically underestimate the strength of the relationship between staffing and quality in American nursing homes.

Policy makers have used OSCAR staffing data to help the public differentiate among nursing homes. The results in Table 4 may be useful to regulatory staff. For example, experts may decide that staffing levels in for-profit facilities and in homes with lower Medicare or Medicaid census warrant more serious scrutiny during the survey process.

Finally, and arguably most importantly, policy makers are now under relatively constant pressure from groups composed of advocates, health professionals, and researchers to develop more stringent staffing requirements for nursing homes. Policy makers want their decisions to be evidence based. All of this gives increased weight to the potentially elevated risk of Type II error in staffing studies using OSCAR data. Bias, in this instance, becomes much more than an academic concern. Research using OSCAR data that has found a relationship between higher staffing levels and better quality may, as noted, underestimate the impact of staffing levels, particularly of RNs, on quality. This potential underestimation of the value of staffing is particularly critical as policy makers consider the cost of various policies aimed at increasing the staffing levels in nursing homes.

Because the OSCAR is an important national nursing home data set with comprehensive facility- and resident-level information used for an array of important activities, we can only agree with other analysts that it is now crucial to improve the process of reporting, collecting, and verifying data on staffing in the OSCAR (Bostick et al., 2006). Failing that, the OSCAR may need to be replaced with an alternative database tied more closely to payroll and cost report data.


    Footnotes
 
This article was supported by Grant 1 R36 HS016229-01 from the Agency for Healthcare Research and Quality. Its contents are solely our own responsibility and do not necessarily represent the official views of the Agency for Healthcare Research and Quality or the Texas A&M University System Health Science Center. Back

1 Department of Health Policy and Management, School of Rural Public Health, Texas A&M University Health Science Center, College Station. Back

Decision Editor: Linda S. Noelker, PhD

Received for publication December 14, 2006. Accepted for publication March 23, 2007.


    References
 TOP
 Abstract
 Methods
 Results
 Discussion
 References
 




This article has been cited by other articles:


Home page
Journal of Applied GerontologyHome page
N. G. Castle
State Differences and Facility Differences in Nursing Home Staff Turnover
Journal of Applied Gerontology, November 1, 2008; 27(5): 609 - 630.
[Abstract] [PDF]


Home page
Health Aff (Millwood)Home page
D. G. Stevenson and D. C. Grabowski
Private Equity Investment And Nursing Home Care: Is It A Big Deal?
Health Aff., September 1, 2008; 27(5): 1399 - 1408.
[Abstract] [Full Text] [PDF]


Home page
Journal of Applied GerontologyHome page
N. G. Castle
Nursing Home Caregiver Staffing Levels and Quality of Care: A Literature Review
Journal of Applied Gerontology, August 1, 2008; 27(4): 375 - 405.
[Abstract] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Services
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
PubMed
Right arrow PubMed Citation


HOME ARCHIVE SEARCH TABLE OF CONTENTS