| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||||||||||
Correspondence: Address correspondence to Dr. Paul Stolee, Southwestern Ontario Regional Geriatric Program, 
St. Joseph's Health Care London, 801 Commissioners Road East, London, Ontario N6C 5J1, Canada. E-mail: paul.stolee{at}sjhc.london.on.ca
 |
Abstract |
|---|
 TOP Abstract MethodsResultsDiscussionReferences |
|---|
Key Words: Staff training • Outcomes • Evaluation • Continuing education • Long-term care
Training staff in the long-term care setting is a relatively new phenomenon. In the early 1980s, researchers in the United States were reporting that nursing assistants were providing up to 90% of resident care and receiving little or no training (Waxman, Carmen, & Berkenstock, 1984). The primary treatment model was based on a custodial model of care in which staff met the basic needs of residents. Burgio and Burgio (1990) noted, in an early review of the nursing assistant training literature, that the few articles published on this topic prior to 1987, with two notable exceptions, debated whether resources should even be invested in training these workers.
The paramount need for staff training was realized in the mid to late 1980s as the organization of long-term care underwent significant change. First, there was a shift in societal attitudes about health and aging toward support for a more therapeutic model of care for elderly people. The current custodial model began to give way to a new, more restorative or rehabilitative model of care. New learning and development was required to effect this change (Burgio & Scilley, 1994).
Second, because of the increase in the capacity for assisted living in the community, the elderly population coming into long-term care began to be those with the greatest disabilities and the most complex health needs. The Canadian Study of Health and Aging (CSHA) found, for example, that 56.7% of elderly residents of long-term care facilities have dementia (Canadian Study of Health and Aging Working Group, 1994) and that an additional 30.0% have cognitive impairment that does not meet current criteria for dementia (Graham et al., 1997). Other mental health problems are also common (Rovner, German, & Broadhead, 1990). The long-term care system needed to develop the capabilities to meet these challenges.
Third, the U.S. Omnibus Budget Reconciliation Act (OBRA) in 1987 mandated new regulations for long-term care. Before OBRA, regulations emphasized the physical plant. The new regulations focused on resident care, mandating the reduction and elimination of physical and chemical restraints and the development of individual care plans to ensure optimal functioning of each resident. To support these and other changes in the long-term care system, OBRA mandated an increase in the number of mandatory training hours for nursing assistants and required them to participate in regular performance assessments to show skill competency.
Although legislation similar to OBRA does not exist in Canada, the new model of care has been encouraged through large-scale initiatives such as the development of restorative care programs (Fitzgerald et al., 2001) and psychogeriatric education. These initiatives can be very resource intensive. For example, a 1998 educational initiative on psychogeriatric care provided training to staff in nearly all of the 500 long-term care facilities in Ontario, at a cost of over $1 million. Currently, this program is being extended over a 5-year period at a cost of $1.1 million per year (Ontario Ministry of Health and Long-Term Care, 1999).
In sum, in recent years, many educational programs have been undertaken or initiated in long-term care facilities. The effectiveness of this training and its long-term impact on resident care are unclear, as most initiatives do not include an evaluation component or are not formally evaluated over time (Beck, Ortigara, Mercer, & Shue, 1999). As Burgio and Burgio (1990) discovered in their review of the nursing assistant training literature, most published reports consist of discussion articles on the importance of training. Moreover, organizational factors have been identified as posing barriers to the evaluability of training (Johncox, 2000).
To date, to our knowledge, a comprehensive review of published studies evaluating training programs in the long-term care setting has not been done. Information about the sustained effectiveness of such programs is primarily anecdotal (Beck et al., 1999). What is generally known in other settings with respect to continuing education is that a challenge exists in the transfer of knowledge into practice. Learners gain new knowledge and acquire new skills but encounter difficulties when attempting to implement the new learning. It has been suggested that only 10–30% of training activity of any kind is actually transferred to ongoing performance (Broad, 1997).
Evidence from a variety of settings referred to in, for example, reviews of continuing medical education (CME) literature (Escovitz & Davis, 1990), as well as editorials and articles in medical, gerontological (Foner, 1995; Koeck, 1998; Osborn & Kotrady, 1994; Schnelle, Cruise, Rahman, & Ouslander, 1998), and nursing journals (Daley, 1997; Gifford, 1994; Scheller, 1993) suggest that organizational and system factors can affect the sustained application of knowledge gained from continuing education programs, as well as the adoption of innovations (Castle, 2001).
The uniqueness of the long-term care setting may present additional or different barriers or facilitators to the effective implementation of continuing education. The culture of long-term care is different from, for example, acute care, in that there is less emphasis and value on training and few incentives are present to encourage staff change or motivation (Burgio & Scilley, 1994). It has also been pointed out that long-term care facilities are highly regulated organizations and tend to be very concerned with labor costs. The various stakeholder groups often work with different goals (Schnelle et al., 1998).
Frail older persons are an undervalued societal group, and their caregivers are similarly devalued (Bond & Fiedler, 1998; Smith, 1998). For example, in contrast to other settings, Mass and colleagues found that 90% of all resident care is carried out by health care aides who have a high school education or less, receive little more than a minimum wage, obtain limited training, and receive minimal long-term benefits, recognition, or support for their physically and emotionally intensive work (Maas, Buckwalter, & Specht, 1996). Continuing education initiatives also have to accommodate a range of abilities of staff, different educational levels, and a mix of learning styles (Dixon, Adams, & Cullins, 1997). There are also likely to be cultural sensitivity issues to address. Many long-term care staff members, particularly in urban areas, are foreign born (Schnelle, McNees, Simmons, Agnew, & Crooks, 1993). All of these factors affect the ability of long-term care staff to implement new learning. Research and innovation in the long-term care setting has been minimal, limiting the possibilities for new learning and development.
|
Methods |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
Articles meeting the following criteria were retained for further analysis: (a) application was continuing education (i.e., excluded academic-based education); (b) the setting was long-term care, excluding veterans, geropsychiatric, or palliative care hospitals; (c) the long-term care setting was not combined in the data analysis with other settings (e.g., Baltes, Neumann, & Zank, 1994); (d) the primary focus of the study was staff training as opposed to programs of care where training comprised one component of implementation (e.g., Rovner, Steele, Shmuely, & Folstein, 1996); (e) the study included an evaluation component offering empirical data for conclusions; (f) the publication date of the study was between 1985 and 2001; and (g) the study was published in an English-language journal.
To examine the education intervention, we followed a classification system developed by Green, Kreuter, Deeds, and Partridge (1980) and modified by Davis, Thomson, Oxman, and Haynes (1992). In this system, educational intervention strategies are sorted by factors relevant to behavior change in health promotion. For this review, these included (a) predisposing factors, which involve primarily the communication or dissemination of information designed to modify an individual's knowledge, beliefs, or attitudes (e.g., lectures, written information, or video presentations); (b) enabling factors, which are conditions and resources within the environment that allow, or enable, an individual to implement new skills (e.g., modified work schedule, practice opportunities, policy or treatment guideline changes, or availability of treatment protocols or algorithms); and (c) reinforcing factors, which provide an individual with cues or reminders to implement new skills, or that reinforce the use of new skills (e.g., peer support, advice, and feedback). The interventions were classified as Type I, predisposing only; Type II, predisposing, plus enabling; Type III, predisposing, plus reinforcing; and Type IV, a combination of all three types of interventions.
Each of the studies reviewed was evaluated according to Kirkpatrick's (1994) schema of levels of evaluation developed in the late 1950s and 1960s to describe the types of evaluation used in studies to test the effectiveness of educational interventions. For many years, Kirkpatrick's model has been the most widely used framework for the evaluation of training programs. One survey of the human resources development and industrial psychology literature found that of articles that mentioned or described training evaluation models, more than three quarters of these included Kirkpatrick's model (Hilbert, Preskill, & Russ-Eft, 1997). Although Kirpatrick's classification system has come under scrutiny in recent years (Hilbert et al., 1997), we found it useful as a simple descriptive tool. Four levels of evaluation are classified: participant satisfaction with the educational training program, staff knowledge and attitudes, staff behavior, and resident outcomes. Evaluation at the level of outcomes is generally considered the most demanding.
To assess the scientific rigor of the methodology used in the evaluation, we followed the method developed by the Canadian Task Force on the Periodic Health Examination (1994). The key features used to evaluate the effectiveness of health care interventions include recommendations of graded strength, based on the quality of the evidence with the greatest weight on the features of the study design and analysis that tend to eliminate or minimize results. The quality of published evidence was categorized as evidence generated from properly randomized control trials (RCT), from well-designed controlled trials without randomization (quasi-experimental designs), and from descriptive studies or case reports. For grades of recommendations, studies were rated by using the following descriptions: good evidence to support recommendations of effectiveness; fair evidence to support recommendation of effectiveness; insufficient evidence to recommend for or against the effectiveness; fair evidence to support the recommendation of ineffectiveness; and good evidence to support the recommendation of ineffectiveness.
|
Results |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
summarizes the location of the study, the type and content of the intervention, the sample, level of evaluation, impact (immediate and follow-up sustainability), level of evidence, and grade of recommendation for effectiveness for the 48 studies reviewed.
|
|
|
|
|
Many of the studies, 19 in total, were concerned with resident mental health issues, and 10 evaluated chemical or physical restraint reduction. Of the remaining 19 studies, 4 focused on attitudes toward elderly people, 4 on continence, 3 on oral care, and 1 each on geriatric rehabilitation, attitudes toward dying, pressure sores, infection control, inhaler techniques, swallowing, diabetes, and drug use.
A major finding of this review is that the intervention type used in 35 of the 48 studies consisted solely of predisposing factors. This means that in almost three quarters of the studies in our sample, new knowledge was provided to staff without any enabling or reinforcing strategies in the form of organizational or system support or change to facilitate the transfer of the new knowledge or behavior to the workplace.
The new knowledge was usually given primarily in a training program format. The length of the training programs ranged from a 10-min educational session (O'Connell, Hewit, Lackner, Pasto, Wong, & Bishop, 1992) to 56 hr of seminars given in twenty-eight 2-hr sessions (Chartock, Nevins, Rzetelny, & Gilberto, 1988). The average length of training was approximately 4 hr offered in a series of ø-hr or 1-hr seminars on a weekly basis. The training methods described in our sample were all very similar, comprising some combination of handouts, audiovisual material, lectures, seminars, experiential learning, role play activities, and group discussion.
Thirteen studies in our sample did offer some organizational or system support along with the training program. Of these studies, two included enabling factors (Type II). In one study, nursing assistants were provided with bedside opportunities for learning the use of oral care aides (Pyle, Massie, & Nelson, 1998); in the other, written treatment protocols were provided (Schnelle, Newman, & Forgarty, 1990). Five studies used an intervention that included, in addition to a training program, clinical instructions and reminders or feedback to encourage behavior change (Type III: predisposing plus reinforcing factors). This was accomplished by having on-site consultation for the duration of the study (Evans, Strumpf, Allen-Taylor, Capezuti, Maislin, & Jacobsen, 1997; Feldt & Ryden, 1992; Kihlgren, Hallgren, Norberg, Brane, & Karlsson, 1990), and performance feedback (McCallion, Toseland, Lacey, & Banks, 1999; O'Connell et al., 1992).
Six of the 48 studies combined all three intervention types. Along with offering new knowledge, the interventions comprised elements to both enable and reinforce the practice of the new knowledge in order to promote and sustain the new learning. Several researchers developed staff–management systems that included self-monitoring and recording of activities, practice plans, treatment algorithms, and supervisory monitoring and feedback (Burgio, Engel, Hawkins, McCormick, Scheve, & Jones, 1990; Campbell, Knight, Benson, & Colling, 1991; Lekan-Rutledge, Palmer, & Belyea, 1998; Stein, Griffin, Taylor, Pichert, Brandt, & Ray, 2001; Stevens et al., 1998). Smyer, Brannon, and Cohn (1992, p. 327) developed a job redesign intervention that "attempts to change the nature of work itself by making it the focus of a guided staff group process."
The methodological designs of the evaluations were limited. Only 7 of the 48 studies used the most rigorous randomized controlled trial design, 19 used a quasi-experimental design without randomization, and 22 (almost half of the studies in the sample) used a descriptive, case study design. Good or fair evidence sufficient to support a recommendation that the educational intervention was effective was possible in less than half of our study sample. Specifically, 23 studies received a grade of A or B. Eight studies received a grade of D or E, indicating that good or fair evidence was provided to support a recommendation that the educational intervention was not effective. In the case of 17 studies, there was insufficient evidence to make a recommendation either way.
Weaknesses in the methodological approaches seriously affected about half of the studies. These studies typically had very small sample sizes, were nonrandom, did not include a control group, and had very low response rates and other threats to validity, including, for example, the use of self-reports for the evaluation process. Overall, the attrition rate in most of the studies was very large, often reaching as high as 50% or more of the sample at the immediate postintervention evaluation (e.g., Bradley, Siddique, & Dufton, 1995; Campbell et al., 1991; Cohen-Mansfield, Werner, Culpeper, & Barkley, 1997; Pillemer & Hudson, 1993; Teri, Baer, Orr, & Reifler, 1991). These findings suggest that the impact of the educational interventions must be interpreted with caution and also illustrate some of the evaluation challenges in this context.
Participant satisfaction was evaluated in 2 studies, staff knowledge, beliefs, or attitudes in 23 studies, staff behavior in 25 studies, and resident outcomes in 14 studies. The majority of studies (n = 32) evaluated only one of these outcome variables, 13 studies evaluated two of these variables, and 3 studies evaluated three of the variables. No studies included all four levels of evaluation. As the majority of studies evaluated staff behavior and resident outcomes, we could say that according to Kirk-patrick's schema, the more demanding levels of evaluation have generally been used to test the effectiveness of the educational intervention.
Of additional importance in the determination of the effectiveness of an educational intervention is whether the learning can be transferred into practice and sustained over time. In almost two thirds (n = 31) of the studies in our sample, researchers evaluated changes in staff knowledge or staff behavior or resident outcomes immediately after staff completed the training program, and they did not do a follow-up evaluation. Without follow-up data there is no evidence that the new learning was sustained overtime. Of the 17 studies that conducted a follow-up, 11 studies concluded that changes or improvement in outcomes were sustained at the follow-up, 4 concluded that changes were not sustained, and 2 studies that found no changes at the posttest also found no change at the follow-up. Staff behaviors were sustained in 8 studies, staff knowledge in 4 studies, and staff attitudes and resident outcomes in 1 study each.
In a few of the studies that evaluated both knowledge and behavior, it was sometimes the case that staff knowledge improved immediately after training whereas staff behavior remained the same (Campbell et al., 1991; Parker, Leggett-Frazier, Vincent, & Swanson, 1995; Simons, Baker, Jones, Kidd, & Beighton, 2000; Smyer et al., 1992). This evidence suggests some difficulties with transferring knowledge to practice.
|
Discussion |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
We can anticipate that the increase in large-scale educational initiatives will be the trend as well in other countries as resident care continues to become more complex, requiring staff to receive new knowledge and skills on an ongoing basis. This review has demonstrated that more attention has to be paid to the development and evaluation of educational interventions.
The methodology used to evaluate the educational interventions has to be more scientifically rigorous. Some of the methodological limitations could be solved if more research funding were directed to studies of the long-term care system. Even so, the studies in our sample reflect a number of characteristics that are chronic to long-term care facilities that hinder the researchers' ability to perform rigorous research. These characteristics include the high turnover of staff; inadequate staffing and overburdened workloads; and the nonparticipation of facilities because of staffing issues. The result is that many studies have small samples, low attendance at training sessions, and a lack of comparison groups, and thus generate findings that have limited validity. Moreover, measured outcome variables differed widely across the studies, suggesting a need for more consistent research protocols.
Overall, we have almost no evidence of the effective sustained application of educational initiatives, as over three quarters of the studies in our sample did not include a follow-up evaluation in their methodological design. Of the 17 studies that did collect follow-up data, 11 studies reported that improvements were sustained; however, only 1 study reported sustained changes to resident outcomes.
Investigators are beginning to realize that factors in the sociocultural environment are important in determining the success or failure of an educational intervention (Beck et al., 1999; Smith, 1998). This includes the corporate philosophy with respect to the mission, vision, values, and goals of the organization. The sociocultural environment of nursing homes is still seen to foster routine, custodial care, which impedes staff behavior change (Lekan-Routledge et al., 1998).
In three quarters of the studies in our sample, training was provided to staff without any intervention to facilitate the use of the new knowledge and skills in the practice site. In almost all cases in which both knowledge and behavior changes were evaluated, staff showed improvement in knowledge and no improvement in behavior (Campbell et al., 1991; Cohen-Mansfield et al., 1997; Parker et al., 1995; Smyer et al., 1992). The critical issue in training is effecting change in practice.
Successful implementation of training must include organizational and system changes. Thirteen of the studies in our sample did attempt some attention to these factors. One of the researchers concluded that, when an intervention involves organizational and system changes, these are best undertaken in the context of an ongoing, longer-term institutional relationship to ensure administrative support (Smyer et al., 1992). Schnelle and colleagues (1998) suggest that the organization itself should be studied to determine whether the organization could modify or handle the intervention.
Conclusions
The purpose of this review was to gain a better understanding of the effectiveness of continuing education in the long-term care setting. This review has identified that there is minimal research being carried out in this area. Long-term care facilities as a setting for research studies have largely been overlooked, and most of the research studies that are being conducted do not use rigorous methodological designs. In addition, organizational and system factors that can facilitate or hinder the implementation of educational initiatives are rarely considered. The results of this review show that educational initiatives are somewhat effective in the short term but that there is limited evidence for the effective sustained application of continuing education. Rigorous research is needed on the effectiveness of continuing education in long-term care facilities with attention to the role of organizational and system factors.
|
Footnotes |
|---|
1 Consortium for Practical Research in Care of the Elderly, Mount Hope Center for Long-Term Care, St. Joseph's Health Care London, Ontario, Canada.
2 Southwestern Ontario Regional Geriatric Program, St. Joseph's Health Care London, Ontario, Canada.
3 Department of Epidemiology and Biostatistics, University of Western Ontario, Canada.
4 Southwestern Ontario Regional Pain and Symptom Management Program, Ontario, Canada.
5 Thames Valley District Health Council, Ontario, Canada.
|
References |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
This article has been cited by other articles:
 |
|
 |
|
  G. J Byrne Training nursing home staff to improve psychosocial care reduces neuroleptic use in people with dementia without an increase in agitation. Evid. Based Ment. Health, November 1, 2006; 9(4): 103 - 103. [Full Text] [PDF]
|
|
|
|
 |
|
 J. Fossey, C. Ballard, E. Juszczak, I. James, N. Alder, R. Jacoby, and R. Howard Effect of enhanced psychosocial care on antipsychotic use in nursing home residents with severe dementia: cluster randomised trial BMJ, April 1, 2006; 332(7544): 756 - 761. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. Stolee, J. Esbaugh, S. Aylward, T. Cathers, D. P. Harvey, L. M. Hillier, N. Keat, and J. W. Feightner Factors Associated With the Effectiveness of Continuing Education in Long-Term Care Gerontologist, June 1, 2005; 45(3): 399 - 409. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. L. Braun, M. Cheang, and D. Shigeta Increasing Knowledge, Skills, and Empathy Among Direct Care Workers in Elder Care: A Preliminary Study of an Active-Learning Model Gerontologist, February 1, 2005; 45(1): 118 - 124. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
