Focus on Alternative and Complementary Therapies
www.pharmpress.com/fact
Focus Alternat Complement Ther©2005 Pharmaceutical Press
Focus Altern Complement Ther 2002; 7: 330–3
Strong public demand for complementary and alternative medicine (CAM) combined with growing interest in these therapies and products by health providers is giving rise to a form of healthcare delivery that seeks to integrate CAM and conventional medical approaches. Such integration has been described as, ‘interdisciplinary teams to deliver an expanded repertoire of safe and effective treatments that include a focus on the whole person’1 or ‘blending the best of conventional (allopathic) and complementary and alternative medicine’.2 Interestingly, the concept of integration is not new and not limited to CAM. For example, Way and Jones’ definition of collaborative practice as ‘an inter-professional process for communication and decision-making that enables the separate and shared knowledge and skills of care providers to synergistically influence the client/patient care provided’ is very similar to the descriptions of integration above, but refers to the collaboration between family doctors and nurse practitioners.3
Integrated health care (while the terms medicine and health care are currently used interchangeably, we prefer the term integrated health care rather than medicine; the former is defined as ‘all services, products and activities used by individuals for the purpose of promoting, maintaining, monitoring or restoring health’),4 which is different from both CAM and conventional medicine, is generally considered to integrate different treatment approaches, to be holistic in nature, to recognise the existence of multiple determinants of health, to offer individualised treatment that aims at restoring physical, emotional, structural, energetic and spiritual balance and be based on the body’s innate ability to heal.5,6 In such a model the whole is more than the sum of the parts and practitioners view synergy between the different interventions as contributing to the desired outcome. Because integrated healthcare delivery programmes are relatively new, the question of appropriate evaluation of such models has not been addressed in detail in the literature. In many ways the evaluation of integrated health care faces challenges that are similar to those encountered in trying to evaluate CAM whole systems such as naturopathy and traditional Chinese medicine.7 In whole systems research, processes (What was done? How well?), outcomes (What are the results of the intervention?) as well as the structure (Who is providing the care? What is their competency?) of the intervention need to be investigated. To answer the same questions about integrated health care, optimal evaluation should be equally inclusive and comprehensive.
The objective of this paper is to identify and discuss appropriate ways of evaluating such a model. The Tzu Chi Institute for Complementary and Alternative Medicine (TCI) in Vancouver, which is one of the first integrated healthcare delivery programmes in Canada, is used as an example to illustrate our ideas. Although our discussion will focus on evaluation at the patient level, it should be recognised that practitioners, administrators and policy makers are also important stakeholders in the study of integrated healthcare delivery.
The TCI is a non-profit organisation, which aims to develop and provide integrated health care that supports people in making positive health changes. This is achieved by providing a combination of complementary, alternative and conventional health care that integrates mind, body, spirit and energy (holistic care). Patients are self-referred and often present with complex health concerns or multiple chronic conditions. They receive care from an interdisciplinary team of practitioners representing medicine, chiropractic, nursing, naturopathy, acupuncture, traditional Chinese medicine, massage therapy, nutrition and mind-body healing. Integrating therapeutic approaches, provided by practitioners with enormously different professional training, is one of the major challenges of establishing an integrated healthcare delivery programme. The practitioners have very different value systems and beliefs about what constitutes health and well-being, appropriate treatment and an optimal patient–practitioner relationship. An integrated assessment approach and a common chart facilitate the ability to care for the patient in an integrated manner.
Evaluation of the integrated care programme is a vital component of the TCI’s mandate. All new patients are asked to participate in the outcomes research as well as in other evaluation projects related to integrated care issues. This serves to improve the programme continuously as well as communicate to stakeholders what the programme’s outcomes are and how such a programme functions and develops over time. The TCI has been systematically collecting data since 1999.
Evaluation of integrated healthcare programmes needs to be comprehensive, multi-faceted and to consist of multiple systematic strategies. Both quantitative and qualitative methods are needed to bring a holistic perspective to the evaluation of such models. A holistic (combined methods) approach is particularly relevant for a programme that aims at providing holistic care since it facilitates assessment of the complexities (e.g. interdependencies of the various aspects of the healing process, idiosyncrasies and the setting and context) of an integrated care model.
The positivist paradigm that underlies quantitative research assumes that there is a fixed, relatively orderly reality that can be studied objectively. The naturalistic paradigm underlying qualitative research holds that there are multiple interpretations of reality, and that the goal of research is to understand how individuals construct reality within their context. Both research traditions, the positivist and the naturalistic, have strengths and weaknesses, which are increasingly recognised by the other. Quantitative methodologists are beginning to recognise that naturalistic enquiry represents a legitimate and logical alternative research strategy, and naturalistic researchers are giving greater attention to the standardisation of analysis and procedures and to the complementary role of experimental-type designs.
The term formative evaluation is used when the aim is to improve a new or ongoing programme by immediately feeding back results into the process of shaping the programme. Such research is often qualitative. Summative evaluation sums up the results of a programme or intervention in order to make an informed decision about the introduction of a certain programme or service.8 This type of evaluation is mostly quantitative. The following represent a range of approaches, both summative and formative, to evaluate integrated health care.
Ideally, integrated health care would, whenever possible, be assessed by means of an RCT to control for bias and enhance internal validity. While strong arguments have been made in the literature that many complex programmes (e.g. psychotherapy, public health and public policy interventions) could and should be evaluated using RCTs,9 drawbacks of this approach have also been identified.10 The intent of classical explanatory RCTs is to demonstrate specific effects over and above non-specific or synergistic effects. The latter are, however, vital to integrated health care. Pragmatic trials in which the effect of the ‘whole package’ is assessed are problematic because they fail to identify what it is that makes integrated care work or to provide information about how the integrated care programme should be changed to maximise its benefit. Randomisation can also be a problem. In a population that is hard to identify (complex multiple chronic and stress-related health problems) and expresses strong preferences for treatment, randomisation may be difficult or unethical. As most integrated healthcare models are still in the development phase, RCTs may simply be premature.
Quasi-experimental designs with non-randomised groups – matched or unmatched – are a possibility when randomisation is not possible. If a sufficient number of people are on the waiting list, these could be used as controls. Richardson used this design to evaluate outpatient acupuncture, osteopathy and homoeopathy services.11 The drawback of this method is that length of follow-up is limited. The use of existing databases to allow for comparisons of outcomes in different groups is still largely unexplored in this area. In Canada, the National Population Health Survey, a longitudinal survey in which data are collected on a wide range of health-related variables,12 could be used as a source for a potential matched control group. The challenge is to find outcome variables that are relevant in integrated health care that have also been collected in the database.
Descriptive research, including surveys and chart reviews, are needed to identify who is using integrated health care and what their beliefs and reasons for use are. Such information is important in planning and targeting integrated healthcare services. Chart reviews at the TCI have identified that the patient population consists of people with complex health histories and multiple stressors in their lives.
In outcomes research, outcomes are assessed at baseline and again after certain intervals of time to determine whether change has occurred. Outcomes research is open to multiple biases and therefore change(s) cannot be attributed exclusively to the intervention over and above the effects of other factors.
An important issue in outcomes research is the choice of appropriate outcome measures. Many desired outcomes of an integrated care programme are similar to those of any mainstream healthcare delivery programme and include alleviating symptoms of disease, improving well-being and patient satisfaction with treatment. At the TCI outcome measures include quality of life (SF36),13 patient satisfaction (which is adapted from the Picker Patient Questionnaire), sense of coherence,14 health locus of control15 and perceived self-efficacy.16 However, other relevant outcomes that form the core of integrated health care, such as self-awareness, motivation, balance, patient empowerment or the ‘integratedness’ of the care delivery system are not yet being measured. For some of these outcomes no measures are available and further exploration by means of qualitative research is required to develop such measures.
Cost effectiveness is an important aspect of a new programme of healthcare delivery. It entails the systematic collection of detailed costing data for the programme under study and a comparison group, so that economic costs as well as benefits can be compared. Similar issues regarding randomisation and the choice of control groups, as identified above, apply in this area.
Quantitative research will identify whether there is, or is perceived to be, an effect of an integrated healthcare programme; however, it does not explain why or how the integrated approach worked. Quantitative research also does not explain how interactions and interventions are translated into health behaviours and cannot accommodate the complexity of interrelated factors that influence health and healing choices.
Qualitative research allows exploring the process, context and meaning of the intervention to patients.17 This information will assist in identifying what does and what does not work. In the TCI we have conducted patient focus groups in which we explored what patients think we should continue to do, what we should stop doing and what things we should start doing. This has led to some important changes in the TCI integrated healthcare programme.
Personal transformation has been identified as an important aspect of healing.18 There are virtually no empirical data to explain how transformation takes place or how the process might be enhanced. Therefore, we have initiated a qualitative study to elucidate some of these issues. As indicated previously qualitative research can assist in developing relevant outcome measures and might also begin to shed light on the concept of synergy between components of integrated care. The obvious drawback of qualitative research, however, is that results cannot be generalised.
Quantitative and qualitative research is often combined in one project to enhance data collection or to build on each other sequentially.19 A useful example relates to goal attainment. In a programme that focuses on individualising treatment, it is very important to assess patients’ treatment goals and the degree to which these are met. Becker et al. describe how goal attainment scaling to measure individual change in intervention studies could be applied quantitatively.20 This method uses a similar list of goals for all participants. However, people may have much more individualised goals than this method can address. At the TCI, information about patients’ personal goals in their own words is collected and recorded upon intake. During patient follow-up, it has been interesting to note how often patients have forgotten or changed their treatment goals. Much could be gained from a systematic qualitative analysis of patients’ goals and assessment of the circumstances in which these goals are being met or changed. This could lead to the development of an instrument that has a standardised as well as a qualitative component.
A range of research methods should be considered when evaluating a new phenomenon such as integrated health care. There is no best method. The choice of design depends on the stage of development of the programme or model of integrated health care and on the research question. We have identified only some of the potential approaches. In addition, much more work needs to be done to define integration and to operationalise the definition. Most important however, is to identify and define the research questions that arise out of the practice of integrated health care. Once this has been done, the correct pathway for inquiry becomes much easier to establish and the results of inquiry, ultimately more useful.
