The Awareness and Use of Outcome Measures by South African Physiotherapists

INTRODUCTION Health care professionals, including physiotherapists, are under pressure to provide evidence for therapeutic interventions (Iles and Davidson, 2006; Jette et al, 2003). It is well documented that healthcare service providers who can demonstrate the effectiveness of care through improved patient outcome, will also be more successful when competing for purchaser funding (Grimmer et al, 2000; Chesson et al, 1996). Purchasers of healthcare may be the patient, hospital administrators, medical aid funders or insurers. An outcome measure (OM) is defined as “a measurement tool (instrument, questionnaire, rating form) used to document change in one or more patient characteristic over time” (Cole et al, 1995). OM’s can be used to measure, demonstrate and monitor therapy within daily physiotherapy clinical practice (Chesson et al, 1996; Lennon, 1995). OM’s can be used to generate evidence to develop new interventions or change existing services (Basmajian and Banerjee, 1996). Chesson et al (1996) affirm that the use of outcome measures “will help to establish credibility, not only with other professions, but also with clients, the community and governments”. Outcome measures can be used to measure the different components of health status, which include the physical impairments caused by a disease, the limitations imposed on activity levels, and restrictions on participation of an individual in society, his/her family life, work and recreation (World Health Organization, 2001). All these components of health are well described by WHO (2001) in the International classification of functioning, disability and health. Quality of life (QoL) measures reflect the patients’ perspective of their day-to-day functioning and well-being (Jette and Jette, 1997). Internationally, physiotherapy organizations have started to advocate the use of OM’s to their members. The World Confederation of Physical Therapy (WCPT), the Chartered Society Correspondence to: Gakeemah Inglis-Jassiem Physiotherapy, Faculty of Health Sciences, Stellenbosch University PO Box 19063 7505 Tygerberg, South Africa Fax: +27 -21 931-1252 Tel: (021) 930-9300 (w) Email: gakeemah@sun.ac.za ABSTRACT: Physiotherapists as well as other health care providers are under pressure to provide evidence for the effectiveness of their interventions. Therefore it has become necessary to employ standardized and robust outcome measures in clinical practice. The objective of this study was to determine the awareness of and use of outcome measures (OM’s) amongst physiotherapists in South Africa. A survey was conducted in 2004 using a self-developed electronic questionnaire consisting of 18 questions, both openand closed-ended. A population-based sample consisting of 1102 members on the email address list of the South Africa Society of Physiotherapy (SASP) was used. Data analysis consisted of both descriptive statistics and qualitative analysis for the open-ended questions. The response rate was 15.2% (n=168). Ninety one percent of respondents reported to have heard of OM’s while 84% reported using OM’s regularly. Impairment related measures were predominantly in use. The two main themes that emerged from the respondents’comments related to reasons for using OM’s were “effective clinical practice”(82%) and “evidence-based practice” (15%). Time constraints and lack of sufficient knowledge in the use of OM’s, were cited as obstacles to using OM’s. These findings have implications for the South African physiotherapy community in terms of education, continuous professional development (CPD) and future research in the usage frequency of OM’s.


INTRODUCTION
Health care professionals, including physiotherapists, are under pressure to provide evidence for therapeutic interventions (Iles and Davidson, 2006;Jette et al, 2003).It is well documented that healthcare service providers who can demonstrate the effectiveness of care through improved patient outcome, will also be more successful when competing for purchaser funding (Grimmer et al, 2000;Chesson et al, 1996).Purchasers of healthcare may be the patient, hospital administrators, medical aid funders or insurers.An outcome measure (OM) is defined as "a measurement tool (instrument, questionnaire, rating form) used to document change in one or more patient characteristic over time" (Cole et al, 1995).OM's can be used to measure, demonstrate and monitor therapy within daily physiotherapy clinical practice (Chesson et al, 1996;Lennon, 1995).OM's can be used to generate evidence to develop new interventions or change existing services (Basmajian and Banerjee, 1996).Chesson et al (1996) affirm that the use of outcome measures "will help to establish credibility, not only with other professions, but also with clients, the community and governments".
Outcome measures can be used to measure the different components of health status, which include the physical impairments caused by a disease, the limitations imposed on activity levels, and restrictions on participation of an individual in society, his/her family life, work and recreation (World Health Organization, 2001).All these components of health are well described by WHO (2001) in the International classification of functioning, disability and health.Quality of life (QoL) measures reflect the patients' perspective of their day-to-day functioning and well-being (Jette and Jette, 1997).
Internationally, physiotherapy organizations have started to advocate the use of OM's to their members.The World Confederation of Physical Therapy (WCPT), the Chartered Society

Correspondence to:
Gakeemah Inglis-Jassiem Physiotherapy, Faculty of Health Sciences, Stellenbosch University PO Box 19063 7505 Tygerberg, South Africa Fax: +27 -21 931-1252 Tel: (021) 930-9300 (w) Email: gakeemah@sun.ac.zaA BSTRA CT: Physiotherapists as well as other health care providers are under pressure to provide evidence for the effectiveness of their interventions.Therefore it has become necessary to employ standardized and robust outcome measures in clinical practice.The objective of this study was to determine the awareness of and use of outcome measures (OM's) amongst physiotherapists in South A frica.A survey was conducted in 2004 using a self-developed electronic questionnaire consisting of 18 questions, both open-and closed-ended.A population-based sample consisting of 1102 members on the email address list of the South A frica Society of Physiotherapy (SA SP) was used.Data analysis consisted of both descriptive statistics and qualitative analysis for the open-ended questions.The response rate was 15.2% (n=168).Ninety one percent of respondents reported to have heard of OM's while 84% reported using OM's regularly.Impairment related measures were predominantly in use.The two main themes that emerged from the respondents' comments related to reasons for using OM's were "effective clinical practice" (82%) and "evidence-based practice" (15%).Time constraints and lack of sufficient knowledge in the use of OM's, were cited as obstacles to using OM's.These findings have implications for the South A frican physiotherapy community in terms of education, continuous professional development (CPD)  of Physiotherapy (CSP) in the United Kingdom (UK) and the Canadian Physiotherapy Association (CPA) have all conducted research to identify their members' needs regarding OM's and have implemented national policies and programs to assist members to effectively utilize these tools (World Confederation of Physical Therapy, 2003;Chesson et al, 1996;Cole et al, 1995).
The current situation regarding the awareness and use of OM's by physiotherapists in South Africa is not known.
In the light of the political and healthcare transformation in South Africa it has become increasingly important for physiotherapists to justify the role of physiotherapy in promoting health to government and private funders.The global drive for the incorporation of OM's and the current national reform motivated the researchers to conduct a survey in 2004-2005.The aims of the survey were to describe the awareness of members of the South African Society of Physiotherapy (SASP) regarding OM's, to describe the usage patterns of OM's as well as the characteristics of the OM's in use.The researchers also wanted to identify the specific educational needs of physiotherapists regarding OM's in South Africa.

METHODOLOGY
An electronic survey was conducted as part of a descriptive study design.A population-based sample consisting of all members of the South Africa Society of Physiotherapy (SASP) listed on the national email address-list held at the SASP Head Office was included.According to the SASP head office there were approximately 1129 members listed in February 2004 (Personal correspondence with the CEO, February 2004).Permission to use the address list for the study was granted by the Chief Executive Officer (CEO) and executive committee of the SASP.Ethical clearance was obtained from the Committee for Human Research at Stellenbosch University (N04/03/055).
The survey questionnaires were sent out electronically for ease of administration, distribution, time efficiency and to limit expenditure.To retain confidentiality of the email address list, the administrative staff at the SASP head office was responsible for emailing the questionnaires to the therapists.The respondents were requested to forward their completed questionnaires to the address of a research assistant stationed at the Physiotherapy Department, University of Stellenbosch (US).To ensure anonymity, the names or email address of the respondents were deleted from all returned questionnaires by the research assistant.These questionnaires were then printed and forwarded to the researchers in hard copy format for further analysis.
Due to a poor initial response rate in 2004 of only 9% (n=102), the survey was re-sent to the entire group of therapists in 2005.Respondents were requested not to respond to the second survey if they had already done so to the initial posting.Physiotherapists who were no longer practicing were excluded as the researchers wanted to identify the awareness and use of OM's in current daily clinical practice.Non-practicing therapists may be familiar with OM's but would not have provided information on the impact of these tools clinically.

Instrumentation 1
A self-developed questionnaire was used for the electronic survey.It consisted of 18 questions of which 5 were open-ended and 13 were closed-ended.The questionnaire comprised of two sections: • Section A related to the respondents' demographic details, and • Section B aimed to capture the respondents' awareness and use of OM's.Respondents had to indicate sources where they had heard of OM's, specific OM's used, frequency of use, possible reasons for the use of OM's as well as barriers.They were also requested to indicate the best ways of disseminating information regarding OM's to other physiotherapists.The OM's ranged from general to those specific to a subject area in physiotherapy.Provision was made for the respondents to add OM's to those listed in the questionnaire.
The choice of the OM's included in the questionnaire was informed by peer consultation, review of the pertinent literature and recent similar studies on OM's.The content validity of the questionnaire was conducted by reviewing the contents of similar published questionnaires (Huijbregts et al, 2002;Kay et al, 2001;Chesson et al, 1996).A list of items to be included in the questionnaire was generated and further content vali-dity was established via consultation with academics and specialists in different physiotherapy subject areas.Academics from South African universities, who taught in the various physiotherapy subject areas, were consulted.These individuals were requested to ascertain whether the list of OM's was comprehensive and inclusive of all OM's used in the subject areas in S.A.
A pilot study was also conducted to assess the user-friendliness of the electronic questionnaire and whether each item on the questionnaire addressed the objectives set.Questionnaires were sent electronically to eight physiotherapists in the Cape Town area who were not members of the SASP and would therefore not form part of the research sample.The respondents were requested to forward any comments regarding the clarity of instructions, ease of completing the questionnaire electronically and to make any other suggestions.Based on their comments, the table formatting was revised before the questionnaire was forwarded to the research sample.

Statistical Analysis
The completed questionnaires were numbered and the data entered into an Excel spreadsheet to allow for initial ordering and capturing of the data.The statistical package, "Statistica", was employed and analysis of the data consisted of calculating descriptive statistics for the responses to all the questions.These descriptive statistics were presented in the form of histograms, bar and pie charts, and tables giving frequencies of responses with percentages.
Qualitative analysis was employed for data derived from open-ended questions which dealt with the main reasons for using OM's, the obstacles to their use and educational needs of physiotherapists with regards to OM's.The researchers grouped the comments to identify major themes for each of the questions.These themes were then depicted by means of pie graphs and typical comments were paraphrased in the text to further describe the identified themes.

RESULTS
In total, 168 completed questionnaires were received at the end of the survey (includes both attempts), resulting in a final response rate of 15.2% (n=168/1102).Fifty three returned questionnaires were however excluded from the final analysis as they were blank, incomplete or "undeliverable".See Figure 1 for more detail.

Demographic information of sample
The majority of respondents (58%) worked in orthopaedic related fields, such as Orthopaedic Manipulative Therapy (OMT), Orthopaedics and Sport.Eighty percent of the respondents were private practitioners while the remaining therapists worked in the public-(10%) or education sectors (10%).The greatest proportion of responding physiotherapists practised in Gauteng (41%) and the Western Cape (30%).The sample represented many of the special interest groups (SIG) such as the "Orthopaedic" SIG (89 respondents), the "Neuro-rehabilitation" SIG (19), "Cardiopulmonary" (10) and the "Public sector" SIG (10).Most of the respondents were well experienced therapists with more than 10 years experience (53%).Twenty five percent of respondents had between 5 and 10 years experience while 20% had less than 5 years experience.
The rest of the results below will depict the exact number of respondents for each section.As can be seen in Table 1, most respondents had been exposed to OM's by reading journal articles (58%) or as students (37%).Most of these therapists were however using impairment-based OM's such as auscultation, ROM and Oxford scale, with very few using participation measures.

Ease of use of OM's (n=different for each OM)
The respiratory-related impairment measures were considered to be the most user-friendly.It should be noted that the OM's most often used were also amongst those reported to be the most user-friendly (Figure 2).Physiotherapists were equally divided in the perception of user-friendliness of function-related measures such as the "Barthel Index".Participation type measures were considered to be of the least user-friendly OM's, e.g. the "Functional Independence Measure" (FIM) and "SF 36".

Main reasons for using outcome measures (n=140)
The two main themes that emerged from the respondents' comments were "effective clinical practice" (82%) and "evidence-based practice" (15%).Included in the first theme there were forty six responses which indicated that OM's were used to "evaluate effectiveness of treatment or intervention".Following this, 38 comments indicated that OM's were used to "measure patient progress or improvement".OM's were considered to be an "objective and reliable measure" of clinical effectiveness and an "accurate reproducible means of record keeping".

Perceived obstacles to the use of OM's (n=133)
Four main themes emerged when respondents reflected on the obstacles to the use of OM's (Fig. 3).Most respondents felt that time and staff shortages were the largest barriers to using OM's (46%).Other obstacles mentioned were related to the instrument or OM (31%), the target client(s) with whom the OM was used (13%) and the therapists' lack of knowledge or experience with OM's (9%).The following are typical comments made by physiotherapists in our survey regarding the OM instruments/tools, • Poor "accuracy", "subjectivity", "sensitivity" and "inappropriate equipment to measure" • OM's were "difficult to use", were "(in) accessible" and often involved "cost" Client related obstacles centred mainly on "patients' perceptions" of OM's and "literacy of clients" in the South African context.Respondents also indicated that their "lack of knowledge", "lack of experience in using OM's" and "inadequate training" were obstacles to using OM's.They were also "not sure which outcome measures to use".

DISCUSSION
The physiotherapy profession plays an integral part in the health care system as it involves the promotion of health, prevention of dysfunction and the acute care and rehabilitation of individuals with disability (Higgs et al, 2001).In order to fulfil their role, "physiotherapists are legally and professionally required to undertake a comprehensive assessment of the client, formulate a physiotherapy diagnosis, plan and implement a therapeutic programme where appropriate, evaluate the outcome of any intervention, and determine discharge arrangements" (WCPT, 1999).The consistent incorporation of OM's has become even more important due to the increasing demand being placed on healthcare professionals to provide evidence to support the effectiveness of their interventions and therapists are encouraged to use reliable, valid and sensitive OM's for all areas of physiotherapy management (Maher and Williams, 2005).
The majority of South African physiotherapists surveyed in 2004 were aware of OM's, but tended to use predominantly impairment-based and non-standardized OM's.This is in contrast to the findings of a study conducted in Nigeria in 2000 that indicated a low level of awareness and use of OM's (Akinpelu and Eluchie, 2006).The Nigerian study, however, utilized a prescribed list of standardized OM's whereas our study used both non-and standardised OM's.The majority of the South African respondents indicated that professional journal articles were the main source of information on OM's and this is similar to previous findings in a Canadian study (Kay et al, 2001).Apart from the written format, Canadian physiotherapists also indicated receiving information on OM's during presentations and in-service training on the topic to further facilitate their clinical use of OM's.The South African Health Department and SASP could attempt similar programmes to assist physiotherapists to use OM's consistently in daily clinical practice.
The findings of this study suggest that the respondents evaluate patient progress and management at the level of impairment as the majority of reported OM's fell into this category.Impairment OM's only reflect information at a physiological or structural level, e.g.goniometry, strength or pain (Hammond, 2000).Respondents indicated that they frequently used generic OM's such as the Visual Analogue Scale ("VAS"), Range of Motion ("ROM") and Manual Muscle Testing ("Oxford Scale").While these OM's may be easy to use, they have poor validity and/ reliability and many are not sensitive enough.Goniometry is considered a valid measure of joint ROM but it has poor inter-tester reliability as application of the goniometer is often not standardized across physiotherapists.Another disadvantage of this tool is the coincidental location of its fulcrum with the axis of rotation of the joint and without proper placement of the goniometer relative to the joint, unreliable results ensue (Brosseau et al, 2001).Manual Muscle Testing remains a subjective measure which lacks sensitivity and has proven poor inter-and intra-tester reliability (Bohannon, 2005;Grimmer et al, 2000).Problems include the differential application and interpretation of muscle tests, the possibility of subjective assessment of response and the categorical nature of the strength scale.The respondents also reported using respiratory related impairment measures, i.e. "Auscultation" and "Peak Expiratory Flow Rate".Auscultation relies on subjective interpretation of lung sounds that can be heard via a stethoscope.The reported poor reliability of auscultation is concerning as physiotherapists often have to treat patients previously examined by other colleagues and use these findings to judge changes in the patients' status (Aweida & Kelsey, 1990;Brooks & Thomas, 1995).
The "Barthel Index" (BI) is an activity measure which only 10 physiotherapists in our study reported using.The BI has several limitations which include omission of some activities of daily living such as cooking or shopping and does not reflect mental status and social well-being (Bowling, 1997).This scale has both floor and ceiling effects in that further physical improvement can occur beyond the endpoints of the scale.Only a few respondents appeared to use QoL measures.QoL measures were however amongst those mentioned as being less user-friendly and this might play a role in the lack of their use.These measures often require more time spent with the patient, skill on the part of the measurer and is often used at the end of a series of interventions or the rehabilitation process (Tooth et al, 2003).It would appear from the array of OM's used that these respondents were more interested in the impairment level and less so in activity and participation level.It could be argued that physiotherapists in this sample were more interested in the immediate effect of therapy rather than the cumulative effect of a rehabilitation process.
Effective clinical practice and evidence-based practice (EBP) were the main themes derived from reasons cited by these respondents for using OM's.Various other uses of OM's are described in the literature which includes justification of interventions to funders and the identification of opportunities for new or changed services (Grimmer et al, 2000).It could therefore indicate that the responding physiotherapists were not aware of these valuable uses of OM's other than its use in direct clinical management.This is concerning as the majority of respondents were in fact private practitioners who are often required to motivate for payment of services or extension of treatment times with medical aids.
A few respondents (n=26) indicated that the use of OM's was mandated in the workplace.Contrary to this, a large proportion of the Canadian sample indicated that the use of OM's were mandated in their organization or service area which may have led to greater usage frequencies (Kay et al, 2001).The CSP in the UK also recently mandated the use of OM's in a document; "Core Standards of Physiotherapy Practice" (Chartered Society of Physiotherapy, 2005).In this practice guideline it is made an explicit requirement for members to use published, standardised OM's in their routine clinical practice.By including the requirement within the core standards, the profile of OM's is highlighted and reflects the increasing need for members to provide evidence on the outcome of their interventions.At present there are no legal requirements or professional incentives in South Africa encouraging physiotherapists to use OM's.Anecdotally however there has been a rise in medical aid scheme requirements for physiotherapists to use objective tools to evaluate patient progress.Physiotherapists have also seen a motion for preferred payment of interventions with a sound evidence base from medical aid schemes in recent years.Past research has proven that passive approaches such as distribution of educational material and clinical guidelines alone do not influence physiotherapy practice, but that mandates given by funders and professional societies are more effective to bring about change in OM usage (Abrams et al, 2006).Other factors such as professional support, active education initiatives and monitoring by peer review added to the success in the previously mentioned Australian study (Abrams et al, 2006).

Perceived obstacles to using outcome measures
Similar to other studies, time constraints were the main obstacle to the use of OM's (Abrams et al, 2006;Huijbregts et al, 2002;Kay et al, 2001).High clinical loads, numerous patient admissions and staff shortages resulting in disproportionate therapist to patient ratios as well as lack of resources are all possible reasons for therapists having insufficient time.However, it could be argued that physiotherapists, who are pressed for time to gather evidence, could motivate for more staff with hospital administrators using objective evidence gathered from consistent implementation of OM's.Further study amongst this sample of predominantly private practitioners is needed to ascertain their unique difficulties related to time constraints.
Numerous databases and strategies are available to assist clinicians to access evidence-based information and OM's with greater ease.Examples of these electronic resources include, The "Cochrane Library" (Metcalfe et al, 2001) and "Physiotherapy Evidence Database" (PEDro) (Moseley et al, 2002).The database held at the National Institute for Clinical Excellence (NICE) in the UK for example, holds more than one hundred OM's (Hammond, 1999).It provides useful information such as the OM's original reference; OM reliability, validity and sensitivity; summaries of time required to complete the OM as well as training and costs involved. 2 In South Africa, health services are being decentralized and community service has become compulsory for health workers, including physiotherapists (Health Western Cape, 2005).The problems related to lack of time and resources are further compounded for these therapists as they are further removed from electronic resources and professional guidance from other therapists.It is therefore important that the greater physiotherapy community should become more proactive in their professional evaluation of clinical practice.Physiotherapy educators, practice/ hospital managers, provincial health departments, SASP and physiotherapists themselves should be striving to implement necessary support and resource structures to assist clinicians in providing evidence-based practice.

CONCLUSIONS, LIMITATIONS AND RECOMMENDATIONS
Although the conclusions drawn from such a small sample is limited, the researchers were able to identify that physiotherapists seemed to be aware of OM's, but tended to use predominantly impairment-based OM's.As in other studies, the respondents continue to use familiar measures despite the ever-increasing number of new standardized OM's.
There were three main limitations to the study.The population-based sample was restricted to physiotherapists on the email list of the SASP and the findings can therefore not be generalised to the rest of the South African physiotherapy population.Bias was introduced as access to email was a prerequisite for participation and therefore a number of therapists were excluded.Respondents were most likely more interested in OM's which introduces bias into responses received and therefore the results of the study.Another possible source of bias is the under-representation of certain regions; geographical as well as model of service delivery (public or state facilities).Future researchers might consider stratified randomized sampling of therapists registered with the Health Professions Council of South Africa (HPCSA) to minimize these sources of bias.
The findings of this study have direct implications for physiotherapy undergraduate training and continuous professional development (CPD) in South Africa.The findings could be used to motivate for the establishment of training workshops by the SASP via its special interest groups in the form of compulsory OM-related workshops.The Society could also implement the following: • A series of articles on OM's in the South African Journal of Physio therapy • A self-study electronic package on OM's with the added incentive of CPD accreditation • Posting more information on OM's on the SASP website, including special links to international databases and contact details of individual physiotherapists that are interested in OM's.

AKNOWLEDGEMENTS
We wish to thank all those physiotherapists who took the time to respond to our survey as well as the SASP and its administrative staff for their cooperation and assistance in distribution of the questionnaires.We also wish to thank Dr Martin Kidd for his assistance with statistical analysis.

Figure 2 :Figure 3 :
Figure 2: Response regarding ease of use of OM's amongst SA physiotherapists (Multiple responses permitted)

Rehab) US 3 1 Lecturer -Physiotherapy Department (Stellenbosch University). 2 Previous Head of Physiotherapy Department (US). 3 Physiotherapist, Cape Town.
and future research in the usage frequency of OM's.