Scale to Measure Patient Satisfaction With Physical Therapy

2 TV Perneger, MD, PhD, is Head, Quality of Care Unit, Geneva University Hospitals, CH-1211 Geneva 14, Switzerland.

Physical Therapy, Volume 82, Issue 7, 1 July 2002, Pages 682–691, https://doi.org/10.1093/ptj/82.7.682

Cite

Dominique Monnin, Thomas V Perneger, Scale to Measure Patient Satisfaction With Physical Therapy, Physical Therapy, Volume 82, Issue 7, 1 July 2002, Pages 682–691, https://doi.org/10.1093/ptj/82.7.682

Abstract

Background and Purpose. Patient satisfaction can be one indicator of quality of care. In this study, a patient satisfaction questionnaire for physical therapy was developed. Subjects. The subjects were a consecutive sample of 1,024 patients who received physical therapy between January and March 1999 at a teaching hospital in Geneva, Switzerland. Methods. A cross-sectional mail survey was conducted in which a structured questionnaire measuring patient satisfaction with various aspects of physical therapy followed by open-ended questions was sent to the subjects. Results. Overall, 528 of 1,024 patients (52%) responded (patient demographics for 501 respondents who provided demographic data: mean years of age=58.6, SD=18.9, range=15–95; 258 men, 243 women). Factor analysis was used to identify main domains of satisfaction, and a scale was constructed to measure satisfaction with each dimension: treatment subscale (5 items), admission subscale (3 items), logistics subscale (4 items), and a global assessment subscale (2 items). All subscales had good acceptability and small floor and ceiling effects. Internal consistency coefficients varied between .77 and .90, indicating good reliability for all subscales. Scale validity was supported by a logical grouping of items into subscales, according to their content, and by correlations of satisfaction scores with the patient's intention to recommend the facility and with the number of positive and negative comments to open-ended questions. Younger patients were less satisfied than older patients for 2 of the subscales (admission and logistics). Discussion and Conclusion. The 14-item instrument is a promising tool for the evaluation of patient satisfaction with physical therapy in both inpatients and outpatients.

Patient satisfaction is increasingly used to monitor patient perceptions of the quality of health care services. 1– 3 Various questionnaires exist to measure satisfaction with hospitalizations, 4, 5 medical office visits, 6– 8 or health care received under a health insurance contract. 9– 12 In our opinion, the variety of tools used suggests that survey instruments should be matched to the health care service being evaluated. Physical therapy has several characteristics that may influence patient satisfaction: the interaction often takes longer than a routine medical visit, it involves more physical contact, therapy usually requires the patient's active participation, and therapy may cause pain and may be perceived as physically threatening. Therefore, a satisfaction questionnaire used for visits to physicians may not be optimal for physical therapy.

At the time this study was planned, no patient satisfaction questionnaire specifically designed for evaluation of physical therapy was available. Since then, 3 such questionnaires have been published. 13– 15 The first questionnaire 13 contains 34 items and focuses predominantly on issues of cost and access, with less emphasis given to the treatment itself. The 4 subscales in the instrument are: enhancers, detractors, location, and cost. The second instrument 14 is shorter (20 items); however, although the authors intended to measure 5 aspects of patient satisfaction, factor analysis revealed that all items formed a single, highly consistent dimension. The third instrument 15 was designed to measure satisfaction with physical therapy given to patients with cystic fibrosis. The factorial structure of this instrument contained 4 dimensions: “effectiveness,” “convenience,” “comfort,” and an overall score. Because these existing questionnaires differ so much, further exploration of how to measure patient satisfaction with physical therapy remains relevant.

We conducted our study among patients who received physical therapy at a large Swiss teaching hospital, with the goal of developing an instrument for measuring patient satisfaction. Furthermore, we sought to identify patient characteristics associated with satisfaction levels. We believe variables such as these should be accounted for in patient satisfaction surveys to serve as descriptors of case mix and to allow for statistical adjustments when comparing the results of facilities that serve different patient populations.

Method

Study Design

We conducted a survey of a consecutive sample of 1,024 patients (both inpatients and outpatients) who were treated at the physical therapy center of the Geneva University Hospitals between January and March 1999. Patients were contacted by mail after completion of their treatment and were invited to fill out a structured questionnaire. Out of the sample of 1,024 patients, 432 patients responded initially. Nonrespondents received a reminder mailing after 3 weeks and another 3 weeks later, if necessary. Another 189 patients responded to these mailings. Because patient satisfaction surveys are considered to be standard management tools at this hospital, the medical director's office exempted this project from formal approval by the institutional review board.

Questionnaire Development

We developed the questionnaire in French, the language spoken in Geneva, Switzerland. We reviewed various satisfaction questionnaires 3– 12 to identify themes that we believed should be part of a patient evaluation of satisfaction with physical therapy. Given the abundance of publications in this field, we did not conduct a comprehensive review, but we performed a selective review that was stopped when little new information was obtained from additional instruments. We were particularly interested in questionnaires developed for single visits that were available in French. We started with review articles 3, 12 that included descriptions of widely used instruments (eg, the Patient Judgment System, 4 the Picker questionnaire, 5 and patient satisfaction survey of the Group Health Association of America 9, 10 ), studies focusing on ambulatory care, 6– 8 and questionnaires that have been successfully used with the local Geneva population. 4, 8, 11

The various aspects of patient satisfaction identified through the literature review were discussed with physical therapy staff from the Geneva University Hospitals. Staff were also invited to add variables to consider according to their experience. Staff proposed the following items for inclusion: whether the patient felt physically secure during treatment (as opposed to fearing pain or injury) ( Appendix 1, item 18), the consistency and logical progression in the patient's treatment ( Appendix 1, item 12), and whether treatment was well adapted to a patient's problem ( Appendix 1, item 19). At this point, we also chose to use a 5-point “poor” to “excellent” response scale for the instrument because previous research indicates that this response scale is better than a 6-point “agree” to “disagree” scale. 16 The first author wrote the items corresponding to the variables selected, following most recommended rules for item writing (simple syntax, lay vocabulary, active voice). What we considered double-barreled formulations were allowed when they served to broaden or clarify content (eg, “Courtesy and helpfulness of secretary” is a double-barreled query, but addresses a single domain [interpersonal skills of the secretary]). One item included a conditional clause (“If several physical therapists took care of you, your perception of these changes”), because this situation did not apply to all patients. The questionnaire was pretested for clarity and adequacy of content with 10 patients.

Study Variables

The questionnaire included 2 descriptive items (whether the patient was an inpatient or an outpatient and who selected the hospital's physical therapy center), 25 statements describing a characteristic of services received using a 5-point Likert scale (“poor,” “fair,” “good,” “very good,” “excellent”), 2 validation items probing future intentions (about recommending the center and coming back to the center if physical therapy was needed again) using a 5-point Likert scale (“certainly not” to “yes, certainly”), and 3 open-ended questions about the reasons for returning (or not returning) to the center and the strengths and weaknesses of the physical therapy center. In addition, information on patient age, sex, inpatient versus outpatient status, and type of rehabilitation (medical, cardiorespiratory, neurological, or orthopedic) was obtained from the hospital's administrative database.

Coding of Comments to Open-Ended Questions

Before the survey questionnaires were sent out, we decided that comments under the heading of “strengths” were to be considered positive and those under “weaknesses” were to be considered negative. The first author verified this classification by reading the responses. This author also classified responses to the questions about the reasons for returning (or not returning) to the center as positive, negative, or neutral and counted the number of separate comments in each category. Comments were counted as separate if they addressed different aspects of a patient's experience. For instance, if a patient wrote “competent staff, access to swimming pool” under strengths, this was coded as 2 positive comments, whereas “nice and competent staff” counted as 1 comment. The coding of comments to open-ended questions was performed by the first author without considering responses to closed-format items. The reliability of the coding process was not verified. The content of the comments was also examined (data not shown).

Statistical Analysis

Distributions of all items were examined for missing data, proportions with lowest response (floor effect) and highest response (ceiling effect), means, and standard deviations. To identify separate dimensions of patient satisfaction, we performed a factor analysis followed by varimax rotation 17 on all items and after exclusion of the global assessment of physical therapy received and the 2 validation items. The latter items were taken out because they address an overall evaluation of the care received, whereas factor analysis attempts to isolate specific dimensions of satisfaction. Examination of the scree plot and the Kaiser rule (eigenvalue of 1) were used to decide on the number of factors to retain.

Based on factor analysis, we computed global summary scores based on items with primary loading on the same factor. The scores were computed for all participants who had missing data on less than half of the relevant items, in effect substituting the respondent's average for missing values. The summary score was the mean of nonmissing values, scaled between 0 (lowest possible score) and 100 (highest possible score). Because items were originally expressed on a scale of 1 to 5 (“poor” to “excellent” or “certainly not” to “yes, certainly”), the transformation equation was: score_0–100=25 × (score_1–5 − 1).

Then, to obtain a shorter measurement instrument, we reduced the number of items in each subscale using the following criteria: maintenance of content validity, an internal consistency coefficient (Cronbach α) of >.75, and preference for items with fewest missing values. 17 These criteria address different aspects of the instrument's performance (internal consistency validity and applicability to the local population), and a compromise was reached at each step. The factorial structure of the instrument was verified after item reduction. To check the reliability of the factorial structure, we used a bootstrap procedure, whereby the study sample was resampled with replacements 30 times, and the factor analysis was repeated on each of the subsamples. 18 To verify that shortened scales retained the content of the longer initial scales, we checked the Pearson correlations between the 2 versions of the scales.

Evidence for the validity of data obtained with the scales came from the items on future intentions (intention to return or to recommend) and positive and negative comments to open-ended questions. Positive comments were expected to be more frequent among satisfied respondents, whereas negative comments were expected to be contributed more frequently by dissatisfied respondents. Similarly, the stronger the patient's recommendation or intention to return, the higher his or her satisfaction should be. The satisfaction scores were used as dependent variables in a one-way analysis of variance, and future intention items and number of comments were used as independent variables. Because the validity hypothesis assumes a gradual relationship between these variables, we estimated the linear trend component of the variance explained and the deviation from linearity separately. When the independent variable comprises p ordinal levels, the differences in the dependent variable between the p subgroups are tested by an F test with p-1 degrees of freedom (where the null hypothesis is that all subgroups have the same mean). 19 The between-groups sum of squares can be split into 2 components: (1) the part that can be attributed to a linear relationship between the dependent variable and the levels of the ordinal independent variable (linear trend test, with 1 degree of freedom, of the null hypothesis that the slope is zero) and (2) the remainder of the sum of squares (test for deviation from linearity, with p-2 degrees of freedom; the null hypothesis corresponds to no deviation from linearity). 19 After construction of the 3 dimension-specific scales, we formed a general satisfaction scale, based on the overall assessment item and the 2 future intentions items.

Finally, the 4 resulting subscales were examined in subgroups defined by the following patient characteristics: age, sex, whether the patient was an inpatient or an outpatient, and the type of physical therapy received (medical, cardiorespiratory, neurologic, orthopedic). Subgroups were compared using Mann-Whitney tests. Because satisfaction increased with age, we tested whether a linear trend in satisfaction was significant across age groups. All tests were performed with a type I error rate of 5%.

Results

Of 1,024 questionnaires mailed out, 621 were returned; however, 87 of those returned were blank, and 6 were almost blank. Therefore, we received 528 usable questionnaires, a 52% response rate based on the initial mailing. Because answering the survey questions requires proficiency in French and because the hospital serves many patients whose mastery of French is limited, we examined the working knowledge of French among a sample of patients who were recently treated; 17 of 88 (19%) were unable to communicate in French with the admission staff. Therefore, a substantial proportion of nonresponse may have been the result of language problems.

Among the 501 respondents who provided demographic information, the mean age was 58.6 years (SD=18.9, range=15–95, quartiles=44, 61, 74 years), and 243 respondents (48.5%) were women. There were 219 outpatients and 305 inpatients (4 subjects removed the study number from the questionnaire and could not be traced); 193 patients received medical rehabilitation, 156 patients received orthopedic rehabilitation, 94 patients received cardiorespiratory rehabilitation, and 81 patients received neurologic rehabilitation.

Item Distributions

Six questionnaires contained only open comments, leaving 522 questionnaires for quantitative analysis. Missing responses per item ranged from 3.1% (“ability of physical therapist to put you at ease and reassure you”) to 53.1% (“parking facilities”). For most items, missing responses were more frequent among inpatients than among outpatients (data not shown). On a scale of 1 (“poor”) to 5 (“excellent”), most item means lay between 3.5 and 4, and most standard deviations lay around 1.

Initial Factor Analysis

A principal component analysis of the 24 closed-format items yielded 3 factors that explained 60% of the total variance. The items regrouped rather logically: 15 items pertaining to physical therapy had high loadings on the first factor, 4 items dealing with the admission process formed the second factor, and 5 items dealing with the physical environment formed the third factor.

Summary Scores

Fifteen items contributed to the measurement of satisfaction with treatment, but some items appeared to be redundant. Item removal was based on preserving content validity and scale internal consistency. Finally, 5 items were retained (items 4–8, Tab. 1). The 5-item treatment subscale was internally consistent (Cronbach α=.90) and correlated with the initial 15-item scale at .97. Using similar methods, the subscales measuring the admission process (items 1–3) and the logistics of treatment (items 9–12) were reduced to 3 and 4 items, respectively. The admission subscale was fully applicable only to outpatients, because inpatients rarely deal with the scheduling and admission procedures.

Distributions of 14 Final Items Probing Patient Opinions About Physical Therapy and Results of Factor Analysis (Loadings

Translation by the authors. Not a validated English-language version of the instrument.

Not applicable to all respondents (eg, inpatients).

Answers ranged between “certainly not” (1) and “yes, certainly” (5).

Distributions of 14 Final Items Probing Patient Opinions About Physical Therapy and Results of Factor Analysis (Loadings

Translation by the authors. Not a validated English-language version of the instrument.

Not applicable to all respondents (eg, inpatients).

Answers ranged between “certainly not” (1) and “yes, certainly” (5).

Distributions of the 12 items included in the 3 domain-specific scales as well as the items for general evaluation and willingness to recommend are reported in Table 1. All 3 scales had means between 60 and 70, standard deviations of about 20, no floor effects and negligible ceiling effects, and internal consistency coefficients (Cronbach α) in the .8 to .9 range ( Tab. 2). The correlation coefficients (Pearson r) were: .52 between the treatment and admission subscales, .61 between the treatment and logistics subscales, and .53 between the admission and logistics subscales. Correlation coefficients between the original longer scale and the corresponding brief scale were high for all domains of measurement.

Distributions of Four Summary Scores Measuring Patient Satisfaction With Physical Therapy, Geneva, Switzerland, 1999

This scale is fully applicable only to outpatients.

Distributions of Four Summary Scores Measuring Patient Satisfaction With Physical Therapy, Geneva, Switzerland, 1999

This scale is fully applicable only to outpatients.

Factor Analysis of Reduced Item Set

A factor analysis restricted to the 12 retained items confirmed a 3-dimensional structure; the 3 factors explained 67% of total variance. The stability of the factor structure was verified by a bootstrap procedure. A 3-dimensional structure was confirmed in 26 of the 30 bootstrap subsamples, and, in the remaining 4 cases, the third eigenvalue was between 0.96 and 0.99. The main loading was on the intended factor in all cases but one, for an overall success rate of 359/360 (the item pertaining to “comfort” had a higher loading on the “admission” scale once in 30 runs). The main loading was greater than 0.50 in 358 cases out of 360. Furthermore, the same 3-factor structure was obtained in men and women and in inpatients and outpatients (data not shown).

Global Scale

The global assessment item and the future intentions items were considered for a 3-item global scale. Because the 2 future intentions items were highly correlated (Pearson r=.78), one was dropped, and a 2-item global scale consisting of the global assessment item and the willingness-to-recommend item was constructed ( Tab. 2). This scale was most closely correlated with the treatment subscale (.82) and less so with the logistics (.57) and admission (.43) subscales. When included in the factor analysis, these global items loaded onto the treatment factor ( Tab. 1). Therefore, the final instrument comprised 14 items ( Appendix 2).

Validation by Future Intentions

Most respondents agreed that they would recommend the hospital physical therapy service to others, and most said that they would be willing to return if they needed physical therapy again ( Tab. 3). Both future intentions items were linearly associated with mean satisfaction scores for all 3 subscales. Both future intentions items were more strongly associated with the treatment score than with the admission or logistics subscale scores, as evidenced by the corresponding Pearson R 2 coefficients.

Validation of Standardized Satisfaction Scores by Future Intentions

Validation of Standardized Satisfaction Scores by Future Intentions

Validation by Comments to Open-Ended Questions

Of the 528 respondents, 291 respondents (55%) provided a comment on the reason why they would or would not come back if they needed physical therapy. This comment was clearly positive for 176 respondents (33%), negative for 31 respondents (6%), and neutral for 87 respondents (16%); the neutral comments mostly pertained to the existence of an alternative source of care. Furthermore, 287 respondents (54%) commented on the strong points of the department; 15 (3%) of these respondents stated that there was no strong point. Finally, 209 respondents (40%) commented on a weak point; 55 (10%) of these respondents stated that there was no weak point.

The number of positive and negative comments was associated with mean scores on all satisfaction items; this was consistent with our assumptions about validity being present ( Tab. 4). Positive and negative comments on open-ended questions explained a greater proportion of variance in the global score (28%, based on the adjusted R 2 statistic) than for the treatment subscale (23%), logistics subscale (13%), or admission subscale (5%). Negative comments predicted lower satisfaction scores regardless of whether the patient made no positive comment, 1 positive comment, or 2 positive comments. Positive comments were associated with higher satisfaction scores regardless of the number of negative comments (eg, treatment subscale score, Figure).

Mean scores of treatment satisfaction, according to number of positive or negative comments to open-ended questions, among patients receiving physical therapy, Geneva, Switzerland, 1999.

Validation of Standardized Satisfaction Scores by Comments to Open-Ended Questions: Mean Differences in Scores, With 95% Confidence Intervals (CI), Compared With Respondents Who Gave No Positive or Negative Comment

Validation of Standardized Satisfaction Scores by Comments to Open-Ended Questions: Mean Differences in Scores, With 95% Confidence Intervals (CI), Compared With Respondents Who Gave No Positive or Negative Comment

Satisfaction Across Subgroups

Only for the logistics subscale were women's scores lower than men's scores ( Tab. 5). Younger patients tended to be less satisfied than older patients for the admission and logistics subscales. Outpatients rated logistics lower than inpatients did. We presume this is because they encountered difficulties with physical access that did not concern outpatients. Across types of treatment received, the only difference concerned the admission subscale score, which was lower for patients who received orthopedic treatments than for patients who received cardiorespiratory care.

Scores for Satisfaction With Physical Therapy in Subgroups of Patients, Geneva, Switzerland, 1999

Test for linear trend.

Scores for Satisfaction With Physical Therapy in Subgroups of Patients, Geneva, Switzerland, 1999

Test for linear trend.

Discussion

The 14-item scale presented in this article, in our opinion, can be used for assessment of patient satisfaction with physical therapy and for both outpatients and inpatients; however, the test-retest reliability of the scores is not yet known. Four scores pertaining to distinct dimensions of patient satisfaction can be computed: a treatment subscale score, an admission subscale score, a logistics subscale score, and a global assessment subscale score. In our sample, all scores had acceptable ceiling effects and no floor effects, which suggests that this instrument is reasonably well suited to the population served by our physical therapy center.

Although our survey instrument had only 14 items, all 4 subscales had satisfactory internal consistency (Cronbach α coefficients). The validity of data obtained with the scales was supported by a stable and interpretable factorial structure, by a strong correlation between assessments of past experience and intentions of future behavior, and by correlations with the number of positive and negative comments to open-ended questions. Furthermore, patient characteristics had only a minor influence on the scores, with the exception of young age. These findings suggest that, in future surveys, measures designed to control for case mix can be kept very simple. The tendency of older patients to give higher satisfaction ratings has been observed repeatedly by other researchers. 20 Standard deviations of all scores were approximately 20, which in our opinion should facilitate the interpretations of survey results: a difference of x points means about the same thing on any of the subscales. No simple interpretation guide can be provided at this point, but some authors in the psychometric literature suggest that a difference greater than 0.8 standard deviation should be considered large (16 points), 0.5 to 0.8 standard deviation should be considered moderate (10–16 points), and 0.2 to 0.5 standard deviation should be considered small (4–10 points). 21 For example, the difference in mean treatment scores between patients who would recommend our facility “certainly” versus “probably” (81 versus 66, or 0.75 standard deviation units) would be moderate, whereas the difference between the group that would certainly recommend and those who would not recommend (81 versus 30, or 2.5 standard deviation units) would be considered very large.

With few exceptions, the final content of the questionnaire addressed generic issues that are of interest in any health care encounter. This finding brings into question our hypothesis that physical therapy would require a specific instrument. The most notable exception to this pattern was the item on the “feeling of security,” which is not commonly found in medical satisfaction questionnaires. The importance of feeling physically secure was noted by both staff and patients during pretests, and the inclusion of this item may render the whole instrument more relevant and acceptable in routine use.

Limitations of the Study

Despite what we would consider the generally good psychometric properties of this instrument, several limitations should be noted. We did not examine test-retest reliability for the various items and scores. The brevity of the questionnaire, although desirable, may cause concern in some circumstances, because brevity may conflict with the content validity of the instrument. Some items that we eliminated based on psychometric criteria, such as timeliness or respect for confidentiality, may nevertheless be of special concern, and the “long” version of this instrument ( Appendix 1) or another more comprehensive questionnaire may prove more appropriate. Furthermore, because the questionnaire was developed at a single facility, confirmation of its relevance elsewhere would be useful.

Another issue is the number of dimensions of satisfaction measured by the instrument. The 3-dimensional structure was derived empirically by exploratory factor analysis (the fourth global dimension was not based on factor analysis, but was defined beforehand). This structure should be confirmed in independently derived samples. The dimensions identified in this study are comparable to those found in a previous study. 15 Oermann et al 15 identified dimensions labeled “efficacy,” “convenience,” “comfort,” and “overall,” which loosely match our dimensions called “treatment,” “admission,” “logistics,” and “global.” In contrast, another study 13 described conceptually quite different dimensions, labeled “enhancers,” “detractors,” “location,” and “cost”; still others have found that all items belonged to a single dimension of patient satisfaction. 14

Why the number of dimensions varies among satisfaction instruments is a complex issue. First, several rules exist for choosing the number of factors, and they do not always concur. Second, the number of dimensions identified is positively correlated with the number of initial items included in the analysis. Third, independence of concepts is not always sufficient for multiple dimensions to emerge—there must be independence of distributions in the studied sample. For example, if therapists who are good at interpersonal aspects of care were also good technically, these 2 conceptually distinct dimensions could not appear as distinct dimensions in the data. Fourth, the “halo” effect, whereby the global impression influences responses to specific items, may be intensified by some features of the questionnaire (eg, grouping all items into a single block) or of the survey (eg, allowing too little time to think about responses). Conversely, other technical features of the questionnaire, such as positive and negative wordings of items, may produce falsely independent dimensions. All these reasons may explain the diversity of factorial structures described in the literature. 13– 15

Another problem we encountered was a high proportion of missing responses for some items. This is understandable for items that do not apply to all patients (such as items related to the admission process, which were not relevant for inpatients), but it raises concerns about the general suitability of the instrument. Furthermore, the overall response rate to the survey was lower than we had hoped. Although this low response rate may have been due to a substantial proportion of non-French speakers in our patient population, improving data collection methods is a priority. Shortening of the questionnaire (from 27 to 14 items) may help achieve higher response rates in future surveys. Previous research indicates that late respondents tend to give lower satisfaction ratings than early respondents. 22

The instrument has been developed in French, and the English translation included in this article has not been revalidated. Those interested in using the questionnaire in a non-French translation should conduct a formal adaptation of the instrument. 23

Conclusion

We have developed multidimensional satisfaction questionnaire intended for patients receiving physical therapy. The instrument has some good psychometric properties, and it joins a recent but expanding pool of questionnaires specifically destined for patients receiving physical therapy. These satisfaction questionnaires could prove useful for monitoring quality of care, complementing instruments that measure other relevant patient outcomes, such as functional recovery. 24