www.sciencedirect.com Open in urlscan Pro
162.159.137.70 Public Scan

Back to summary

Submitted URL:
https://u44122502.ct.sendgrid.net/ls/click?upn=u001.o6vyTBLb80epiNVqC3LNJBDGcCoa4AhyLeC-2BHQ-2FRDdyN1zjKH6lgET0suTQvwXZ1tAS9mm8c1h...
Effective URL:
https://www.sciencedirect.com/science/article/pii/S2590109524000612
Submission: On October 17 via manual (October 17th 2024, 1:32:37 am UTC) from US — Scanned from DE

Form analysis
1 forms found in the DOM

GET /search#submit

<form class="QuickSearch" action="/search#submit" method="get" aria-label="form">
  <div class="search-input">
    <div class="search-input-container search-input-container-no-label"><label class="search-input-label u-hide-visually" for="article-quick-search">Search ScienceDirect</label><input type="search" id="article-quick-search" name="qs"
        class="search-input-field" aria-describedby="article-quick-search-description-message" aria-invalid="false" aria-label="Search ScienceDirect" placeholder="Search ScienceDirect" value=""></div>
    <div class="search-input-message-container">
      <div class="search-input-validation-error" aria-live="polite"></div>
      <div id="article-quick-search-description-message"></div>
    </div>
  </div><button type="submit" class="button u-margin-xs-left button-primary small button-icon-only" aria-disabled="false" aria-label="Submit search"><svg focusable="false" viewBox="0 0 100 128" height="20" class="icon icon-search">
      <path
        d="M19.22 76.91c-5.84-5.84-9.05-13.6-9.05-21.85s3.21-16.01 9.05-21.85c5.84-5.83 13.59-9.05 21.85-9.05 8.25 0 16.01 3.22 21.84 9.05 5.84 5.84 9.05 13.6 9.05 21.85s-3.21 16.01-9.05 21.85c-5.83 5.83-13.59 9.05-21.84 9.05-8.26 0-16.01-3.22-21.85-9.05zm80.33 29.6L73.23 80.19c5.61-7.15 8.68-15.9 8.68-25.13 0-10.91-4.25-21.17-11.96-28.88-7.72-7.71-17.97-11.96-28.88-11.96S19.9 18.47 12.19 26.18C4.47 33.89.22 44.15.22 55.06s4.25 21.17 11.97 28.88C19.9 91.65 30.16 95.9 41.07 95.9c9.23 0 17.98-3.07 25.13-8.68l26.32 26.32 7.03-7.03">
      </path>
    </svg></button><input type="hidden" name="origin" value="article"><input type="hidden" name="zone" value="qSearch">
</form>

Text Content

JavaScript is disabled on your browser. Please enable JavaScript to use all the
features on this page. Skip to main contentSkip to article
ScienceDirect
* Journals & Books

* Help
* Search

My account
Sign in

* View PDF

Search ScienceDirect

OUTLINE

1. Abstract
2.
3. KEYWORDS
4. List of abbreviations
5. Methods
6. Results
7. Discussion
8. Conclusions
9. Disclosures
10. References

Show full outline

FIGURES (1)

TABLES (5)

1. Table 1
2. Table 2
3. Table 3
4. Table 4
5. Table 5

ARCHIVES OF REHABILITATION RESEARCH AND CLINICAL TRANSLATION

Available online 11 August 2024, 100363
In Press, Corrected ProofWhat’s this?

ORIGINAL RESEARCH
NONRESPONSE BIAS ON INPATIENT REHABILITATION HOSPITALS’ EXPERIENCE OF CARE
QUALITY MEASURE SCORES

Author links open overlay panelAllen Walter Heinemann PhD a, Anne Deutsch RN,
PhD, CRRN a b, Dave Brewington PhD a, Linda Foster PT c, Karon Frances Cook PhD
d, Ana Miskovic BA a, David Cella PhD d
Show more
Outline
Add to Mendeley
Share
Cite
https://doi.org/10.1016/j.arrct.2024.100363Get rights and content
Under a Creative Commons license
open access

ABSTRACT

OBJECTIVE

To describe the magnitude of nonresponse bias on inpatient rehabilitation
facility (IRF) experience of care survey data in patients with neurologic
disorders.

DESIGN

Cohort study of patients at 2 IRFs. Patients reported experience of care via an
IRF-administered survey as part of routine operations approximately 2 weeks
after discharge. A partially overlapping sample of research participants
completed a similar survey approximately 2 weeks and 30 days after discharge.

SETTING

Two inpatient rehabilitation facilities.

PARTICIPANTS

Patients aged ≥18 years with neurologic disorders who were discharged from 2
IRFs.

INTERVENTIONS

None.

MAIN OUTCOME MEASURES

Experience of care data collected via an IRF Survey (self-report or proxy
responses) and a Research Survey (self-report only).

RESULTS

Of the 1055 patients admitted during the study period who met the age and
diagnosis criteria, 490 (46.4%) completed one or both of the surveys. Of the 325
IRF Survey respondents, 202 were self-report, 99 were proxy respondents, and 24
were unknown respondents. Only patients completed the Research Survey (N=285).
One hundred twenty patients completed both surveys, of which 7 were proxy IRF
Survey respondents. IRF Survey respondents had higher cognitive function than
nonrespondents; patients with spinal cord injuries were more likely to complete
the IRF Survey than other patients. There were no differences in the proportions
of patients answering favorably on the IRF Survey (all respondents) compared
with the Research Survey, except for physician communication and discharge
information. Mutual information analysis revealed agreement between the scores
produced by the 2 data sources.

CONCLUSIONS

There were subtle, potentially important differences in quality measure results
across surveys, reflecting the extent to which patients are encouraged to
complete experience of care surveys. There was higher agreement on questions
about global hospital perceptions than specific aspects of patients’ experience.

KEYWORDS

Neurological rehabilitation
Nonresponse bias
Outcome assessment (health care)
Patient reported outcomes
Quality indicators
Rehabilitation

LIST OF ABBREVIATIONS

ACA
Affordable Care Act
CAHPS
Consumer Assessment of Healthcare Providers and Systems
HCAHPS
Hospital Consumer Assessment of Healthcare Providers and Systems
IRF
inpatient rehabilitation facility
PAC
postacute care
Inpatient rehabilitation facilities (IRFs) provide comprehensive, intensive
medical rehabilitation services for patients who experience a major illness or
injury that results in functional limitations with a goal of optimizing
patients’ functional abilities and maximizing community participation.1 Gans2
identified experience of care as a key aspect of IRF quality of care. The
Centers for Medicare and Medicaid Services’ IRF Quality Reporting Program
sponsored research to develop an IRF experience of care survey, which could be
used to develop experience of care quality measures.3
If IRF experience of care quality measures were implemented under an
accountability program, understanding potential nonresponse bias would be an
important consideration, because many patients in IRFs have neurologic
conditions such as stroke, spinal cord injury, and brain injury that result in
motor and/or cognitive functional limitations that may affect a patient's
ability to respond to experience of care surveys. Further, patients with both
motor and cognitive limitations may be at higher risk for receiving lower
quality care during an IRF stay because of their complex care needs and the
involvement of multiple clinicians from different disciplines. Patients often
have cognitive and communication limitations that affect patient-staff
communication and are dependent on care staff for daily needs.
Previous research has focused on short-stay acute care hospitals’ experience of
care survey data used to calculate quality measures for the inpatient acute care
hospital value-based purchasing program.4 These studies have found moderate,
selective nonresponse rates that may translate to a small amount of nonresponse
bias in hospital-level data that are not case-mix adjusted.5,6 Case-mix
adjustment eliminates most nonresponse bias.5,6 Simon et al7 found no evidence
of nonresponse bias for mental health providers with a response rate of 33.8%.
Similarly, Fowler et al8 tested various administrative modes for the Consumer
Assessment of Healthcare Providers and Systems (CAHPS) surveys for primary care
practices and found little evidence of nonresponse bias despite response rates
that varied from 20%-40%.
We had the opportunity to explore nonresponse bias, including the use of proxy
responses, in IRF experience of care survey data for a sample of patients from 2
IRFs admitted in 2015. We matched data from 2 sources (1) routinely collected
experience of care data from patients discharged from 2 IRFs (“IRF Survey”) and
(2) research project experience of care data (“Research Survey”).9,10 A team of
researchers expended considerable effort to obtain the Research Survey data from
research participants after discharge. This contrasts with the IRFs’ real-world
efforts to obtain responses to a mailed survey, which are typically more
limited. Access to both routinely collected IRF Survey data and Research Survey
data offers the ability to compare experience of care data when respondents
completed only the IRF Survey, both surveys, or only the Research Survey.
Using data from these 2 sources, this study had 5 aims (1) to evaluate the
representativeness of respondents with neurologic disorders who returned a
completed IRF Survey after discharge; (2) to compare IRF Survey responses for
self-report and proxy respondents on individual questions; (3) to compare the
association between IRF Survey and Research Survey responses for similar
experience of care questions when both surveys were completed; (4) to compare
“top-box” scores (numerator, reflecting the most favorable response options)
between IRF Survey and Research Survey respondents; and (5) to compare top-box
scores for the IRF Survey and Research Survey when both surveys were completed.

METHODS

SAMPLE

At 2 IRFs, patients reported experience of care in 2 ways (1) IRFs administered
an identical survey as part of routine operations approximately 2 weeks after
discharge; and (2) research participants completed a survey approximately 30
days after discharge. Study eligibility criteria were admission between March 25
and October 23, 2015, a primary neurologic condition requiring IRF admission,
and age ≥18 years. Research participants provided informed consent.

SURVEY INSTRUMENTS

IRF Survey: The 2 IRFs contracted with Press Ganey,11 a national vendor, to
gather patients’ perspectives of their IRF care experiences. Survey questions
addressed topics such as clinician communication, responsiveness of the staff,
support and encouragement, discharge information, cleanliness of the hospital,
and overall rating of the hospital.12 Patients rated the extent of agreement
with statements about their care using response categories that ranged from 1
(very poor) to 5 (very good). The IRF Survey has demonstrated evidence of
adequate reliability and validity.13 Proxies could complete the IRF Survey in
lieu of patients.
Research Survey: The Research Survey adapted the acute care version of the
Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS)
Survey,12 a patient experience of care survey for use in inpatient
rehabilitation. HCAHPS questions address communication with nurses and doctors,
responsiveness of hospital staff, cleanliness and quietness of the hospital
environment, pain management, communication about medicines, discharge
information, overall hospital rating, and likelihood of recommending the
hospital. The Research Survey added 3 questions about therapists’ communication
and 3 questions about patient care technicians’ communication that parallel the
HCAHPS doctor and nurse questions. Only patients completed the Research Survey.

STUDY PROTOCOL

IRF Survey: As part of hospital operations, the IRFs securely transmitted
patient contact information electronically to a vendor for all discharged
patients, after removing patients who requested not to be contacted or were
deceased. Within 2 weeks of discharge, the vendor mailed a paper survey with a
cover letter and self-addressed, stamped return envelope to all the former
patients except those who had received a paper or electronic survey in the
previous 90 days. The request to complete the survey was on the hospitals’
letterhead and included the chief executive officer's signature; the return
address was the vendor's data processing facility. IRF staff contacted
discharged patients by telephone about 1 week after the vendor received contact
data to remind patients to return the survey. For patients who did not respond
within 30 days of the original mailing, the vendor mailed a second survey.
During the study period, a total of 325 surveys (30.8%) were returned.
Research Survey: For the research project primary data collection, patient
eligibility, accrual, and retention rates were described previously.10 Briefly,
the research project sought to evaluate IRF inpatients’ willingness and ability
to complete a patient experience of care survey and the burden of completion on
patients and staff. Of the 1055 patients admitted with neurologic conditions,
781 (74%) met the study eligibility criteria, 398 (51%) of eligible patients
completed the survey at discharge, and 285 (36%) of eligible patients completed
the survey 1 month after discharge. Half of the respondents required at least 2
reminder calls; research staff made up to 4 reminder calls. Research Survey
respondents who completed the postdischarge Research Survey had significantly
higher cognitive abilities at IRF admission as evidenced by higher functional
independence measure (FIM) instrument cognitive scores14, 15, 16 than
noncompleters, and those discharged to institutional settings were less likely
to complete the study (65%) than patients discharged home (76%).

DATA LINKING

An IRF staff member linked the IRF Surveys and the Research Survey using
patients’ medical record number and admission dates.

SURVEY QUESTION MATCHING

Two investigators reviewed the IRF Survey questions as well as the Research
Survey questions and proposed matches based on item content. Project staff
reviewed the initial matches and reconciled differences in matching. Table 1
shows the matched questions from the 2 surveys.

Table 1. Spearman correlation coefficients for responses of similar Research
Survey and IRF Survey questions.

Research Study QuestionsIRF Survey QuestionsSelf-Respondents Only (N=202)All
Respondents (N=325)Empty CellEmpty CellnCorrelationPnCorrelationPDoctor
communicationHow often did your doctor listen carefully to you?Doctor's concern
for your questions and worries580.37.004710.35.003How often did your doctor
explain things in a way that was easy for you to understand?How well the doctor
explained your hospital rehabilitation program570.46<.001690.49<.001How often
did your doctor treat you with courtesy and respect?Courtesy of the
rehabilitation doctor1000.46<.0011170.46<.001Nurse communicationHow often did
your nurses explain things in a way that was easy for you to understand?How well
nurses kept you informed about your treatment and
progress990.56<.0011150.56<.001How often did your nurses treat you with courtesy
and respect?Courtesy of the nurses1000.55<.0011150.59<.001Therapist
communicationHow often did your therapists explain things in a way that was easy
for you to understand?How well the occupational therapist explained your
treatment and progress980.35<.0011140.37<.001How often did your therapists
explain things in a way that was easy for you to understand?How well the
physical therapist explained your treatment and
progress1000.39<.0011160.46<.001How often did your therapists treat you with
courtesy and respect?Courtesy of the occupational
therapist980.29.0041130.34<.001How often did your therapists treat you with
courtesy and respect?Courtesy of the physical
therapist1010.14.1651160.25.007Responsiveness of hospital staffAfter you pressed
the call button, how often did you get help as soon as you wanted it?Promptness
in responding to the call button990.60<.0011140.58<.001Support and
encouragementHow often did you feel supported and encouraged by your
rehabilitation team?Extent to which staff gave you
encouragement1000.32.0011150.37<.001Discharge informationDuring this
rehabilitation hospital stay, did the rehabilitation team provide you with the
training and information you and your family needed for your discharge?Training
given to you and your family about care after
discharge950.51<.0011110.53<.001When you left the rehabilitation hospital, were
you given all the information needed to manage your medications?How well the
nurses instructed you about caring for yourself at home (including
medications)900.35.0011050.39<.001Cleanliness of hospital environmentHow often
were your room and bathroom kept clean?Daily cleaning of your
room980.44<.0011140.46<.001Overall hospital ratingUsing any number from 0 to 10,
where 0 is the worst rehabilitation hospital possible and 10 is the best
rehabilitation hospital possible, what number would you use to rate this
rehabilitation hospital?Overall rating of care you received during your
stay970.49<.0011130.53<.001Willingness to recommend hospitalWould you recommend
this rehabilitation hospital to your friends and family?Likelihood of your
recommending our facility to others970.60<.0011130.67<.001

QUALITY MEASURE SCORE CALCULATIONS

We used the HCAHPS top-box algorithm to calculate quality measure results using
the IRF Survey and Research Survey data.17 We calculated quality measure results
as the percentage of “very good” responses on IRF Survey questions and the
percentage of “always” responses on research study questions. Consistent with
the HCAHPS approach, quality measure results were calculated using either one or
multiple responses. Quality measure score calculations that rely on response to
a single item were cleanliness of hospital environment, responsiveness of
hospital staff, support and encouragement, and overall rating of the hospital.
The remaining quality measure score calculations used responses to multiple
items: doctor communication (3 items), nurse communication (2 items), therapist
communication (2 items), and discharge information (2 items). The research
project's nurse communication measure used 3 items; however, the IRF Survey had
no analogous third question, so the quality measure results were calculated
based on the 2 similar questions. The IRF Survey questions specified the type of
therapists as physical therapist or occupational therapist, whereas the research
study questions asked about therapists without specifying the type of therapist.
For comparison to the general therapist questions in the Research Survey, the
top-box percentages of the IRF Survey were defined as responses of either (1)
“very good” for both physical therapy questions; (2) “very good” for both
occupational therapy questions; or (3) “very good” for all 4 therapy questions
(2 physical therapy and 2 occupational therapy questions). This top-box
comparison was compared to the 2 broad therapy questions from the Research
Survey. We also compared the top-box percentage of the 2 IRF Survey occupational
therapy questions with the top-box percentage of 2 broad therapy questions from
the Research Survey. When comparing the Research Survey data (2 broad therapy
questions) with the IRF Survey data, the comparison was limited to 2 questions,
and the IRF Survey defined top-box as “very good” for both physical therapy
questions, “very good” for both occupational therapy questions, or “very good”
for all 4 therapy questions (2 physical therapy and 2 occupational therapy
questions).

STATISTICAL ANALYSIS

The demographic and clinical characteristics of patients with neurologic
conditions who completed the IRF Survey were compared with those who did not
complete the survey using Pearson chi-square tests. Patient characteristics were
then compared by the source of information (patients or proxy respondents) using
the Mann-Whitney U test to identify statistically significant differences
between self and proxy respondents on IRF Survey responses. A Bonferroni
correction of P<.003 was applied to account for 14 multiple comparisons.
The association between responses to the IRF Survey questions and analogous
research study questions were calculated using Spearman correlation coefficients
because of the ordinal nature of the rating scales.
After calculating quality measure scores, Pearson chi-square tests were used to
test for differences between self- and proxy-reported responses. A mutual
information technique18 was used to estimate the shared information between the
IRF Survey and Research Survey responses. The technique approximates a
chi-square statistic, allowing estimation of statistical significance. In
situations the mutual information value is statistically significant and the
calculated agreement level is greater than the disagreement level, the questions
are considered to provide congruent (or concordant) information. Likewise, if
the mutual information value is statistically significant and the agreement
level is lower than the disagreement level, then the 2 instruments provide
discordant information. A Bonferroni correction of P<.005 was applied to account
for 11 multiple comparisons. After assessing the shared survey information, the
McNemar test was used to identify significant differences in the proportions of
IRF Survey and Research Survey top-box scores.19
Northwestern University's Institutional Review Board approved this study. IRF
operations staff provided deidentified, matched data records to research staff
to maintain patient privacy.

RESULTS

Of the 1055 patients with neurologic conditions admitted during the study period
who met the age criterion, 490 (46.4%) completed one or both of the surveys.
Figure 1 displays the unique and overlapping sets of survey respondents. A total
of 325 IRF Surveys were completed (30.8%), of which 202 were self-report, 99
were proxy respondents, and 24 were unknown respondents. Only patients completed
the Research Survey (N=285). Both surveys were completed by 120 respondents, of
which 7 were proxy respondents for the IRF Survey and self-reported for the
Research Survey and 10 were unknown respondents for the IRF Survey and
self-reported for the Research Survey. Notably, none of the 273 patients (or
their proxies) who were ineligible for participation in the Research Survey
responded to the IRF Survey, suggesting nonresponse bias.
1. Download: Download high-res image (335KB)
2. Download: Download full-size image

Fig 1. Unique and overlapping sets of survey completion by admitted patients
with neurologic disorders (N=1055).

IRF SURVEY

Table 2 shows that that the IRF Survey response rate was unrelated to age, sex,
ethnicity, length of stay, marital status, discharge living arrangement, or
admission self-care and mobility abilities (FIM instrument scores). Response
rate was related to race, discharge location, and cognitive abilities (FIM
instrument cognition scores) such that IRF Survey respondents had higher
cognition than did nonrespondents. Response rates were also related to primary
medical condition such that respondents were more likely to have spinal cord
injuries and nonrespondents were more likely to have traumatic brain injuries
and strokes.

Table 2. Comparison of IRF Survey respondents and nonrespondents, patients with
neurologic conditions (N=1055).

VariableSurvey ReturnedSurvey Not ReturnedPN=325N=730Mean age ± SD
(y)61.1±18.362.5±19.0.257 Range18-9118-98Sex.240 Male192 (69%)403
(55%) Female133 (31%)327 (45%)RaceN=152N=257.002 White128 (84%)175
(68%) Black/African American17 (11%)57 (22%) Asian2 (1%)7 (3%) More than once
race0 (0%)6 (2%) Other5 (4%)12 (5%)EthnicityN=304N=679.260 Hispanic10 (3%)35
(5%) Not Hispanic or Latino294 (97%)644 (95%)Marital statusN=323N=726.057 Single
(never married)78 (24%)176 (24%) Married173 (54%)341 (47%) Divorced20 (6%)79
(11%) Widowed49 (15%)115 (16%) Separated3 (1%)15 (2%)Primary
impairment Traumatic SCI/nontraumatic SCI87 (27%)121 (17%)<.001 TBI/nontraumatic
brain injury70 (21%)186 (45%) Stroke120 (37%)328 (45%) Other48 (15%)95
(13%)Interrupted stay.030 Yes19 (6%)21 (3%) No306 (94%)709 (97%)Discharge
location<.001 Home172 (53%)279 (38%) Short-term acute care hospital40 (4%)88
(12%) Skilled nursing facility49 (15%)171 (23%) Home health organization60
(25%)185 (25%) Hospice (home)–1 (<1%) Intermediate care–5 (1%) Another IRF2
(1%)– Not listed1(<1%)1(<1%)Discharge living arrangement.390 Alone7 (4%)18
(7%) Family/relatives158 (92%)249 (89%) Friends2 (1%)6 (2%) Attendant1 (1%)2
(1%) Other4 (2%)4 (1%)Mean length of stay ± SD
(d)22.72±15.9420.7±16.1.059 Range1-1181-171 Median1917Mean FIM cognition score ±
SD27.4±7.023.9±8.1<.001 Range5-355-35 Median2925 Score <21137373 Score
21-2790215 Score 28-3598142Mean FIM mobility score ±
SD9.4±5.498.7±5.5.180 Range0-261-30 Median98 Score < 8139356 Score
8-12102198 Score > 1284176Mean FIM self-care score ±
SD16.8±7.216.5±7.4.061 Range5-325-35 Median1716 Score <21130313 Score
21-27123224 Score 28-3572193

NOTE: Response categories were combined for the following variables for
analysis: race (White vs Black vs all other), marital status (married vs all
other), occupational status (working vs all other), discharge location (home vs
all other), and discharge living arrangement (alone vs all other).
Abbreviations: SCI, spinal cord injury; TBI, traumatic brain injury.
Table 3 shows that proxy response was unrelated to the patient's age, race,
ethnicity, education, marital status, primary impairment, discharge living
arrangement, or length of stay. However, patients for whom proxies responded had
lower self-care, mobility, and cognition abilities (ie, lower FIM instrument
scores) than did self-respondents. Proxies were more likely to be the respondent
for male patients, patients with interrupted stays, and those discharged to
acute care hospitals or skilled nursing facilities.

Table 3. IRF Survey: comparison of self and proxy respondents (N=301).*

Empty CellSelf-RespondentProxy RespondentPEmpty Celln=202n=99Empty CellMean age
± SD (y)60.7±17.861.4±19.5.756 Range18-9118-91 Median63.565Sex.026 Male108
(53.5%)67 (67.7%) Female94 (46.5%)32 (32.3%)Race.478 White148 (74.4%)68
(69.4%) Black/African American35 (17.6%)18 (18.4%) Asian2 (1.0%)5 (5.1%) Other14
(7.0%)7 (7.1%)Ethnicity>999 Hispanic6 (3%)3 (3%) Not Hispanic or Latino185
(97%)89 (97%)Education.839 <HS3 (3%)– HS/GED22 (18%)3 (19%) Some college36
(30%)4 (25%) College degree59 (49%)9 (56%)Marital status.363 Single (never
married)49 (24.4%)26 (26.3%) Married103 (51.2%)57 (57.6%) Divorced11 (5.5%)4
(4.0%) Widowed34 (16.9%)11 (11.1%) Separated4 (2.0%)1 (1.0%)Primary
impairment.434 TSCI/NTSCI54 (26.7%)29 (29.3%) TBI/NTBI40 (19.8%)21
(21.2%) Stroke73 (36.1%)39 (39.4%) Other35 (17.3%)10 (10.1%)Interrupted
stay.032 Yes8 (4%)11 (11%) No194 (96%)88 (89%)Discharge location.014 Home115
(57.2%)41 (41.4%) Short-term general hospital17 (8.5%)22 (22.2%) Skilled nursing
facility26 (12.9%)18 (18.2%) Home health organization42 (20.9%)17
(17.2%) Another IRF1 (.5%)1 (1.0%)Discharge living arrangement.942 Alone5
(4.3%)1 (2.4%) Family/relatives106 (92.2%)38 (92.7%) Friends2 (1.7%)0
(0.0%) Attendant–1 (2.4%) Other2 (1.7%)1 (2.4%)Mean length of stay ±
SD22.3±15.424.8±19.6.026 Range1-901-118 Median1820 1-14 d29 (29.3%)75
(37.1%).365 15-23 d28 (28.3%)55 (27.1%) >23 d42 (42.4%)72 (35.6%)Mean FIM
cognition score ± SD28.3±6.525.4±7.9<.001 Range5-3510-35 Median3027 <2158
(58.6%)69 (34.2%) 21-2719 (19.2%)64 (31.7) 28-3522 (22.2%)69 (30.2%)Mean FIM
mobility score ± SD10.0±5.48.0±5.7.005 Range1-260-26 Median97 <851 (16.9%)78
(25.9%) 8-1234 (34.3%)61 (30.2%) >1214 (14.1%)63 (31.2%)Mean FIM total
self-care score ± SD18.1±7.214.5±6.8<.001 Range1-265-29 Median1914 <1554
(54.5%)65 (32.2%) 15-2233 (33.3%)78 (38.6%) >2212 (12.1%)59 (29.2%)

NOTE. Data from 14 surveys were excluded because we did not know whether the
patient or a proxy completed the survey. Response categories were combined for
the following variables for analysis: race (White vs Black vs all other),
marital status (married vs all other), occupational status (working vs all
other), discharge location (home vs all other), and discharge living arrangement
(alone vs all other).
Abbreviations: GED, general educational development test; HS, high school; NTBI,
nontraumatic brain injury; NTSCI, nontraumatic spinal cord injury; TBI,
traumatic brain injury; TSCI, traumatic spinal cord injury.
⁎
Significance was calculated using the Welch-Satterthwaite method; variances are
not equal.
Table 1 summarizes item-level correlations between self-report and all
respondents using Spearman's ρ for 16 survey questions that were analogous on
the IRF Survey and the Research Survey. When self-reported data only were
examined, values ranged from 0.14 (therapists treat you with courtesy/courtesy
of physical therapist) to 0.60 (responsiveness of hospital staff, willingness to
recommend the hospital). When all data were examined, including proxy- and
self-reported and unknown source data, correlations were generally of the same
or greater magnitude.
Table 4 reports quality measure results calculated from the IRF Survey sample
data and Research Survey sample data. The quality measure results based on the
Research Survey data indicate better experiences than scores from the IRF Survey
from self-respondents, except in the case of “responsiveness to hospital staff”
and “overall hospital rating.” The quality measure results based on all
respondents to the IRF Survey were 0-5 percentage points lower than quality
measure results based on self-respondents, except for “nurse communication” and
“therapist communication,” suggesting a more favorable perception by proxies.

Table 4. Quality measure scores for IRF Survey and Research Survey data for
similar questions.

Empty CellResearch Survey Data (Self-Respondents)IRF Survey Data
(Self-Respondents)IRF Survey Data (All
Respondents*)TopicsNumeratorDenominator%NumeratorDenominator%NumeratorDenominator%Doctor
communication1732856163116549618552Nurse
communication167277601062015316931154Therapist
communication†220277791582006625131979Therapist communication
(PT)---1482017423431973Therapist communication
(OT)---1391947222030971Cleanliness of hospital
environment189281671171985917731656Responsiveness of hospital
staff8928132961974914931447Overall hospital
rating208280741521987722831672Willingness to recommend
hospital239280851581978024731578Support and
encouragement219282781431977321731469Discharge
information14528152811984312730941

Abbreviations: OT, occupational therapy; PT, physical therapy.
⁎
Includes self-respondents, with proxy, and unknown respondents.
†
The Research Survey included questions about therapists broadly, and the IRF
Survey included separate questions about PT and OT. To compare the Research
Survey responses with the specific IRF Survey responses, we defined IRF Survey
top-box as “very good” for both PT questions, “very good” for both OT questions,
or “very good” for all 4 therapy questions (2 PT and 2 OT questions).
Data from patients who completed both surveys were examined to better understand
the response differences between the 2 surveys. Adjusting for multiple
comparisons with the McNemar test, table 5 summarizes no differences in the
proportions of patients answering favorably on the IRF Survey compared with the
Research Survey, except for doctor communication (regardless of source) and
discharge information (only when all respondents were included in the IRF
Survey). Favorable ratings for doctor communication ranged from 42% (n=35) based
on the IRF Survey to 61% (n=51) based on the Research Survey data (bottom half
of table 5 in the doctor communication and discharge information rows).
One-quarter of the respondents (n=21) who reported less favorably on the IRF
Survey reported more favorably on the Research Survey. Five respondents (6%) who
reported unfavorably on the Research Survey reported favorably on the IRF
Survey.

Table 5. Comparisons between paired Research Survey and patient experience of
care top-box measures (N=120).

TopicsResearch SurveyIRF SurveyMcNemar Test PAgreementDisagreementMutual
InformationPEmpty CellNumerator%Numerator%DenominatorEmpty CellEmpty CellEmpty
CellEmpty CellEmpty CellSelf−respondents onlyDoctor
communication44623245101<.0010.21−0.140.06.003Nurse
communication6364565799.1890.50−0.250.26<.001Therapist
communication*79777977102>.9990.25−0.160.10<.001Therapist communication
(PT)*77757675102.6780.25−0.160.10<.001Therapist communication
(OT)*8081727399.4050.25−0.250.26<.001Responsiveness of hospital
staff3838494999.0890.37−0.220.15<.001Support and
encouragement83827574101.1080.16−0.110.05<.001Discharge
information5054404393.0300.24−0.170.07<.001Cleanliness of hospital
environment6970596098.0890.30−0.190.11<.001Overall hospital
rating7981828498.7740.34−0.150.19<.001Willingness to recommend
hospital8889858699.2890.27−0.100.17<.001All respondents†Doctor
communication51613542118<.0010.22−0.150.08<.001Nurse
communication71626154117.0640.53−0.250.27<.001Therapist
communication*89759177118.6900.27−0.160.11<.001Therapist communication
(PT)*89758774118.8390.33−0.180.14<.001Therapist communication
(OT)*87768473115.7000.26−0.160.10<.001Responsiveness of hospital
staff43375749114.0520.37−0.220.15<.001Support and
encouragement93808472115.0890.21−0.140.08<.001Discharge
information59544440116.0030.27−0.190.08<.001Cleanliness of hospital
environment79696657114.0370.25−0.180.07<.001Overall hospital
rating92819180113.8040.34−0.160.18<.001Willingness to recommend
hospital100879683113.1800.35−0.120.24<.001

Abbreviations: OT, occupational therapy; PT, physical therapy.
⁎
The Research Survey included questions about therapists broadly and the IRF
Survey included separate questions about PT and OT. To compare the Research
Survey responses with the specific IRF Survey responses, we defined IRF Survey
top−box as “very good” for both PT questions, “very good” for both OT questions,
or “very good” for all 4 therapy questions (2 PT and 2 OT questions).
†
Includes self−, with proxy, and unknown respondents.
Table 5 also reports comparison of paired quality measure scores using the IRF
Survey for self-respondents and all respondents with Research Survey data.
Mutual information analysis revealed significant shared information for all
measures (far right column). Local mutual information agreement levels for all
measures were higher than the local disagreement levels, indicating agreement
between the scores produced by the 2 data sources.

DISCUSSION

This study compared experience of care data reported by patients with neurologic
conditions from 2 IRFs using 2 different data sources: routine IRF data
collection and a Research Survey. Across the 2 data sources, we observed subtle,
potentially important differences on quality measure scores that may be
attributable to motor and cognitive limitations and reflect the extent to which
patients are encouraged to provide hospital ratings. Nonresponse bias may be
more important in IRFs than general acute hospitals given the higher rates of
motor and cognitive limitations in patients in IRFs. We observed higher
agreement for questions about global aspects of hospital performance than
specific aspects of patients’ experience. Efforts to develop quality measures
for IRFs should consider as risk adjusters some of the variables associated with
top-box variations in this study.
Our results are consistent with previous research examining experience of care
survey data from patients discharged from short-stay acute care hospitals, which
found moderate, selective nonresponse bias prior to risk adjustment.5,20 In
contrast, other studies examining data for mental health providers7 and primary
care providers8 did not find nonresponse bias, despite low-response rates.
We noted differences in proxy responses compared with patient responses, which
may indicate the need for risk adjustment for type of respondent. Response rates
for the IRF Survey varied for some patient characteristics, including race,
discharge location, cognitive function, and primary medical condition. Efforts
that improve response rates will result in quality measure results that are more
generalizable to IRFs’ entire patient populations; they also may be less
favorable.
As noted by Gans,2 the patient's voice represents an important aspect of quality
measurement for inpatient rehabilitation. If quality measures based on
experience data are implemented for IRFs, consideration of nonresponse and proxy
bias will be important. To allow more robust comparisons, future research should
collect data from a larger sample of IRFs and simultaneously collect responses
from patients and their proxies. In addition, research focused on case-mix
adjustors is needed to determine the extent to which nonresponse bias may be
attenuated with risk adjustment.

STUDY LIMITATIONS

Readers should note the study's limitations. Sample data were collected from 2
IRFs in the Midwestern United States and are not representative of all IRFs.
Questions were similar but not identical in the Research Survey and the IRF
Survey, and rating scales were not the same. Subtle differences in item wording
and rating scale anchors may have affected responses. We do not know to what
extent proxy respondents obtained patient input to provide answers to survey
questions. Finally, not all respondents responded to each question; therefore,
the denominator for quality measure scores varied.
Readers should note that the samples’ inpatient rehabilitation stays concluded 8
years before publication of this manuscript. Health policy changes, service
innovations, and the COVID-19 public health emergency since then may limit the
relevance of the findings in today's context. Arguably, the major health policy
change in the past 2 decades was the passage of the 2010 Affordable Care Act
(ACA), which led to significant reductions in the uninsured population and
concurrent improvement in quality of care with the rate of hospital-acquired
conditions such as adverse drug events, infections, and pressure ulcers
declining substantially.21 However, this policy change predated participant
enrollment by 5 years and is unlikely to have affected rehabilitation services
after this study concluded. Increases in serial postacute care (PAC) transfers
from one PAC setting to another PAC setting began after the implementation of
the inpatient rehabilitation prospective payment system in 2002, which is a
per-discharge payment system. Prvu Bettger et al22 examined stroke survivors’
postacute service and found ≥3 care transitions after hospital discharge in a
large sample derived from administrative claims data. Similarly, Bryden and
Gran's23 qualitative study of PAC transitions in a spinal cord injury sample
found 4 transitions in the first 3 months. The ACA did not include IRF payment
changes, only the authorization of an IRF quality reporting program. The Centers
for Medicare and Medicaid Services has expressed an interest in implementing
experience of care surveys in IRFs, and although a survey was developed, it has
not been implemented. Finally, the COVID-19 public health emergency affected
health care delivery, including in PAC settings resulting in staff shortages,
therapy restrictions, and discharge barriers.24 The consequences of increased
PAC transitions on patient experience of care deserve careful scrutiny. It may
be that patients’ preparedness for discharge is reduced or that follow-up
efforts must be intensified if patients have not returned home. In summary, any
influence of the ACA on care transitions following this study seems unlikely.
The extent to which patients’ perceptions of staff and communication during the
rehabilitation stay are affected by more care transitions, and the concordance
between experience of care surveys collected at different times or using
different methods requires further study.

CONCLUSIONS

There were subtle but potentially important differences in experience of care
quality measure scores, reflecting the extent to which patients are encouraged
to complete the surveys as well as their functional status. Although responses
to questions about the overall perceptions about IRF care were similar,
questions about specific aspects of care showed some differences.

DISCLOSURES

K. F. C. has financial relationships with Clinical Outcome Solutions (COS),
Serano Merck, AVR, and Stanford (CERSI#63-FDA), outside the submitted work. The
other authors have nothing to disclose.
Recommended articles

REFERENCES

1. 1
G Stucki, J Bickenbach, W Frontera
Why rehabilitation should be included in international healthy ageing
agendas
Am J Phys Med Rehabil, 98 (2019), pp. 251-252, 10.1097/PHM.0000000000001111
View in ScopusGoogle Scholar
2. 2
BM Gans
Evolving models of rehabilitation-related patient safety and quality:
PIECES
Arch Phys Med Rehabil, 99 (2018), pp. 1033-1034, 10.1016/j.apmr.2018.03.001
View PDFView articleView in ScopusGoogle Scholar
3. 3
Centers for Medicare & Medicaid Services. Inpatient Rehabilitation Facility
(IRF) Experience of Care. Updated April 10, 2024. Available at:
https://www.cms.gov/medicare/quality/inpatient-rehabilitation-facility/irf-patient-experience-care.
Accessed June 13, 2024
Google Scholar
4. 4
N Chiu, R Aggarwal, Y Song, RK Wadhera
Association of the Medicare value-based purchasing program with changes in
patient care experience at safety-net vs non-safety-net hospitals
JAMA Health Forum, 3 (2022), Article e221956,
10.1001/jamahealthforum.2022.1956
View in ScopusGoogle Scholar
5. 5
MN Elliott, C Edwards, J Angeles, K Hambarsoomians, RD Hays
Patterns of unit and item nonresponse in the CAHPS Hospital Survey
Health Serv Res, 40 (6 Pt 2) (2005), pp. 2096-2119,
10.1111/j.1475-6773.2005.00476.x
View in ScopusGoogle Scholar
6. 6
MN Elliott, AM Zaslavsky, E Goldstein, et al.
Effects of survey mode, patient mix, and nonresponse on CAHPS hospital
survey scores
Health Serv Res, 44 (2 Pt 1) (2009), pp. 501-518,
10.1111/j.1475-6773.2008.00914.x
View in ScopusGoogle Scholar
7. 7
G Simon, C Rutter, M Crosier, J Scott, BH Operskalski, E Ludman
Are comparisons of consumer satisfaction with providers biased by
nonresponse or case-mix differences?
Psychiatr Serv, 60 (1) (2009), pp. 67-73, 10.1176/appi.ps.60.1.67
View in ScopusGoogle Scholar
8. 8
FJ Fowler Jr, C Cosenza, LA Cripps, S Edgman-Levitan, PD Cleary
The effect of administration mode on CAHPS survey response rates and
results: a comparison of mail and web-based approaches
Health Serv Res, 54 (2019), pp. 714-721, 10.1111/1475-6773.13109
View in ScopusGoogle Scholar
9. 9
A Deutsch, AW Heinemann, KF Cook, et al.
Inpatient rehabilitation quality of care from the patient's perspective:
effect of data collection timing and patient characteristics
Arch Phys Med Rehabil, 100 (2019), pp. 1032-1041,
10.1016/j.apmr.2018.10.019
View PDFView articleView in ScopusGoogle Scholar
10. 10
AW Heinemann, A Deutsch, D Cella, et al.
Feasibility of collecting patient-reported outcomes for inpatient
rehabilitation quality reporting
Health Serv Res, 53 (2018), pp. 1834-1850, 10.1111/1475-6773.12729
View in ScopusGoogle Scholar
11. 11
Press Ganey. Welcome to Human Experience Healthcare. Available at:
https://www.pressganey.com. Accessed July 25, 2024.
Google Scholar
12. 12
Centers for Medicare & Medicaid Services. Hospital Consumer Assessment of
Healthcare Providers and Systems. Updated June 12, 2024. Available at:
http://www.hcahpsonline.org. Accessed June 13, 2024.
Google Scholar
13. 13
AP Presson, C Zhang, AM Abtahi, J Kean, M Hung, AR Tyser
Psychometric properties of the Press Ganey(R) Outpatient Medical Practice
Survey
Health Qual Life Outcomes, 15 (2017), p. 32, 10.1186/s12955-017-0610-3
View in ScopusGoogle Scholar
14. 14
AW Heinemann, J Michael Linacre, BD Wright, BB Hamilton, C Granger
Measurement characteristics of the Functional Independence Measure
Top Stroke Rehabil, 1 (1994), pp. 1-15, 10.1080/10749357.1994.11754030
View in ScopusGoogle Scholar
15. 15
JM Linacre, AW Heinemann, BD Wright, CV Granger, BB Hamilton
The structure and stability of the Functional Independence Measure
Arch Phys Med Rehabil, 75 (1994), pp. 127-132
View PDFView articleView in ScopusGoogle Scholar
16. 16
BD Wright, JM Linacre, RM Smith, AW Heinemann, CV Granger
FIM measurement properties and Rasch model details
Scand J Rehabil Med, 29 (1997), pp. 267-272
CrossrefView in ScopusGoogle Scholar
17. 17
National Quality Forum (NQF)
National Voluntary Consensus Standards for Clinicians—Additional
Performance Measures 2008: A Consensus Report
National Quality Forum (2009)
Google Scholar
18. 18
Y Kang, MR Steis, AM Kolanowski, D Fick, VV Prabhu
Measuring agreement between healthcare survey instruments using mutual
information
BMC Med Inform Decis Mak, 16 (2016), p. 99, 10.1186/s12911-016-0335-y
Google Scholar
19. 19
National Quality Forum. Measure Evaluation Criteria and Guidance Summary
Tables Effective for Projects Beginning after January 2011. Available at:
http://www.qualityforum.org/docs/Measure_Evaluation_Criteria_Guidance.aspx.
Accessed June 13, 2024.
Google Scholar
20. 20
MN Elliott, DE Kanouse, CA Edwards, LH Hilborne
Components of care vary in importance for overall patient-reported
experience by type of hospitalization
Med Care, 47 (2009), pp. 842-849, 10.1097/MLR.0b013e318197b22a
View in ScopusGoogle Scholar
21. 21
B Obama
United States health care reform: progress to date and next steps
JAMA, 316 (2016), pp. 525-532, 10.1001/jama.2016.9797
View in ScopusGoogle Scholar
22. 22
J Prvu Bettger, L Thomas, L Liang
PCORI Final Research Reports. Comparing Recovery Options for Stroke
Patients
Patient-Centered Outcomes Research Institute (PCORI), Washington (DC)
(2019)
Google Scholar
23. 23
AM Bryden, B Gran
Seeking sufficient and appropriate care during the first year after spinal
cord injury: a qualitative study
Spinal Cord, 62 (2024), pp. 241-248, 10.1038/s41393-024-00974-x
View in ScopusGoogle Scholar
24. 24
AA Kelly, CA Lewis, MX Escalon
Inpatient rehabilitation issues related to COVID-19
Phys Med Rehabil Clin N Am, 34 (2023), pp. 513-522,
10.1016/j.pmr.2023.04.001
View PDFView articleView in ScopusGoogle Scholar

CITED BY (0)

Supported by Patient-Centered Outcomes Research Institute (grant no. CD
12-11-4201).
© 2024 The Authors. Published by Elsevier Inc. on behalf of American Congress of
Rehabilitation Medicine.

www.sciencedirect.com Open in urlscan Pro 162.159.137.70 Public Scan

Form analysis 1 forms found in the DOM

GET /search#submit

Text Content

www.sciencedirect.com Open in urlscan Pro
162.159.137.70 Public Scan

Form analysis
1 forms found in the DOM