Satisfaction Survey

The Resident Satisfaction Survey – Review and Interpretation

RVM and the PRS system

Why RVM may be doing better than the numbers indicate

What else would be good to know

by Bob Buddemeier

On July 20 Dave Keaton presented results of the March 2023 Resident Satisfaction Survey.  To see the survey results, log in to MyRVM; the URL for the data slides is https://files.mwapp.net/FILES/141583233.pdf, and for the video of the presentation, https://retirement.app.box.com/s/rzzxupn336i077u98h86r2suj1tkrhcq.  If you are interested in the management and future of RVM, I recommend that you review that material.  As was the case with the presentation of the Strategic Plan, I feel that both the content and the conduct of the Survey presentation were positive contributions to building an informed and engaged resident community.

The results are part of a cycle of surveys of PRS-managed CCRCs, and include comparisons with eleven other communities over resident ratings on 14 factors.  The comparisons provide useful information, but need to be interpreted in terms of systemic differences between RVM and most other members of the PRS group of CCRCs.

The presentation files provide multiple views of the information; for the sake of convenience in reading this article, Table 1 contains some of the key data.

TABLE 1:  Satisfaction Survey, RVM and PRS System

RVM 23 = RVM 2023 survey results for each factor; N 23 = number of respondents for the question, RVM 18 = RVM 2018 survey results; PRS 23 = composite score, all PRS CCRCs

Key differences between RVM and other PRS CCRCs:

• RVM is not associated with a university or other institution, as the Davis and Tempe facilities are.
• RVM has 900+ residents, compared with 200-300 at many of the other facilities.
• RVM has both towers and a large number of cottages on a large campus; most others are apartment buildings only, with relatively small grounds.
• RVM is not in or near a major metropolitan area, unlike the others. This means there is a different lifestyle potential and ambience, and that RVM residents are drawn from more widely dispersed market areas.

What do these difference imply about interpretation of results?

I suggest that one of the most important points is the combination of (a) the number of residents, (b) the variety and dispersion of housing arrangements, and (c) the variety of backgrounds and geographic origins of the residents.  These result in a wide range of attitudes and interests among residents, greater average social distances among individuals and subcommunities, and greater challenges in communication, both among residents and between the administration and residents.

These factors imply that the residents will be less likely to operate on the basis of the same information and assumptions, and that peer interactions will have less influence on collective behavior, so consensus and concerted action will be more difficult to achieve.

This also means that the range of opinions – or survey responses — is likely to be broader at RVM than elsewhere.  I’ll make further comments on this later, but a specific result is that, in the context of this survey, a broader range of opinions is likely to result in a lower score.  The survey defines “average” as 3 on a 5-point scale.  If the average of the results were close to three, more outliers would not necessarily change the score much.  However, for the 14 factors and 12 institutions, 123 of the 168 scores are between 4 and 5, and almost all of the others are between 3 and 4.  This means that introducing more variation into the dataset will reduce some of the highest scores, since there is no way to increase them.

My conclusion: The process of comparison is weighted against RVM, and relatively speaking, we are doing better than we might be expected to based on the scores of the other CCRCs.

So what? one might ask.  If we want to be the best we can be, we should strive for the magic 5.0 in all categories, regardless of what the others do.  Well, yes, given infinite resources.

Unfortunately, we are not given infinite resources, and doing the best we can with what we have means prioritizing goals to maximize satisfaction even if we can’t perfect it.  In turn, that means understanding what the results tell us about where to dedicate resources.  The resources to be dedicated may be dollars to fix known problems in some cases, or human resource time to analyze and identify problems.  In either case, it’s important to identify both the importance of problems, and the ease with which they can be solved.

A few specific examples:

1. About 27% of RVM respondents (132 people) answered “no” or “don’t know” to the “would you recommend RVM?” question.  That’s somewhat higher than the PRS system value, and considerably higher than both the RVM value in 2018 and the Holleran (survey contractor) benchmark value.  It’s also significant for marketing.  To address that directly, I would want to find out whether the responses were associated with a diffuse negative attitude, or correlated with a few specific dissatisfactions.
2. Subjective and qualitative factors are difficult to deal with – Administration Responsiveness and Resident Association Effectiveness are examples, especially since these are probably affected by the size and diversity of the resident population to be dealt with. Definition and analysis are needed.
3. Staff Friendliness probably has an ameliorating influence on some of the other factor ratings, since people are likely to overlook minor deficiencies if an employee is friendly, polite, and responsive to requests. However, almost all of the institutions ranked high on this factor, so it probably isn’t a relative advantage for RVM – but it is something to work hard on maintaining.
4. Reaching for the low-hanging fruit: something that is missing from the results is a breakdown by residence – at least towers vs. cottages, and maybe further (Manor/Terrace vs Plaza, south vs north villages). If concerns are significantly different, focused responses can be more economical and effective.

A final comment – public opinion and public behavior can be very slow to change.  The problematic Dining Experience factor seems to me to be like steering an ocean liner.  The whole process of menu development, supply, staff acquisition and training and resident information takes a lot of time and effort, and it may take years to find out what difference a change makes.  I think that we are still operating in the shadow of the COVID years, and it is not clear how the many non-equilibrium factors affecting the future are going to shake out – not just for dining, but for the organization as a whole.

It would be nice to have some simple but effective ways to assess resident attitudes and opinions – and especially changes.  Unfortunately I don’t have a specific proposal, but one approach might be modification of Residents Council organization to incorporate a layer of contacts or representatives at the neighborhood level.  In addition to providing information relevant to Council and Administration decision-making, this might also improve the Resident Association Effectiveness rating on the next survey.

Appendix for those familiar with some basic statistics:

I am not a statistician, but I am a believer in both use and constructive misuse of statistics to explore and understand datasets.

It might be useful to analyze the distribution of scoring patterns for individual respondents.  It’s encouraging if people think everything is very good, but their answers reduce the ability of the analysis to discriminate among the factor ratings, and don’t provide guidance about what needs to be done.  Reanalysis after excluding individual responses consisting of uniformly high scores (e.g., average >4.5) might provide better differentiation of the factors, as well as a more nearly normal distribution of data.

Another need I see is for a useful assessment of uncertainty and significance.  The data produced by the survey present some problems: on a 5-point scale, median and range do not provide much precision, and the rating distributions are far from the bell-shaped curve norm.  However, standard deviation is a convenient calculation that can provide a semi-quantitative estimate of uncertainty adequate for a rough ranking (or grouping) of factors, even though the distribution doesn’t support conventional quantitative interpretations.

Approximations of the latter sort can be done by reverse-engineering the data that are released, but a firm like Holleran undoubtedly has the capability to make much more sophisticated calculations, and I hope that if this has not already happened, PRS and RVM will request further exploration of some of the points raised above.