The 2008 Shorewood Community Survey represents a less than complete look at the opinions of Shorewood residents, undercounting low income residents and renters. As such, any guidance drawn from these survey results when developing policy initiatives will automatically be skewed toward higher income residents and homeowners. The methodology section of the survey contains the following excerpt, which outlines the demographic makeup of survey respondents.
"Almost half (49%) have lived in Shorewood for 16 years or more. The average household size is 2.33 people per household. Nineteen percent of all households were retired. Over 80% own their own home, and the majority (64%) lives in single-family homes. Compared with Census 2000 findings – and similar to 2003 results – the respondents tended to have higher household incomes than the average Shorewood."
The home ownership statistics jump out immediately as being skewed. The 2000 Census said about 52% of the units in the Village are rental units. This undercounting of renters also skews the average household size, which is larger for homeowners (2.51) than for renters (1.69).
Ways for adjusting for data bias include data weighting, or oversampling populations who have lower response rates. These methods can ensure the respondent pool makes up an accurate snapshot of the Village Population, and can accurately inform decision makers on the opinions of Village residents. When I inquired into why this was done, the representative from the UWM Center for Urban Initiatives and Research (CUIR), who conducted the survey, said the intent was to mirror the 2003 Village survey, where data was also not weighted, and where results also skewed toward higher income residents.
There is value in tracking opinions over time, as was done here. However, this methodological choice makes it difficult or impossible to draw conclusions about Village-wide opinions on any item where opinions differ between income levels or home ownership status. You can still look at opinions within subgroups, but statements that begin with "residents believe..." or "overall, Shorewood residents rated..." should be taken with a grain of salt. CUIR notes this limitation within the body of the report.
"...The survey respondents should not be construed as a representative sample of Shorewood’s population. In particular, homeowners and higher income households are over-represented among survey respondents than the households found in the Village of Shorewood. However, it may be interesting to note that income measures are not standardized, so it is entirely possible that both survey respondent and Shorewood resident populations are becoming increasingly wealthier, especially compared with 2000 Census figures."
I have requested the raw data, and will look at a few re-weighting options when I receive it.
