Weighting Best Practices

In our previous post on sampling best practices, we mentioned weighting is sometimes an option to bring sample imbalances in line under certain circumstances. While data is typically weighted to match sample specifications that are not achieved naturally, weighting is NOT a replacement for appropriate sampling.

Some benefits of weighting include:

• Helping a sample to be more representative of the population.
• Adjusting for different response rates.
• Allowing comparisons across samples. It is important in tracking that differences detected are true differences rather than driven by different types of respondents in the sample.
• Sometimes, appropriate data weighting can be more cost-effective than obtaining a representative sample.

The number of respondents does not change when weights are applied. Some people’s responses simply carry more weight than others. We assign a weight to each respondent; some respondents are weighted up and others are weighted down to reflect desired proportions. For simple weighting schemes, weights are calculated by dividing the target % (representative proportion in the population) by the actual % (sampled).

Example:

There are two main types of Weighting: Target and Rim. Target weighting, also called Cell Matching, is appropriate for a simple weighting scheme, and is based on one variable. This single variable, however, can be nested (e.g., education within gender). The weights can be calculated manually as shown in the table above.

RIM weighting, sometimes called Raking, is appropriate for more complex weighting schemes with two or more variables. RIM weighting calculates weight factors (via data processing software) by iteratively adjusting weights to align with one variable’s targets, then for another, etc., until an adequate solution is found that brings all variables into line with their respective targets. RIM weighting uses non-nested variables, therefore, knowledge of the relationship between variables (e.g., education within gender) is not required. RIM weighting results may also be more stable than using nested variables when sample sizes are small.

Appropriate actions should be taken to sample the population correctly, and only weight results when necessary. Big Village has the expertise to guide when weighting is appropriate and walk you through the steps to ensure data remains representative of the population of interest.

Written by Sheilah Wagner, Research Director at Big Village Insights.