In this blog post, we are going to discuss about the process of engineering metrics to capture the two major concepts in our research question using the British Election Study panel data:
How do attitudes toward immigrants & immigration influence the volatility of opinions on BREXIT?
Anti-Immigration Sentiment (AIS)
To quantify individual’s immigration sentiment, we identified 4 survey questions that explicitly asked about respondents’ attitudes towards immigrants and immigration. Given that the 4 survey questions were asked across multiple waves, to compute one immigration index on a person level, we first need to confirm that one’s immigration attitude does not vary drastically over time. In particular, we want to look at the range of attitude change. If respondants’ immigration sentiment vary within a small range over time, we can assume that by taking the average of an individual’s responses across multiple waves we have an accurate representation of their attitude at any point of time.
In practice, we computed the range of change by taking an individual’s maximum and minimum responses over the period in which they participated in the survey. We defined ‘tolerable change’ as an individual has changed in their reponses at most by 1 level for a 5-level response questions (e.g. from “Disagree” to “Strongly Disagree”), or at most 2 levels for a 7-level response questions (e.g. from 2 to 4 on a scale of 1 to 7) in the entire course of their survey. Any changes more than described above are defined as ‘significant change’.
Fig.1 below shows that more than 75% of the respondants exhibit no change or tolerable change in their immigration attitude. This is true for all 4 survey questions we identified. Therefore, we decide to quantify an individual’s immigration attitude using their mean response.
The four questions are:
- immigEcon: “Do you think immigration is good or bad for Britain’s economy?”
- immigCultural: “Do you think that immigration undermines or enriches Britain’s cultural life?”
- immigrantsWelfareState: “How much do you agree or disagree with the following statements? ‘Immigrants are a burden on the welfare state?’”
- changeImmig: “Do you think that immigration are getting higher, getting lower or staying about the same?”
Fig.2 Correlation matrix between responses to the 4 survery questions on immigration sentiments.
Fig.2 shows us that the responses to the first 3 questions are highly correlated (correlation coefficient > 0.75). This helps us make the decision to compute an overall index to quantify Anti-immigrant Sentiment (AIS) using responses from the first 3 questions.
Volatility of BREXIT Voting Preference
Next, we want to find a way to quantify opinion volatility in respondants’ attitude towards Brexit. Except for wave 5, the survey asks respondents “If there was a referendum on Britain’s membership of the European Union$turnoutText, how do you think you would vote?”. We flag the responses if it is of a polarized switch from “stay” to “leave” or from “leave” to “stay” as compared to their previous responses in last wave.
We computed two person-level metrics: switch_ratio and if_switch. switch_ratio is calculated by using the number of switches an individual had divided by the number of waves they participated – 1. This is because for n waves an indiviudal participated in the survey, there are only n – 1 waves they can possibly switch their voting preference. It measures how frequent an individual switches their opinion. if_switch is a binary variable that indicates if an individual has switche their voting preference at all.
In practice, we found that if_switch is much more helpful in differentiating respondants in terms of opinion volatility. Firstly, approximately only 1 in every 7 respondants switched their voting preference from one side to the other side. (1 indicates switch and 0 otherwise).
##
## 0 1
## 36892 7932Therefore, in effect, the distribution of switch_ratio looks like: 
Such a highly skewed distribution makes it difficult for switch_ratio to effectively represent opinion volatility in modeling tools.
In addition, we looked at the relationship between switch_ratio and immigration sentiment (AIS). 
As the graphs above suggest, AIS remains relatively constant regardless of switch_ratio. We observe volatility of AIS when switch_ratio exceeds 0.5, but the extremely small sample size in that range undermines the significance of such observation. Given that we do not observe a difference in immigration sentiment in respondents no matter how frequent they switch their voting preference, we hypothesize that the frequency of switch is not as critical as whether individuals switch their opinion at all. Therfore, we deicde to use if_switch instead of switch_ratio to advance our investigation.
Later on, we use combine if_swtich and respondents’ final vote to categorize respondants into 4 different voter types:
- Voters who always voted for “stay”
- Voters who always voted for “leave”
- Voters who have switched their opinion and voted for “stay”
- Voters who have switched their opinion and voted for “leave”
With this categorization, we developed an interactive Shiny application to directly visualize the difference in immigration sentiments amongst 4 different voter types.
Source: British Election Panel Study Data
http://www.britishelectionstudy.com/data-objects/panel-study-data/