Insights from FluTracking Analyses

Throughout the year, we will post interesting FluTracking analyses for your feedback.

Out of Order Surveys Analysis

What did we do? We checked how many people do their FluTracking surveys out of order (for example missing a few weeks and then doing the most recent survey before the earlier ones) and whether this influenced when FluTrackers report being vaccinated.
What did we find? Only a small fraction of people completed their survey out of order sequence and even smaller fraction changed their vaccination status at the same time.
So what? We are now more confident that FluTracking data on the timing of vaccination is not being largely influenced by surveys being done out of order.

One question the FluTracking team had was how many of our participants complete their surveys in chronological order. More specifically, we wanted to find out how many participants complete their surveys out of order and whether that has any impact on collected data used for analysis. This question was especially important for participants who want to update their vaccination status, as they may inadvertently tell us they were vaccinated at a time they weren’t if they are answering several surveys out of order.

Out of order survey completion refers to surveys which are completed before other surveys that were sent earlier. For example; you could miss one or more surveys (perhaps you were away on holiday), and when you get back you click on the latest FluTracking email. However; because newer emails appear at the top of your inbox you probably complete the latest survey before the earlier ones that you missed. As a result; your responses would be 3-4 weeks out of order.

Normally it doesn’t make any difference to our data as long you remember to complete all the surveys; however, it makes a big difference if you change your vaccination status in an out of date survey because your vaccination time may be incorrectly recorded, and this could impact on certain analyses.

Because of the way the FluTracking platform sends out the surveys, it is likely many people have completed at least one survey out of date if they simply click the latest email they receive. However, because we allow our participants to ‘back fill’ up to five additional weeks they may have missed, it is possible that completing surveys out of order could mean our recorded ‘vaccination dates’ are out by up to 6 weeks! It was therefore quite important that we check the extent to which this problem might be present in our data.


Year Analysed Number of Weeks Country Method Used to Analyse Data and Produce Charts
2018 20 Australia We used 11 SQL queries and VBA code to select, collate, and analyse the data, Microsoft Excel to produce charts from generated statistical data , and TechSmith Snagit (screenshot program) to convert generated charts into graphics we posted on this page.
Figure 1 – Analysis Method
Thankfully, the analysis showed there was really nothing to worry about. Most participants complete their surveys on time and don’t have too many surveys to back fill. Out of all the surveys we received, less than 10% were classed as out of date (i.e. were completed before earlier surveys). Importantly, when we looked at only those participants who changed their vaccination status and completed surveys out of order, the figure was even smaller. Less than 1% of participants who had to complete two or more surveys on the same day changed their vaccination status at the same time (refer to figure 2, figure 3, and figure 3 for comparisons).

NOTE: As a further precaution, we checked the length of time people generally back filled when they completed surveys out of order. Most participants (83.99%) who completed a survey out of order only missed 1 week, and only 8.50% participants filled back 4 weeks. Nobody was out by 5 or 6 weeks. Looking at all the collected data for 2018; there is really nothing to worry about. These analyses help us be confident in our analyses, particularly when we look at how Influenza Like Illnesses might differ for people who have been vaccinated against the flu.

Figure 1 - Total Out of Date Order Comparison Chart Image
Figure 2 – Total Out of Date Order Comparison Chart
Figure 2 - Weeks Out of Order Comparison (Vaccinated Status Only) Image
Figure 3 – Weeks Out of Order Comparison (Vaccinated Status Only) Chart
Figure 3 - Weeks Out of Order Comparison (Vaccinated Status Only) Image
Figure 4 – Weeks Out of Order Comparison (Vaccinated Status Only) Table