-
Qualtrics Platform -
Customer Journey Optimizer -
XM Discover -
Qualtrics Social Connect
Anonymity (EX)
About Anonymity
There are 2 levels of anonymity: basic and enhanced. Basic anonymity is enabled by default in each dashboard, while enhanced anonymity can be turned on for additional layers of protection. For more information, see Basic vs. Enhanced Anonymity.
Any changes you make in anonymity settings will immediately be applied to your dashboard.
Basic vs. Enhanced Anonymity
Basic Anonymity
Basic anonymity thresholds determine how many responses must be included for a given data point before it can appear in your dashboard. This is a great way to protect the privacy of employees’ responses. The anonymity threshold applies to each data point for metrics (favorability score, average, etc.) but not counts of data (number of responses).
Basic anonymity is a simple way to protect employee responses while still allowing flexibility with data analysis.
When adding filters to a page, you are able to select values below the threshold, but you won’t see any data until the total value of all data selected in the filters meets your anonymity threshold. See Filtering Dashboards for more information.
When breaking out data on a single field, any metrics below the anonymity threshold will be hidden and counts will be shown. When breaking out data on multiple fields, both metrics and counts below the anonymity threshold will be hidden.
Response Rates widgets will show counts below the anonymity threshold.
Enhanced Anonymity
In addition to the features covered by basic anonymity, enhanced anonymity adds additional layers to filters and widget breakouts that can improve anonymity in certain use cases. With enhanced anonymity, the anonymity threshold applies to all data points (metrics and counts) for sensitive fields, but not for any data points for not sensitive fields.
Enhanced anonymity provides advanced protection for employee responses, and as a result there is less flexibility with data analysis.
Anonymity Thresholds
The anonymity threshold determines how many responses must be included for a given data point or comment before it can appear in your dashboard. Data points can be as broad as a widget or as specific as a bar within a chart.
The default anonymity threshold is set to 5 for both data points and comments.
To view and modify anonymity thresholds, follow these steps:
- While viewing your dashboard, click Settings.
- Go to the Anonymity tab.
- Under Minimum responses to display data points, decide how many responses must be collected before data will show up in the widgets. This limit is applied to all data breakouts in all widgets.
Example: The image below shows a Comparison widget that is broken out by the active Org Hierarchy. If units have response counts below the Anonymity Threshold, then those units will display no data in the widget. Units with a response count above the Anonymity Threshold will display data as normal. Since the number of responses for the Mary Shelley and Mark Twain units falls below the Anonymity Threshold, the widget displays the message, “Too few responses” for these units.
- Under Minimum responses to display comments, decide how many responses must be collected before open-ended text responses will show up in the widgets.
Enabling Enhanced Anonymity
Enhanced anonymity can be turned on and off at the level of the dashboard to improve anonymity.
For example, let’s say Barnaby’s team has 15 people. When we look at the engagement scores for his team, we don’t really know how each team member responded to questions about their manager’s effectiveness. However, let’s say there are only 2 women on his team. Regular anonymity thresholds will ensure we can’t see the women’s responses directly, however, if we add a gender filter, we can make a pretty good guess what each of the women on his team had to say. Enhanced anonymity senses disparities of this kind. It ensures that data from groups that don’t meet the anonymity threshold is combined with the next smallest group to hide their answers when breaking out data or using filters.
- While viewing your dashboard, click Settings.
- Go to the Anonymity tab.
- Enable Turn on enhanced anonymity.
Field-Level Settings
When you have enhanced anonymity enabled, you can customize the level of anonymity for each field in your dashboard by marking fields as Extremely sensitive, Somewhat sensitive, or Not sensitive. This changes the way breakouts are treated and displayed in widgets, and allows you to group data points below the response threshold on some fields while hiding data points below the response threshold on others.
There may be some dashboard fields participants can be identified by, which should be considered sensitive and should be marked extremely sensitive or somewhat sensitive, while fields that are not sensitive should be marked not sensitive. For example, tenure, gender, and the team to which someone belongs can all be used to figure out who they are. However, questions asked in an Employee Experience survey are almost always not sensitive, with the exception of demographic questions like language, location of office, and age.
When fields are marked extremely sensitive, the data from groups that do not meet the anonymity threshold will be combined with the next smallest group in order to protect the identities of the respondents. When fields are marked as somewhat sensitive, their data from groups that do not meet the anonymity threshold will not be displayed. When fields are marked as not sensitive, grouping will not happen unless you filter or break out by a sensitive field.
To edit which fields are sensitive and which are not, you will want to do the following:
- Go to Settings.
- Select Anonymity.
- Select Customize field-level settings.
- Use the dropdown next to each field to select how sensitive it is. You can choose from the following options:
- Extremely sensitive: These fields contain information that will identify participants. When a data point falls below the response threshold, enhanced anonymity settings apply and data will be grouped with the next smallest data point(s). These fields were previously called identifiable fields.
Qtip: All metadata fields are marked as extremely sensitive by default.
- Somewhat sensitive: These fields contain information that may identify participants. Data points that fall below the response threshold will not be displayed, the same as with basic anonymity. This option is useful for fields such as dates or time periods.
Qtip: With this option it is still possible to figure out which participant provided a certain response; for more sensitivity, fields should be marked as extremely sensitive.
- Not sensitive: These fields do not contain information that can identify participants. All data points are shown even when responses fall below the response threshold. These fields were previously called non-identifiable fields.
Qtip: All question fields are marked as not sensitive by default.
- Extremely sensitive: These fields contain information that will identify participants. When a data point falls below the response threshold, enhanced anonymity settings apply and data will be grouped with the next smallest data point(s). These fields were previously called identifiable fields.
- Click Confirm.
- To return to the original configuration and remove all your changes, click Reset.
Example: In our dashboard, we did not mark engagement questions as identifiable, because they are not demographic and cannot be used to identify their respondents in any way.
Let’s say the dashboard’s threshold is 5. If we made a table displaying how employees responded to a non-identifiable field like “I feel proud to tell people where I work,” answers will not be grouped. See below how “Strongly Disagree” appears, although it only has 1 response.
Note that the total number of responses in the widget must still meet the threshold. This graph has a total of 90 responses. If it had fewer than 5, the graph would be blank because the default behavior of the anonymity threshold is to hide data from widgets that do not meet the threshold.
Field Interactions
If you are using a table or chart to display an extremely sensitive or somewhat sensitive field with a not sensitive field, your data should be grouped the same way either way you transpose the data. Grouping logic is applied consistently based on the field settings, no matter which field is configured as a row or column.
Setting the Dashboard Data Source(s)
When using Enhanced Anonymity, if you are also using historical sources in your dashboard, it’s important to go to general dashboard settings and limit page filters by the current year’s data.
Otherwise, dashboard filters, except the Org Hierarchy filter, which will default to the primary data source, will include data from all data sources in the dashboard data. This means historical data may be included in response counts, and can skew anonymity groupings. For example, if current and historical results are counted for a small team rather than just the current year’s results, the small team seems bigger than it really is, and may not fall below the anonymity threshold. Limiting the filters to the primary data source (i.e., the current project or current year’s data) resolves this issue.
Filter Behavior
Once you’ve enabled enhanced anonymity and added a filter to your dashboard, your anonymity settings will determine how filters behave.
Page filter behavior depends on the field-level setting for each field:
- Not sensitive fields: These fields allow you to select any value, even if it is below the threshold.
- Somewhat sensitive fields: These fields only allow you to select values that meet or exceed the threshold.
- Extremely sensitive fields: These fields group any values below the threshold. Results below the threshold will be combined with the next smallest option in the filter before any selections are made. If you have just 1 group that falls below the threshold, this will be combined with the next smallest group, regardless of whether the next group meets the threshold or not. This is to ensure that even if only 1 group doesn’t meet the threshold, their data is protected.
As filters are added or removed, enhanced anonymity will take these into account and change the groupings accordingly.
Example: In the screenshot below, we are trying to filter by department. This is a small company, so Finance, Support, and Human Resources have very small teams, below the anonymity threshold we have set.
You’ll see that Finance, Support, and Human Resources are under the header Grouped for Anonymity. If I try to select just one, they will both automatically be selected. If I try to deselect one, both are deselected. This prevents users from figuring out the values of below-threshold groups.
Org Hierarchy Filters
Org hierarchy units that do not meet the anonymity threshold will be grayed-out and have a lock icon next to them. You will not be able to select any units that do not meet the anonymity threshold. This is to protect the anonymity of respondents.
Breakout Behavior
When enhanced anonymity is enabled, the field-level settings for each field determine how data will be displayed in widgets where data has been broken out into certain groups. This includes line widgets where an x-axis dimension has been defined, widgets with comparisons added to them, demographic breakout widgets, heat map widgets, and any other widget configuration that isolates groups that may be smaller than the Anonymity Threshold.
- Not sensitive fields: These fields will show all data points (metrics and counts), even if they are below the threshold.
- Somewhat sensitive fields: These fields will only show data points (metrics and counts) that meet or exceed the response threshold.
- Extremely sensitive fields: These fields will group any data points (metrics and counts) below the threshold.
The exceptions to this rule include widgets broken out by org hierarchy. Some widgets (heat map, demographic breakout) support a One Level Below breakout that shows data for each child unit of the currently selected unit in the org hierarchy filter. Other widgets (response rates, comparison, bubble chart) support drilling down into the hierarchy, showing data for each unit and allowing the user to select them. For any widget broken out by org hierarchy, Enhanced Anonymity is not applied. This means no units will be grouped for anonymity.
If you were to change the metric to an average engagement score or an NPS, enhanced anonymity would prevent you from figuring out the smallest office’s data by not allowing dashboard users to isolate that office’s data. This is useful in cases where, for example, we do not want the ratings of each member of the smallest office to be easily calculated.
Example:
In the following example, our dashboard has an anonymity threshold set for the “country” field to help protect responses from employees in small offices. In the widget below, we have set the y-axis dimension of a bar chart to be the country where the employee’s office is located. Australia and Mexico have very small offices, below the anonymity threshold we’ve set. As a result, their responses have been combined.
If you were to change the metric to an average engagement score or an NPS, enhanced anonymity would prevent you from figuring out the smallest office’s data by not allowing dashboard users to isolate that office’s data. This is useful in cases where, for example, we do not want the ratings of each member of the smallest office to be easily calculated.
Multiple Breakouts
Some widgets break out on multiple dimensions – for example, line and bar widgets let you add both an x-axis value and a data series, and tables let you add both rows and columns.
The more you break out data, the smaller each category can get, and the more categories that get grouped together under anonymity. And because there are now 2 dimensions to the breakout, there may be different combinations of categories that need to be grouped together for anonymity. Thus, categories grouped for anonymity will be labeled Grouped for Anonymity, and you can hover over them to determine what specific categories were grouped.
You’ll also notice that in the legend, anonymity groups are labeled as Grouped for Anonymity, not a compound name. This is to take into account how groupings might change based on how multiple breakouts interact, and to prevent labels that are too long.
Example: This dashboard has a threshold of 5. In the graph below, we broke out the countries our employees work in by attrition risk. We can see a light green block for Mexico in the Low Risk bar, but not in the High Risk bar.
When we highlight the Grouped for Anonymity blocks for each bar, we discover that there’s a difference: Mexico was grouped with Japan and Poland in the High Risk bar, but only Japan and Poland were grouped in the Low Risk bar.
Look at the screenshot below. In High Risk, Mexico has no individual data ( – ), but Japan + Mexico + Poland shows data, since these are grouped together for anonymity (5). Under Low Risk, Mexico meets the threshold, so it does not need to be grouped, and shows individual data (17), while Japan + Poland are grouped together (5).
Basic Anonymity
If you’re breaking out data by 2 or more fields with basic anonymity, each of the breakout fields will be considered separately. When breaking out data on multiple fields, metrics and count data points below the threshold will be hidden.
Basic anonymity will not count responses without value (empty/null) when comparing response count against anonymity threshold. This is to protect cases where a survey taker who might not be eligible to answer a question (e.g. survey logic says only managers can answer a question) have their responses counted against the anonymity threshold.
Enhanced Anonymity
Enhanced anonymity counts responses without values (empty/null) when comparing response count against anonymity threshold.
Multi-field breakout behavior depends on the field-level settings for the 2 fields involved. This table shows how data will be hidden for double breakouts between different field types.
| Not Sensitive | Somewhat Sensitive | Extremely Sensitive | |
| Not Sensitive | All data points (metrics and counts) will be shown. | Values from the somewhat sensitive field will be hidden if they are below the threshold. | Values from the extremely sensitive field will be grouped if they are below the threshold. |
| Somewhat Sensitive | Values from the somewhat sensitive field will be hidden if they are below the threshold. | All data points (metrics and counts) below the response threshold will be hidden. | Values from the somewhat sensitive field will be hidden if they are below the threshold and values from the extremely sensitive field will be grouped if they are below the threshold. |
| Extremely Sensitive | Values from the extremely sensitive field will be grouped if they are below the threshold. | Values from the somewhat sensitive field will be hidden if they are below the threshold and values from the extremely sensitive field will be grouped if they are below the threshold. | Data points below the response threshold will be grouped. |
Response Rates Behavior
Response rates display how many responses you received, and what percentage of your participant list has completed the survey. This kind of data is reported by participation summary and response rate widgets.
By default, the dashboard considers response rates to be sensitive information. This means response rate data can be used to identify participants, and so to protect your participants, response rates are subject to being grouped by anonymity.
Example: When you create a response rate widget, you can add a field to break out the widget. Below, we have broken out our response rates by country, which has resulted in Australia, Germany, and Mexico being grouped for anonymity.
Thus, response rate widgets display the same breakout behavior other widgets do. If the response count is below the threshold, you will not see data in the participation summary widget.
When enhanced anonymity is enabled, response rates behavior depends on the field-level setting for each field:
- Not sensitive fields: These fields show all data points, even if they are below the threshold.
- Somewhat sensitive fields: These fields only show data points that meet or exceed the response threshold.
- Extremely sensitive fields: These fields group any data points below the threshold.
Anonymity Settings for the Entire Organization
You can set the anonymity threshold for your entire organization, ensuring all EX projects meet the same privacy standard. See Anonymous Responses (Admin).