Skip to main content
Skip to article
  • Qualtrics Platform
    Qualtrics Platform
  • Customer Journey Optimizer
    Customer Journey Optimizer
  • XM Discover
    XM Discover
  • Qualtrics Social Connect
    Qualtrics Social Connect

Concept Testing Program

Was this helpful?


This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

The feedback you submit here is used only to help improve this page.

That’s great! Thank you for your feedback!

Thank you for your feedback!

About the Concept Testing Program

Concepts tests are a method to validate your best product ideas. Identify which product or service offers the best market opportunities and which are not the best use of your resources. Test every aspect of your product concept, from features, to branding, to messaging, in order to set your product up for success.

Now in addition to testing multiple concepts at a time, you manage them all in one place, view top concept scores within a larger concept testing database, and make dashboards to report out to executives.

Creating a Concept Testing Program

image of Concept testing program appearing under programs in the Catalog

You can create a Concept Testing Program by going to the Catalog and either searching it by name or filtering for Strategic Research.

Additional Types of Concept Tests

There are a few concept tests available in the Catalog.

  • Concept Testing (Basic): A project for conducting concept tests. Has a custom report, but no dashboard. Survey content is similar to but missing improvements present in Concept Testing (Enhancement).
  • Concept Testing (Enhanced): An improvement upon the Concept Testing (Basic), including a more robust survey and a custom dashboard. For example, this iteration includes Attributable Effect Analysis, which identifies the attribute(s) that offer the most opportunity and most risk in driving purchase intent.
  • Concept Testing Program: The holistic program where multiple Concept Testing projects can be compiled and reported upon. By default, when you add a new concept test to the overarching program, it is an Enhanced Concept Test.

Adding Concept Tests to the Program

Then purpose of a concept test is to get feedback on your product concept before you launch it to consumers.

Your Concept Testing Program will start out empty. That’s because you need to fill it with your own concept tests! Don’t worry, creating these is easy – we’ll guide you through the setup right inside Qualtrics, asking you for key information before putting it all together into a premade survey.

  1. On your program page, click Create new concept test project.
    Image of "create ew concept test program" button - not the dropdown next to it
  2. Name your concept test.
  3. Click Create project.
    Image of a screen explaining what the concept test is best used for. There is a field for the name at the very end
  4. Enter introduction text to explain the purpose of this survey.
    Survey Details window opened, with numbers by the paragraphs - B2C is selected

    Example: Treadmade would like to gather feedback on some potential new footwear options. You’ll be asked for your feedback on these concepts. Thank you for your input!
  5. Select whether this is a B2B (business-to-business) or B2C (business-to-consumer) product.
    Qtip: B2B companies sell products to other businesses. B2C companies sell products directly to consumers.
  6. Select the broader industry / category for your product concepts.
    Qtip: The options available will vary based on whether you chose B2C or B2B. If you select “other,” you can enter your industry.
    Qtip: We are asking for your company type and broader industry so that in the future, if we offer benchmarks with similar data, it will be easier to add them to your program.
  7. Now you’ll fill out a series of screening questions. First, enter the product category of the concepts you’re testing, e.g., clothing.
    Image of a screen that states the following: "You may want to screen respondents based on broader category usage (that is, to include only current category users in your sample of respondents). If you prefer not to use a screening question, check out the guidance pane that appears on the right side of the screen after completing the setup assistant for instructions on removing the question and the corresponding survey flow branch. In this section, you will need to enter the broader category of interest, the concept’s type of product, other type of products, how the respondents interact with the category (consume, use, purchase), the timeframe within which the respondent last interacted with the category. Together, these inputs form the screening question: Which of the following have you {consumed, used, purchased} in the past {timeframe}? The concept’s type of product and other types of product are the answer choices"
  8. Enter the specific product category you’re testing, e.g., footwear.
    Qtip: This choice will be used in the screening question. Only respondents that select this choice will be shown the concept testing questions.
  9. List three to five other products that are related to your product category (e.g., cake, cookie, scone). Click Add Another to add an option, or use the minus sign ( – ) to remove an option.
    3 paragraphs explaining the types of questions and framework needed, paragraphs numbered 9-11

    Qtip: These choices will be used in a screening question. Only respondents who have used that product that you want to test will be shown the concept testing questions.
  10. Enter the timeframe you want to test (e.g., past 3 months).
  11. Select the word that best describes how respondents interact with this product. Your options are:
    • Consume
    • Use
    • Purchase
  12. Enter the name of the brand you want to test.
    Large box highlighted, titled 'Define your Concepts' numbered 13
  13. Define your concepts. For each concept, provide a name, description, and an image. Each one will be shown to respondents along with a series of measurement questions for each concept.
    Qtip: Images can be a maximum size of 1MB. For file types and other troubleshooting advice, see Troubleshooting Graphics.
    Qtip: You can remove a concept by selecting Remove in the upper -right. You can remove images from concepts by selecting Remove beneath the file name.
  14. You can add up to five other attributes for respondents to rate. This is optional. Make sure to phrase attributes as adjectives – e.g., “Sturdy,” “Made of nice material.”
    Steps 14-16 of concept test instructions outlined

    Qtip: In addition to providing diagnostic feedback, these attribute questions are used in the Attributable Effects Analysis you will find in the dashboard.  Attributable Effects Analysis identifies the attribute(s) that offer the most opportunity and most risk in driving purchase intent.
    Qtip: Respondents will be already asked about their initial reaction, concept likes/dislikes, likelihood to purchase, uniqueness, believability, relevance, addition/replacement, strength of need, need vs. want, and fit with brand.
  15. Choose the type of concept test you want to perform:
    • Respondents see a single concept. Also known as monadic tests, these tend to be shorter surveys with more respondents. These are best used for later stages of concept development and for concepts that will be included in a concept program/database.
      Qtip: As they are developed, benchmarks will only be available for monadic tests.
    • Respondents see multiple concepts. Also known as sequential monadic tests, these tend to be longer surveys, have fewer respondents, and are best used for earlier stages of concept development.
      Qtip: How you answer this question will affect the type of information included in your dashboard – for example, if you’re testing multiple concepts, your dashboard will make sure to give you an overall summary, in addition to deep dives dedicated to each concept you entered.
  16. If you decide respondents should see multiple concepts, you will be asked to specify the number of concepts that should be presented to each respondent.
    Qtip: We recommend no more than four concepts per person. The solution will require you to choose a number between two and four.
  17. Note the recommended sample size provided to the right. How you answered previous questions, such as the type of test you want to perform, will adjust this number.
    Image of the last steps - the recommended sample size is in a sidebar and not in the main steps
  18. Click Next.
  19. Choose the demographics or firmographics you’d like to include.
    Multiple choice question showing types of demographics to include in survey, labeled 19 and 20

    Qtip: We recommend five or fewer to prevent survey fatigue. Your options will vary based on whether you chose B2C (demographics) or B2B (firmographics).
  20. Click Finish.

Included with Each Product Concept Test Project

Included with each product concept test that your create in your program is guidance, a premade survey, a premade report, and a premade dashboard.

Certified Questions

The questions within the “Concept Appeal” block can’t be edited. That’s because they are certified questions, which can be used to benchmark and compare your concepts. Although they cannot be edited, you can move and reorder questions as needed. See the Certified Questions support page for more details.

If you want to use different wording in these questions, you can delete the certified question and add your own, revised question; however, if you do this, you will not get the added benefits of the certified questions. Try the Certified Questions You Can Customize instead.

Removing Screening Questions

There is one screening question in your survey, which will remove respondents who have not engaged with the type of product concept you’re studying for the timeframe you specified. If you prefer not to use a screening question, then complete the following steps to delete the screening functionality.

  1. Go to the survey flow.
  2. Move the question blocks and groups out from under the first branch. Be careful about the order blocks appear in. Do not deconstruct the branches that appear under the group.
  3. Delete the first branch. This should be the branch based on Q1, “Which of the following…”

For more troubleshooting advice, see the Survey Flow page. If you’re interested in adding additional screen-outs, see Screen-Out Management.

Dashboard Tab

When you visit the Dashboard tab of your individual Concept Test, you’ll find a premade dashboard developed by subject matter experts. You need to collect data before it will populate with data. Otherwise, the widgets will be empty when you edit the dashboard.

An image of the survey tab of the project - along the top, the last tab is "Dashboard"

You can edit this dashboard just like any other – mapping new fields, adding new widgets, or removing content as needed.

Dashboard Editing

Editing dashboards in guided solutions is the same as editing a CX Dashboard project. Check out these introductory pages to get started.

Unavailable Dashboard Features

The following features are not available in the Dashboard tab of the Concept Testing solution.

Managing Your Program

Once you are finished creating your Concept Testing program, it can be found any time on the Projects page of your account. When you click the program’s name, you’ll find yourself on a page with the various resources we have developed to help you run your program.

An image of the program and the widgets that show high level data

Several resources are included, which you’ll see listed on the left under “Projects.” These include all the concept tests you’ve added to the program. We recommend you keep these related to the same categories; if you want to do concept testing for different products or services, we recommend setting up additional programs.

Qtip: Using the dropdown next to “Create new concept test project,” you can add and remove different types of projects to and from this program. For example, you could add custom dashboards you’ve created, or related ad hoc surveys you have run.

In the center is a watchlist. This is a helpful place to keep track of the overall performance of your concept tests. The widgets you’ll see include:

  • Concept test totals: The number of concept test projects in your program, and the total number of concepts tested across all projects. For example, if you have one concept test project with four different concepts in it, you’d see one project and a total of four concepts tested.
  • Latest concept test summary – Concept most likely to purchase: Amongst the concepts tested, which one respondents seem most likely to purchase. This is only displayed for sequential monadic tests.
  • Top 10 concepts: A list of the top 10 concepts tested. In addition to name, the concept test where the concept is from, and the date tested, you’ll also see the following data:
    • Concept score: The percent of respondents who said they would be extremely likely or somewhat likely to purchase.
    • Tier: The program automatically stack ranks all concepts tested within the program and breaks them into quartiles, so you can easily see what has been testing in the top 25% (Tier 1), bottom 25% (Tier 5), etc.
    • Sample size: The number of people who responded to the concept test this concept was presented in.
      Qtip: If you click “See all concepts” at the bottom, you’ll get a list of all the concepts tested across your program, including their tier, concept score, and sample size.
Qtip: Data may reindex while collecting responses, meaning these widgets may not display data right away.

To learn more about generally navigating a program, see the Programs support page.

Qtip: You can share this program with other users in your Qualtrics license.