Surveys and Scales

Choosing/writing questions/items:

• Will respondents understand the question?
• Will they have the necessary information to answer it?
• Will they be willing to answer it?
• Does the question “lead” respondents toward a particular answer (influence them to show themselves in a good light or anticipate what they think researchers want to hear)?
• Fewest number of items you can tolerate
• If the item is not measuring a variable to analysis or description of sample, delete it

Instructions: clear, consider participants in choosing how they respond, pilot the survey

Order of questions: 

• Influence: will some questions influence the answers to later questions?
• Demographic questions – first (easy, gets participants involved but may be boring) or last (easy to finish when participants are tired)?

Types of questions: 

• Closed (easier to analyze, does not allow for option you may not have considered; easier for participants) 
• Open (have to be coded but allows for more depth; participants may get tired and skip or not write much)
• Check questions: Questions asked in a different way multiple times 
• Polarity rotation questions – questions worded negatively to be sure participants are paying attention

Scales (two most common) 

• At least three scale items for any variable measured
• scales allow you to use statistics that open ended questions may not
• need to report reliabilities of scales: Cronbach Alpha

Likert

Within this specific course, how well do the following behaviors, thoughts, and feelings describe you? (1)
Staying up on the readings
1                        2                        3                        4                                    5           
Not characteristic of me at all                                                      Very characteristic of me

Semantic Differential

Describe the quality of communication in this conversation (2)
Relaxed             1            2            3            4            5            6            7            8            9        Strained

Resources:

General guide for survey research and design: 

http://writing.colostate.edu/guides/guide.cfm

Scale construction and reliability resources

http://www.personal.kent.edu/~dfresco/CRM_Readings/Clark_and_Watson_1995.pdf

http://www.joe.org/joe/1999april/tt3.php

http://www.statisticshell.com/docs/reliability.pdf

Levels of Measurement for scales

Writing survey items - closed versus open ended questions

Guide for writing survey items - University of Central Florida

Explanation of various types of survey questions

Comparing open and closed ended survey questions - overview

Indepth explanation of open versus closed questions

*Source: Reinard, J.C. (2008). Introduction to Communication Research, 4th Edition. New York: McGraw-Hill.

(1) Dixson, M.D. (2010). Creating effective student engagement in online courses: What do students find engaging? Journal of Scholarship of Teaching and Learning, 10(2), pp. 1-13.

(2) Duck, S., Rutt, D.J., Hurst, M.H., & Strejc, H. (1991). Some evident truths about conversations in everyday relationships: All communications are not created equal. Human Communication Research, 18, 228-267.

Dixson, Mack @ FACET, 2014