UP504 • Survey Research

last updated: Saturday, April 12, 2008

link to survey research page


Assignment Four

due Monday, March 17

return to UP504 main page

Please write concise answers. Good survey research requires clarity, precision and consistency. See the survey research page (including the detailed class notes) for more information, including the course readings on survey research. As in previous assignments, work in teams of two students.


1. Measures: In urban research we often use concepts that are not easily measurable, and yet measures are necessary for quantitative urban research. Define simple measures of the following two concepts. For each, you may either locate and use standard measures (such as government definitions or accepted social science practices -- be sure to cite the source) or else develop your own. IMPORTANT: In either case, be specific and operational. Don't just describe the measure, but concisely explain how it is actually measured/calculated. (In other words, the reader should know what data is needed, what calculations would be performed on the data, what the scale is, and how to interpret the results -- e.g., does a higher number represent more or less segregation?)

Overall: it is important is to develop an operational indicator (rather than just a vague definition). It doesn't have to be a great, ideal measure, but it should be plausible -- and if it clearly has some shortcomings, some mention of this would be useful. Simple ("back-of-the-envelope") measures that are transparent, use readily available data, and are adaptable are often more effective than complex, opaque measures using unreliable data and/or problematic assumptions.

Also: if you are measuring a complex, multi-faceted concept, you may need to choose between two approaches: (a) developing a complex, multi-faceted measure (e.g., an aggregation of multiple simple measures, where you will also need to decide how to aggregate and weight each component), or (b) simply present a series of component measures and leave it up to the user/reader to decide how to aggregate.

a. racial segregation

There are several common ones, including measures "dissimilarity," "concentration," "clustering" and "isolation." Segregation is a complex concept open to different interpretations: one might calculate, for example, what percent of the population would need to move to another place to achieve a homogenous distribution. Alternately, one might view segregation as the opposite of propinquity (or the likelihood of contact/interaction) based on physical proximity (defined at a specific geographic scale, such as in the same block or census tract). Issues to address:

For a summary description of various segregation measure techniques, see U.S. Census Bureau, Housing and Household Economic Statistics Division, Housing Patterns - "Racial and Ethnic Residential Segregation in the United States: 1980-2000. APPENDIX B: MEASURES OF RESIDENTIAL SEGREGATION.
see also: Population Studies Center (UNiversity of Michigan), Racial Residential Segregation Measurement Project
Edward L. Glaeser and Jacob L. Vigdor. 2001. "Racial Segregation in the 2000 Census: Promising News." Center on Urban & Metropolitan Policy, Brookings. pdf


b. suburban sprawl

There is a wide range of possible answer. Better measures address not just population density, but also include other characteristics of "sprawl," such as

There are fine points about how to actually measure land. e.g., how does one deal with parkland and open space, lakes, etc.?

Some measures may deal directly with the sprawling patterns of land use, while others may instead choose to examine the consequences of sprawl (e.g., auto-dependence, increased pollution, etc.) The latter are not direct measures of sprawl, though these consequences may be correlated to the former. (However, that link is not uncontested, so direct measures of sprawl itself are preferable.

"Sprawl" may imply a contrast to a "non-sprawling" landscape. If so, you may need to define the threshold between a non-sprawling and sprawling landscape. This raises issues of calibration.

Note: measures vary on definitions of the concept: Dolores Hayden (2004, 8) defines sprawl as “a process of large-scale real estate development resulting in low-density, scattered, discontinuous car-dependent constructions, usually on the periphery of declining older suburbs and shrinking city centers”. But there are other competing measures.

several publications on measuring "sprawl":
Paul M. Torrens, Marina Alberti. 2000. "Measuring Sprawl," Centre for Advanced Spatial Analysis. Working Paper #27. pdf
Burchell, R W; Shad, N A; Listokin, D ; Phillips, H ; Downs, A ; Seskin, S ; Davis, J S; Moore, T ; Helton, D ; Gall, M. "The Costs of Sprawl - Revisited." (see especially Ch 1, Defining Sprawl) pdf


2. Sampling Scenario (answer limit: 1 page)
You are a graduate planning student writing a thesis on New Urbanism (a contemporary planning/design movement to build higher density communities that encourage greater walking and mass transit, promote more neighborhood identity, reduce the environmental impacts of urban development, etc.). You want to do a survey of home builders in the U.S. to examine their attitudes about New Urbanism. You are particularly interested in whether potential support for New Urbanist developments among the construction industry varies regionally, by size of builder, and by the housing type they construct. You contact the National Association of Home Builders (NAHB), which has over 65,000 members nationwide. They have a membership list for sale, which includes names, addresses, phone numbers, size of business, predominant type of housing constructed (e.g., single family, townhouses, etc.) and average sales price of housing units constructed. However, they want to charge you $20,000 for the list. After much negotiation, they agree to give you the names of 200 members for free. (This makes you happy, but you have concerns about the sampling error when making inferences from small subpopulations of your sample.) You can specify what criteria the data base manager at NAHB will use to select the 200 cases from the mailing list data base of over 65,000 names.

Your task: Develop a sampling strategy. How would you select your sample, and why?

Your approach will depend on whether you make the assumption that you can know the characteristics of the list as a whole.

If you DO assume that you know the characteristics of the list: then stratified sampling is the way to go. Knowing the breakdown of the 65,000 members by characteristics would help you identify what subgroups are relatively small (and thus would benefit from "over sampling" using stratified sampling to ensure a critical mass of respondents in each subgroup, such as builders of high-density, "New Urbanist" communities). The trick is how many dimensions to use to select strata: e.g., geography, size, type and cost of housing. It may not be easy or necessary to stratify along ALL these four dimensions -- especially with only 200 cases to use. Select what you think are the one or two key characteristics, such as geography (e.g., region) and housing type. It is easier to select strata if one has a good sense of what strata lead to the greatest variation of the dependent variable in question.

(If you don't make this assumption about knowing the characteristics of the list as a whole, then stratified sampling may be less effective because you don't know the relative size of each population strata within the population (65,000) as a whole.)

Remember: stratified sampling can have at least two benefits: (1) ensuring that relevant subgroups in the sample have enough respondents to allow for statistically significant conclusions about that subgroup; (2) reducing sampling error (especially when subgroups are relatively more homogenous than the population as a whole: that is, variation is greater across subgroups than within subgroups).


Note: for questions 3 and 4, consider not only the specific wording/formatting, but also the broader issues of categories, concepts vs. measures, scale, measurement units, any conflicts between the intent of the question and the way the question was asked, etc.

3. Survey Questionnaires (Mail)
For each of the following mail survey questions, briefly explain what is wrong with the question (if anything), and how it might lead to biased, inaccurate or otherwise poor results. Then suggest your own version of the question: 

a. What is your occupation now, and what was it five years ago? __________________________________
  Three major problems:

  1. two time periods asked -- separate into two questions.

  2. Use filtering questions to filter out those who were not employed now or five years ago.

  3. Many will confuse occupation with industrial sector, so best to use standard Census occupational categories -- either on the questionnaire itself (as a closed-ended question) or as a separate coded list.

b. How do you travel to work each day?
  ___ car
  ___ bus
  ___ subway
  ___ other

  1. use a filtering question for those who don't work -- or work at home (and good to know the difference between these two)
  2. use more options: e.g., walk, carpool, etc.
  3. allow for modal splits (i.e., combination of several modes)
  4. many commute trips involve walking (e.g., walking to the bus stop, even walking from the central city parking garage to the office). How will you treat walking as a component of an overall journey, especially if it is a secondary aspect?
  5. clarify "each day"-- what to do with people who vary their mode: e.g., ask their most common / typical mode, or "how did you get to work yesterday?" etc.

c. Would you be willing to ride mass transportation if it was available in your area? (Circle one number)
very willing
not willing at all

  1. Is the ordered scale the best approach? Perhaps it is too abstract. Instead, the responses will likely be more useful and accurate if alternatives/trade-offs presented.
  2. time and space frames are too vague
  3. what kind of transit, cost, alternatives? One should offer more specifics. e.g., bus, light-rail, trolley, etc.
  4. One will need to know more about the person's current transportation choices and options. (this would involve two stages: several preliminary questions about current habits and options, and then questions about anticipated behavioral changes to changes in mass transit options).

4. Survey Questionnaires (Telephone)
After conducting a telephone survey of 400 people, you get the following results. How could you have changed the question to get more useful results?

QUESTION: If gas prices went up $2.00 per gallon, how much less would you use your car?


[a range of answers were given, including more than one answer; these varied responses were grouped together as the following:]
  about the same 42%
  a bit less 27%
  between 10 and 30 percent less 12%
  less weekend trips 7%
  take public transit more to work 17%
  carpool 6%
  more than 30 percent less 17%
  give up driving 5%
  sell car 3%
  less summer trips 6%
  I would trade in my big car for a small one 9%
  I would look for a job closer to home 4%
  I would look for housing closer to work 2%

1. The disparate responses suggest that one should use a more closed-ended format: e.g., asking about either travel mileage changes (though that might be too abstract), or which trips might be forgone, etc. There are several dimensions of answers -- pick one, or use several questions to get at each dimension.

2. Distinguish between short-term and long-term: in the short-term one might simply absorb the additional costs (and spend less elsewhere), in the medium term one might change travel behavior, and in the long term one might buy a different kind of car (or even move closer to work). The extent of behavior changes will be a function of the respondent's expectations about the future price of fuel (e.g., is the price increase simply a temporary spike, or the start of a long-term increase in prices?).

3. This question is -- for an economist -- going after price elasticity with respect to demand. It might also be useful to know and specify the current price of gas -- and therefore be more clear about a $2 increase from what current level?

In conclusion, greater precision and a more structured question will lead to more consistent and useful answers.