Using Larger Cash Incentives to Reduce Non-Response

[i] in which different combinations of bills and amounts were implemented.  The results of both experiments showed that for Spanish-speaking and Black households, the combination of a $10 incentive comprised of one five-dollar bill and five one-dollar bills achieved higher cooperation rates than all the other experimental conditions.

In the third experiment, conducted in July 2003, households that informed us on the phone that they were under age 35 Age of Head (including Black & Spanish) were randomly assigned to six conditions of various incentive amounts and combinations of bills[ii].  The results of this experiment showed that the combination of one $10 bill and one $5 bill achieved higher cooperation rates than the other experimental conditions.

These results will be discussed in terms of their implications in light of the extensive research literature that exists about how cash incentives affect survey cooperation rates.  The possibility of a causal relationship between 1) the incentive amount and the response rate, 2) the number of bills sent and the response rate and 3) the denominations of bills used and the response rate will be examined.

Introduction

There is a long tradition in survey research to use cash incentives to raise overall response rates for a survey.  This technique dates back to the early 1930’s (Shuttleworth, 1931).  However, there is little evidence in the literature of differential incentives being targeted to specific demographic groups in order to raise the response rates among these groups.

This paper reports on methodological research that was conducted to learn how to increase response rates among Black households, Spanish-speaking households and younger households (e.g., age of householder 18 to 34 years).  During the course of this experimentation, the possibility arose that the number and denomination of bills sent could have an impact on cooperation.

Company X conducts extensive surveys to measure household television viewing through its National Station Index[iii] (NSI) Diary service, which is conducted several times each year in 210 local market areas throughout the U.S.  In most major market areas, Company X has a need for viewing data of demographic sub-groups, specifically under age 35 Age of Householder (AOH) households, Black households and Spanish-speaking households.  Company X currently identifies these subgroups by asking a question in the recruitment call about the age of the householder in all market areas, a question about the race of the householder in 92 market areas and a question about whether the householder speaks Spanish in 52 market areas, many of which overlap with the race-question market areas.

It is generally acknowledged that in recent years, the survey research industry has experienced a steady deterioration in phone and mail survey cooperation rates.  The Company X has also endured a continual deterioration in cooperation rates of its diary service over the last several decades which has also has had a negative impact on sample representation[iv].  The cooperation rates for under age 35, Black, and Spanish Language treatment diary households historically have been considerably lower than the cooperation rate for other age, race and non-Spanish language households.  Over the years, Company X has conducted numerous methodological studies, including many true experiments, in an effort to improve cooperation rates.  Research has been conducted on many phases of the recruitment process including: monetary incentive amounts, non-monetary incentives, the timing, number and content of contacts, the diary data collection tool, the method of delivery, the script, the skill of telephone interviewers and other procedural aspects (cf. T. and Bennett, 2001).  The results of numerous experiments have been incorporated into Company X’s current survey process.

Since the start of its diary service in the 1950s, Company X has used a small non-contingent monetary incentive to increase cooperation rates.  Company X was faced with a need to increase the representation of Black households and Spanish language households in markets with significant populations of these demographic groups and under age 35 households in markets where they were greatly under-represented.  Many years ago, based on experimental evidence, Company X instituted increased incentives to these households in certain markets with high Black or Spanish-Speaking penetrations or lower than acceptable sample representation.  At the time this current research was embarked upon, the incentive for all Spanish-speaking households and Black households was $5.  When they were initially instituted, these amounts were considerably more than were offered to households who were not identified as Black or Spanish speaking, who historically cooperate at a higher rate.

In May 2000, in an effort to increase cooperation among low cooperating households, the incentive structure was changed for non-Black non-Spanish-speaking households.  The incentive was increased to five dollars for households that refused to keep a diary, but who had a mailable address, and those that could not be reached on the phone (despite 15 call attempts) but for which a valid address was available.  In May of 2001, Company X further upgraded its incentive structure when it began sending $5 to all households that were identified in the RDD recruitment phone call as having a householder under the age of 50.  As a result, beginning in early 2001, Black households and Spanish Speaking households were no longer receiving a higher incentive than most of the other households in our survey.

Past Research on the Effects of Incentives in Surveys

It has been argued that the proper use of monetary incentives is second only to the importance of multiple contacts in improving mail survey response rates (Dillman 1991, Dillman 2000).  This is attributed to Social Exchange theory that presumes that even a small monetary incentive is effective as it makes the respondent feel a sense of social obligation, i.e., “they gave me a monetary gift, so now I should complete the survey for them.”

A review of the available literature on monetary incentives reveals many published studies that confirm their positive impact on response rates, but little data on the optimum level to maximize response rates at an affordable level or their impact on specific demographic subgroups.  In addition, little evidence has been collected on the impact on response rates of the number of bills sent or the denominations.

Kulka (1994) conducted an extensive review of empirical research and current research practices on the use of incentives with particular emphasis on “Hard to Reach” respondents.  He concluded that the literature supported the generally accepted belief that monetary incentives do increase response rates and can generally be attributed to the previously mentioned social exchange theory.  He noted that incentive payments have been used effectively for many years (Shuttleworth, 1931) to improve cooperation rates especially in mail surveys.  (Armstrong, 1975; Church, 1993; Duncan, 1979; Fox, Crask, and Kim, 1988; Harvey, 1987; Heberlein and Baumgartner, 1978; Hopkins and Gullickson, 1992; Kanuck and Berenson, 1975; Linsky, 1975; Yammarino, Skinner and Childers, 1991; Yu and Cooper, 1983).  He also supported Dillman’s assertion that “the importance of financial incentives is ‘second only’ (perhaps) to the use of follow-up mailings or prompts in improving response rates” (Kulka, 1994; p. 262).  His review of the literature on the effectiveness of prepaid versus contingent incentives found many studies (Armstrong, 1975; Berk, Mathiowetz, Ward and White, 1987; Berry and Kanouse, 1987; Blumberg, Fuller and Harr, 1974; Church, 1993; Furse and Stewart, 1982; Hopkins and Gullickson, 1992; James and Bolstein, 1992; Kanuck and Berenson, 1975, Linsky, 1975; Peck and Dresch, 1981; Skinner, Ferrell and Pride, 1984; Wotruba, 1966; Yu and Cooper, 1983) that showed non-contingent (i.e., those paid before the respondents have completed the task) incentives are more effective than promised incentives.

Kulka’s (1994) review of the literature on the effects of incentive size was rather inconclusive, in that most of the studies available for his review dealt with incentives of one dollar or less.  (Armstrong, 1975, Fox et al., 1988; James and Bolstein, 1992; Jobber and Saunders, 1988; Kanuck and Berenson, 1975; Linsky, 1975; Yammarino et al., 1991; Yu and Cooper, 1983).  He found relatively few studies that examined larger incentive amounts and those that did were mostly very small studies that surveyed small, specialized groups.  (Berry and Kanouse, 1987; Gunn and Rhodes, 1981; James and Bolstein, 1992).

In the Tailored Design Method, Dillman (2000) advised that a one-dollar bill is the smallest practical amount to send in a mail survey.  Dillman stated that “Support exists for somewhat larger amounts being more effective, but there are clearly diminishing returns, with far more of an increase coming from the first dollar than from five or ten times that amount,” (p. 168).  He did find two dollars to be even more effective in increasing response rates.  James and Bolstein (1992) found similar results with the one-dollar bill in their study and with the relative value of one dollar versus two dollars in their earlier studies (James and Bolstein, 1990).

Additional reviews of recent studies dealing with monetary incentives and their impact on telephone-survey response rates (Singer et al., 1999; Singer, Hoewyk, and Maher, 2000) have similar findings as to the effectiveness of non-contingent monetary incentives.  Shettle and Mooney (1999) concluded that monetary incentives in a mailed survey do not negatively influence data quality or result in respondent ill will.  One study (Warriner, Goyder, Gjertsen, Hohner and McSpurren, 1996) found that a five-dollar cash incentive was more effective than a two-dollar incentive and as effective as ten dollars.  The sample size was rather small however.

Very few studies reviewed were conducted on a large-scale national random sample of the general population.  Some focused on small sub-groups of the population, such as physicians (Berry and Kanouse, 1987; Gunn and Rhodes, 1981); college graduates (Shettle and Mooney, 1999); cable TV subscribers in a single metro area (James and Bolstein, 1990); owners of small contracting companies (James and Bolstein, 1992) or other specialized groups/areas.  Others were national in scope, but involved factors other than incentive or reported on one, two or perhaps three incentive amounts that do not correspond with Company X’s incentive structure (typically $1, $2 or $5 or larger amounts such as $20, $25 or $50) or examine the impact on demographic subgroups.

The literature is virtually unanimous in the conclusion that the addition of a small non-contingent monetary incentive will increase cooperation rates in mailed surveys and are more effective in that regard than the promised reward for completing the survey.  Of note, no reference could be found to an experiment that specifically examined the incremental impact of incentives to Black households or Spanish-speaking households or to the impact of the number or denomination of bills sent.

Purpose of the Research Being Reported in this Paper

At the time of this research, Black households and Spanish-speaking households in Company X’s diary sample are given special phone and mail treatments to try to raise their cooperation and representation.  Among these treatments, a $5 cash incentive, paid in single one-dollar bills, was mailed within the diary packet.

As stated above, for many years, Company X has sent out a five-dollar incentive (paid as five $1 bills) to Black households and Spanish-language sample households.  However, given the general erosion in cooperation that we were experiencing, it was imperative to determine what higher non-contingent incentive level would provide the most cost effective means of raising cooperation rates and sample representation among these groups.

In May 2000, Company X performed an experiment among Black households and Spanish-speaking households to determine the cooperation benefit of higher incentives at the  $6, $7, $8, and $10 levels.  The $6, $7 and $8 incentives were paid via one five-dollar bill and one, two or three one-dollar bills respectively.  The $10 incentive was paid via two five-dollar bills.  The $8 and $10 incentive amounts showed the highest cooperation gains.  ($8 Black +4.3 P.P. (S.E. = 1.4), $10 Black +5.1 P.P. (S.E. = 1.4), $8 Spanish + 5.1 P.P. (S.E. = 1.9), $10 Spanish + 8.0 P.P. (S.E. = 1.9)).  However, due to production phone and mail limitations at that time, the test was not able to take into account: 1) mention of the incentive amount in the recruitment call (among Black households) as was done in production, and 2) different combinations of bill denominations.  Since the May 2000 test did not reveal the most effective incentive methodology for production implementation, further experimentation with the $8 and $10 in the new Priority Mail production environment was needed.

The results of a major Company X experiment in November 2002 (T. and Lavrakas 2003) led to the conclusion that the most effective combination of mail treatments of those tested is to send $10 incentive via Priority Mail to AOH under age 35 HH and $5 incentive via Priority Mail to age 35 to 49 AOH HH.  This combination of mailers and incentives not only improved sample representation among these important demos, but it also improved their cooperation and greatly reduced the amount of over sampling required.  Based on this research, this incentive plan was implemented with the July 2003 diary measurement.

Even with this improved performance, the younger demographic groups (especially the under age 35 AOH households) were still under represented in the intab[v] sample.  Additionally, the AOH aged 50 and above households were still over represented in the intab sample and even though they are sent between $1 and $3 currently depending on how many TV’s in the HH and whether the phone number is listed or unlisted.

This paper will report on three large methodological experiments carried out throughout the United States.  These were designed to assess the impact on cooperation rates within key demographic groups, in particular: two experiments of sending $5, $8, or $10 monetary incentives in different combinations of bills in diary packets to Black households and Spanish-language sample households and a third experiment of sending $10, $12, or $15 monetary incentives in different combinations of bills in diary packets under age 35 Age of Head households.

The purpose of this experimentation with Black and Spanish-language households was to test the effect on diary-keeping cooperation levels of $8 or $10 monetary incentives.  The sample sizes in this experiment were designed to be large enough to detect small differences achieved among the $8 and $10 incremental treatments.  The objective was to determine cooperation levels of two samples with $8 and two samples with $10 incentive amounts during four weeks of the November ’01 Diary measurement.  Another objective was to determine the effect of how the money was paid, i.e., in what combination of bills.

The objective of the third experiment was to send either $12 or $15 to 18 to 34 Age of Householder test condition households (including Black & Spanish) during four weeks of the July ‘03 Diary measurement.  For these 18-34 year-old households, mentioning the revised $12 or $15 incentive amount within their recruitment call (except Spanish which mentioned “some money”), rather than the $10 that the remainder of this demo receives.  It was theorized that this additional increase in incentive would further improve the cooperation of this hard to reach demographic and thus improve their representation in the intab sample.

Methodology

Experimental Design – Experiments 1 and 2

The first experiment was conducted during the November 2001 diary measurement but was suspended after two weeks due to the unfortunate national circumstances surrounding terrorism threats of mailed anthrax and the fall-out from the events of 9/11/01.  Company X management stopped this experiment for weeks 3 and 4 as it was reasoned that the results of this experiment could be distorted by the atypical national mood because of the terrorism-related events and fears.  The experiment was repeated during all four weeks of the May 2002 measurement using the same criteria as the November experiment.

The November 2001 incentive experiment (experiment 1) was conducted in a production sample during the first two weeks of the November 2001 diary measurement period.  The experiment tested the effect on cooperation rates of sending standard diaries and materials to test households with incentives of $8 or $10.  In this experiment, all Black Race or Spanish-speaking households were mailed their diary packets via Priority Mail, regardless of the age of the householder (The standard treatment was to send diaries via Priority Mail only to households identified as under the age of 50).  The experimental conditions are outlined in Table 1.

Test Condition	Incentive Amount	Number of Original Telephone Records		Number of Households Mailed Diaries		Incentive Bills	Total Number of Bills	Number of $1 bills	Number of $5 bills
		Exp. 1	Exp. 2	Exp. 1	Exp. 2
Control	$5	163,667	158,315	3,074	6,308	5 $1 bills	5	5	0
1	$8	49,100	47,496	948	1,906	8 $1 bills	8	8	0
2	$8	49,099	47,496	944	1,877	1 $5 bill & 3 $1 bills	4	3	1
3	$10	49,099	47,494	954	1,918	1 $5 bill & 5 $1 bills	6	5	1
4	$10	49,099	47,494	982	1,886	2 $5 bills	2	0	2

Sample Size – November 2001 (Experiment 1).  For Black households, a random portion of 198,398 phone numbers (limited to the 92 Black treatment markets) out of the RDD sample of 1,257,485 were selected for possible inclusion in test condition treatments and 163,677 phone numbers were selected for possible inclusion in the control condition.  (See Table 1).  After recruitment phoning, qualified households within the test sample were assigned to test conditions based on the digits of a unique identifying “Household Number” to assign test treatment for these households.  For the Spanish language sample, a random portion of 137,355 phone numbers (limited to the 52 Spanish-language treatment markets) out of the RDD sample of 1,257,485 were selected for possible inclusion in test condition treatments and 84,596 phone numbers were selected for possible inclusion in the control condition.  The counts are shown in Table 2.

Test Condition	Incentive Amount	Number of Original Telephone Records		Number of Households Mailed Diaries		Incentive Bills	Total Number of Bills	Number of $1 bills	Number of $5 bills
		Exp. 1	Exp. 2	Exp. 1	Exp. 2
Control	$5	84,596	89,259	1,696	3,333	5 $1 bills	5	5	0
1	$8	33,839	35,703	694	1,395	8 $1 bills	8	8	0
2	$8	33,839	35,703	653	1,310	1 $5 bill & 3 $1 bills	4	3	1
3	$10	33,838	35,703	624	1,336	1 $5 bill & 5 $1 bills	6	5	1
4	$10	33,838	35,703	687	1,341	2 $5 bills	2	0	2

Sample Size – May 2002 (Experiment 2).  For the Black household sample, a random portion of 189,980 Black treatment market phone numbers out of the RDD sample of 1,289,753 were selected for possible inclusion in test condition treatments.  The counts are shown in Table 1.  For the Spanish language sample, a random portion of 142,812 Spanish treatment market phone numbers out of the RDD sample of 1,289,753 were selected for possible inclusion in test condition treatments.  The counts are shown in Table 2.  After recruitment phoning was completed, qualified households within the test sample were assigned to test conditions based on the digits of a unique identifying “Household Number” to assign test treatment for these households.

Incentives.  All cash incentive amounts for both experiments were sent in single dollar bill or five-dollar bill denominations.  Unpublished research conducted in the mid 1980’s by Company X   indicated that sending five $1 bills resulted in a higher cooperation rate than one $5 bill.  However, for the amounts being sent in this experiment, it was unclear what combination of bills would be most effective[i].  In the control condition, five one-dollar bills were mailed.  The first $8 condition (Test condition 1) was comprised of eight one-dollar bills and the second $8 condition (Test condition 2) was comprised of one five-dollar bill and three one-dollar bills.  The first $10 condition (Test condition 3) was comprised of five one-dollar bills and one five-dollar bill and the second $10 condition (Test condition4) was comprised of two five-dollar bills.  The entire incentive amount was mailed in the diary packet per Company X’s standard procedure.  All households received a pre-designated incentive amount based upon their test condition.

Phoning Treatment.  The selected phone records were flagged to receive an age of householder question[ii] during the recruitment call.  The purpose of the question was to gather age information for mailing and analytic purposes.  The selected phone records were also flagged to receive a Race question[iii] and / or a Spanish language use question[iv] during the recruitment call.  This was used to determine which households met the criteria for inclusion in the test.  All numbers were dialed as many as 15 times, at different days of the week and time of day.  Refusal conversion attempts were made for all initial refusals except those in which the household had specifically stated that they did not want to be called back.

Mail Treatment.  In each test condition, all Black Race and Spanish-speaking households for which a valid address was available were sent diaries with the special test incentive amounts.  Non-Black, non-Spanish-speaking households and households that refused to participate in the diary survey or were not contacted at the recruitment call stage were not included in the test condition treatments.  The mailing materials for all test conditions were sent within a Priority Mail envelope.

Data Analysis.  Three dependent variables were created to test the effects of the experimental treatment.  First, a Return variable was created to represent whether a diary was received from a household, regardless of whether it was completed correctly.  Second, a Cooperation variable was created for all sampled households to represent if a fully completed diary was received from a household.  Third, a Quality of Return variable was created indicating the proportion of households that returned diaries that were fully and correctly completed among all households that returned diaries in a specific experimental treatment condition.  Some households return diaries that are either blank or incorrect which are not reportable in the Ratings.  The Return Rate is a measure of the percentage of all diaries returned and is calculated by the formula:

Thus, the Return Rate is the number of households that return diaries, whether usable or not, as a percentage of households mailed.  For the purposes of this analysis, the Cooperation Rate (Company X’s InTab rate) was calculated using the following formula:

Thus, the Cooperation Rate is the number of households returning usable diaries as a percent of households mailed.  Due to the unique, multi-mode nature of our survey, it is not possible to adhere closely to the AAPOR standard definitions guidelines for calculating response rates.  (The reader should take careful note of the definition above.)

The Quality of Return Rate is defined for the purposes of this analysis as the number of intab households as a percentage of all households returning dairies.  In addition to these three dependent variables, several independent variables were created to examine the possible relationship between the number of bills and the cooperation rate and the denominations of the bills and the cooperation rate.

Tabulations of these three dependent variables will be presented including Chi Square statistical significance results.  For the Cooperation Rate dependent variable, which is the most critical to the survey, Analysis of Variance was conducted using the number of television sets in the household as a covariate.  (Among the households used in the three experiments, 29 did not report the number of television sets in their households and thus are excluded from these ANOVAs.)  Another covariate was the status of whether we had an address on the record before we contacted the household via phone to conduct the recruitment interview.

The next step in the analysis was to create 3 additional independent dichotomous dummy variables.  This allowed the ANOVA analysis to be followed up with logistic regression analysis

on the data for the following dichotomous independent variables:

1.           Test condition,

2.           Incentive amount,

3.           Denomination of bills and

4.           Number of bills

The dependent variable in this analysis was cooperation, i.e., did the household return a diary or diaries that Company X could use in it’s reports?  Variables for Spanish, Black, Region (Northeast, South, Midwest, and Midwest), unlisted/listed phone status, number of TV’s in the household and whether we had an address on the original phone file before recruitment call phase (matched), age and experiment were included in the regression analysis.  These were added because they are expected to be statistically significant covariates and thus their inclusion will make the tests of the key independent variables more precise/powerful.  These variables are known to have an impact on cooperation rates in our surveys.  These first two experiments were analyzed together,

Experimental Design – Experiment 3

This incentive experiment was conducted in a production sample during all four weeks of the July 2003 diary measurement period.  The experiment tested the effect on cooperation rates of sending standard diaries and materials to test households with various combinations of incentives of $12 or $15 randomly assigned to six conditions.  In this experiment, all under age 35 AOH households were mailed their diary packets via Priority Mail.  All other recruitment scripting, mail materials and phone/mailing time schedules were standard procedures for Company X.

Sample Size – July 2003.  For the Under age 35 AOH household sample, a random portion of 333,380 phone numbers (in all 210 markets) out of the RDD sample of 1,333,512 were selected for possible inclusion in test condition treatments.  After recruitment phoning, qualified households within the test sample were assigned to test conditions based on the digits of a unique identifying “Household Number” to assign test treatment for these households.

All households that were identified in the Diary recruitment call as having a Householder 18 – 34 years of age with a valid mailing address were randomly assigned into a test or control condition.  The test treatments were limited to only those households with an 18-to-34 year-old AOH including Black race and Spanish-speaking (as well as non-Black and non-Spanish-speaking) and accepted the diary.  The only change among the test group 18-to-34 year-old AOH households was that test households received either $12 or $15 instead of the standard $10.

Test Condition	Incentive Amount	Number of Original Telephone Records	Number of Households Mailed Diaries	Incentive Bills	Total Number of Bills	Number of $1 bills	Number of $5 bills	Number of $10 bills
Control	$10	313,329	26,094	One $5, Five $1	6	5	1	0
1	$12	66,676	2,025	One $10, Two $1	3	2	0	1
2	$12	66,676	2,000	Two $5, Two $1	4	2	2	0
3	$15	66,676	2,053	One $10, Five $1	6	5	0	1
4	$15	66,676	1,931	Two $5, Five $1	7	5	2	0
5	$15	66,676	1,930	One $10, One $5	2	0	1	1

Incentives.  In this third experiment, households that informed us on the phone that the Householder’s age was under 35 (including Black & Spanish) were randomly assigned to one of six conditions of various incentive amounts and combinations of bills.  All under age 35 households received a pre-designated incentive amount based upon their test condition.  One condition involved sending $10 incentive (control) to households that informed us on the phone that the householder was under the age of 35.  Two conditions involved sending $12 and three conditions involved sending $15.   One $12 condition (Test condition 1) consisted of one $10 bill and two $1 bills and the other (Test condition 2) had two $5 bills and two $1 bills.  The first $15 condition (Test condition 3) included one $10 bill and five $1 bills.  The second $15 condition (Test condition 4) had two $5 bills and five $1 bills.  The third $15 condition (Test condition 5) consisted of one $10 bill and one $5 bill.  The entire incentive amount was mailed in the diary packet.

Phoning Treatment.  The selected phone records were flagged to receive an age of householder question during the recruitment call.  This was used to determine which households met the criteria for inclusion in the test.  In some markets, selected phone records were also flagged to receive a Race question and / or a Spanish-language use question during the recruitment call.  All numbers were dialed as many as 15 times, at different days of the week and time of day.  Refusal conversion attempts were made for all initial refusals except those in which the household had specifically stated that they did not want to be called back.

Mail Treatment.  In each test condition, all under age 35 AOH households with a valid address were sent diaries with the special test incentive amounts.  Households with an over age of 35 Householder (or refused to tell us) or who did not accept the diary (whether by refusing or not being contacted during recruitment phoning) were not included in the test condition treatments.  The mailing materials for all test conditions were sent within a Priority Mail envelope.

Data Analysis.  The dependent variable in this analysis was cooperation, i.e., did the household return a diary or diaries that Company X could use in it’s reports?  Variables for Spanish, Black, Region (Northeast, South, Midwest, and Midwest), unlisted/listed phone status, number of TV’s in the household and whether we had an address on the original phone file before recruitment call phase (matched) were included in the regression analysis.  These were added because they are expected to be statistically significant covariates and thus their inclusion will make the tests of the key independent variables more precise/powerful.  These variables are known to have an impact on cooperation rates in our surveys.

Results

1.           Test condition

First we will examine the results of the three experiments for return rate, cooperation rate and quality of return rate by test condition.  We will look at the Spanish Speaking sample separately from the Black-race sample for experiments 1 and 2 and then examine the Under-age-35 sample for the third experiment.  Then we will review the ANOVA and logistic regression results by test condition.

Table 4 details the results of the overall experiment for both the November 2001 Experiment 1 and the May 2002 Experiment 2 among Black households for each incentive condition, including the return rate, the cooperation rate, and the quality of the return rate.  Of note, both the return rate and the cooperation rate differences are highly significant for all test conditions in the May 2002 experiment, whereas test condition 3 was the only significant condition in the November 2001 experiment.  Additionally, test condition 3 showed a non-significant decrease in the quality of the return rate in November 2001 and a significant increase in the quality of return in the May 2002 experiment when compared to the control condition.

Test Condition	Incentive Bills	Incentive Amount	Return Rate		Cooperation Rate		Quality of Return Rate
			Exp. 1	Exp. 2	Exp. 1	Exp. 2	Exp. 1	Exp. 2
Control	5$1 bills (Control)	$5	36.5%	36.9%	30.2%	30.3%	82.5%	81.9%
1	8 $1 bills	$8	39.0%	41.6%***	32.3%	34.7%***	82.7%	83.5%
2	1 $5 bill and 3 $1 bills	$8	37.3%	41.2%***	31.1%	34.5%***	83.5%	83.7%
3	1 $5 bill and 5 $1 bills	$10	43.8%**	43.3%***	35.8%**	37.0%***	81.8%	85.4%***
4	2 $5 bills	$10	39.0%	42.7%***	32.9%	35.1%***	84.3%	82.1%

Test Condition	Incentive Bills	Incentive	Return Rate		Cooperation Rate		Quality of Return Rate
			Nov. ‘01	May ‘02	Nov. ‘01	May ‘02	Nov. ’01	May ‘02
Control	5$1 bills (Control)	$5	34.4%	36.7%	28.9%	31.0%	84.0%	84.5%
1	8 $1 bills	$8	36.3%	37.3%	30.5%	31.6%	84.1%	84.6%
2	1 $5 bill and 3 $1 bills	$8	36.0%	41.5%**	30.9%	34.7%*	86.0%	83.6%
3	1 $5 bill and 5 $1 bills	$10	39.3%*	40.8%**	34.5%*	35.3%**	87.8%*	86.6%
4	2 $5 bills	$10	37.7%	40.6%*	31.9%	34.6%*	84.6%	85.1%

Table 5 details the overall results of experiments 1 and 2 among all Spanish Language households for each test incentive condition, including the return rate, the cooperation rate, and the quality of the return rate.  Of note, both the return rate and the cooperation rate are highly significant for all experimental conditions in the May 2002 experiment, except condition 1.  As with the Black households, only test condition 3 was significantly different from the control condition for the return rate and the cooperation rate in the November 2001 experiment.  Additionally, of all the test conditions in both experiments only test condition 3 in November 2001 showed a statistically significant difference in the quality of the return rate when compared to the control condition.

An examination of cooperation by the Age of Householder by incentive condition for the Black treatment households in both the November 2001 and May 2002 experiments shows that for all three demographic age groups, test condition 3 achieved the highest cooperation rate in both November 2001 and May 2002.  An examination of cooperation by Age of Householder by the incentive condition for the Spanish Language treatment households in both the November 2001 and May 2002 experiments shows that the sample sizes were not large enough to show statistical significance for most conditions.  The cooperation rate, although trending positive for most conditions in the November 2001 experiment, did not differ significantly from the control in any of the Spanish Language conditions for any of the demographics.  The cooperation rate trended positive for all Spanish Language experimental conditions in the May 2002 experiment, but only two conditions were statistically different from the control condition.  Test condition 2 among age 35 to 49 households had a cooperation rate of .337 versus .287 for the control (P < .05) and test condition 3 among age 50 plus households had a cooperation rate of .432 versus .372 for the control (P < .05).

The results of the analysis of variance (ANOVA) for Experiments 1 and 2 showed that there was a significant main effect (p < .001) associated with Age, whereby the 18-34 AOHs had a mean cooperation rate of .253 vs. the rate of .309 for the 35-49 AOHs and .393 for the age 50 plus AOHs.  There was also a significant main effect associated with the incentive test condition (p < .001), which is somewhat consistent with the data presented in tables four and five.  There was also a significant main effect associated with the Region of the country (p < .001), with cooperation rates of .316 in the South, .322 in the Northeast, .348 in the Midwest and.353 in the West.

The ANOVA for Experiments 1 and 2 further revealed that there was a marginally significant main effect associated with Black or Spanish status (p < .08), but there was a significant two-way interaction effect (p < .001) between Black or Spanish status and age, whereby the cooperation rate among those households that were under age 35 and Spanish was .281, among those that were aged 35 to 49 and Spanish was .305 and among those that were aged 50 plus and Spanish was .383.  The cooperation rate for Black race households under the age of 35 was .234, among those that were Black race and aged 35 to 49 was .314 and the cooperation rate for those that were Black race and aged 50 plus was .404.  This shows that as the age of the householder increases, the cooperation of the household increased more among Black race households than among Spanish households.  Which week of the experiment did not have a significant effect on cooperation, though which experiment did have a significant but small impact (P < .01, B = .077) on the cooperation rate.

		B	S.E.	Wald	Df	Sig.	Exp(B)
Step 1(a)	Number of TV’s	-.082	.009	86.934	1	.000	.921
	AOH_Under35	-.634	.030	449.560	1	.000	.531
	AOH_35to49	-.345	.026	181.094	1	.000	.708
	Experiment	.077	.024	9.996	1	.002	1.080
	Listed	.095	.026	12.860	1	.000	1.099
	Matched Address	.147	.031	22.565	1	.000	1.158
	Northeast	-.170	.047	12.908	1	.000	.844
	South	-.182	.026	48.181	1	.000	.833
	Spanish	-.051	.023	4.724	1	.030	.951
	Test Condition 1	.120	.034	12.137	1	.000	1.128
	Test Condition 2	.145	.035	17.447	1	.000	1.156
	Test Condition 3	.273	.034	63.938	1	.000	1.314
	Test Condition 4	.182	.034	27.941	1	.000	1.199
	Constant	-.453	.063	52.092	1	.000	.636

Table 6 confirms that for Experiments 1 and 2, the logistic regression analysis results are consistent with the ANOVA results.  The suppressed reference categories of age 50 plus households, Black race, Midwest, and West all were significantly positive variables while the test condition 5 (control) was a significantly negative variable, (test conditions 1 through 4 were all significantly positive, P < .001).  Additionally, the Number of TV’s, the listed/unlisted status of the phone number associated with a particular household and whether we had a valid address on the original phone file were all significant factors in the equation.

Test Condition	Incentive Bills	Incentive Amount	Return Rate	Cooperation Rate	Quality of Return Rate
Control	1 $5 & 5 $1	$10	46.6%	42.0%	90.3%
1	1 $10, 2 $1	$12	44.1%*	39.8%*	90.2%
2	2 $5, 2 $1	$12	47.6%	43.8%	92.0%
3	1 $10, 5 $1	$15	46.7%	43.3%	92.7%**
4	2 $5 & 5 $1	$15	47.6%	43.3%	91.0%
5	1 $10 & 1 $5	$15	48.8%	45.5%**	93.2%***

 

Table 7 details the results of the overall experiment for the July 2003 Experiment 3 among all under-age-35 AOH households for each incentive condition, including the return rate, the cooperation rate, and the quality of the return rate.  None of the experimental conditions in the July 2003 experiment showed a statistically significant increase in the return rate compared to the control and test condition 1 actually showed a statistically significant decrease in both return rate and cooperation rate.  All other test conditions showed an increase in cooperation rate but only test condition 5 was significantly different from the control condition.  Additionally, of all of the test conditions only the two $15 conditions that included a $10 bill  (test conditions 3 and 5) showed a statistically significant difference in the quality of the return rate when compared to the control condition.

ANOVA in Experiment 3 showed that there was a significant main effect (p < .001) associated with the households status as ethnic (i.e., Black or Spanish Speaking) or non-ethnic (i.e., neither), whereby the Black households had a mean cooperation rate of .269, the Spanish Speaking households had a mean cooperation rate of .322 and the “non-ethnic” households rate had a mean cooperation rate of .459.  Of note, there was no significant main effect associated with the incentive test condition (P = 210) or the region of the country (P = .727).  There was a significant main effect associated with number of TV’s in the household, listed/unlisted status of the phone number, and whether a valid mailing address was available before recruitment phoning.

The results of the logistic regression analysis in Table 8 indicate that the main effect (previously identified with ANOVA) for Experiment 3 associated with the households’ status as Black or Spanish Speaking, or non-ethnic, is confirmed, as both the Black race dummy variable and the Spanish Speaking dummy variable are a highly significantly negative part of the regression equation.  Also confirmed are the significance of the impact on cooperation rates of the number of TV’s, the listed / unlisted status and the presence or absence of a matched address.

		B	S.E.	Wald	Df	Sig.	Exp(B)
Step 1(a)	Number of TV’s	-.167	.009	316.707	1	.000	.846
	Listed	.262	.035	55.564	1	.000	1.300
	Matched Address	.225	.038	34.896	1	.000	1.253
	Spanish	-.517	.037	197.476	1	.000	.596
	Black Race	-.675	.039	306.382	1	.000	.509
	South	-.108	.026	16.915	1	.000	.897
	Midwest	.073	.030	6.076	1	.014	1.076
	Test Condition 1	-.091	.048	3.593	1	.058	.913
	Test Condition 5	.140	.048	8.449	1	.004	1.150
	Constant	-.059	.034	2.976	1	.084	.943

However, the regression analysis for Experiment 3 (Table 8) does identify a significant positive main effect on cooperation for Test condition 5 (P < .01) and a marginally significant negative main effect for Test Condition 1 (P < .06).  This is consistent with the results presented in Table 7.  The regression analysis also identifies the presence of a significant negative main effect for the South region (P < .001) of the Region of the Country variable and a significant positive main effect for the Midwest (P < .05) region.  The cooperation rate for the South region was .387, the cooperation rate for the Midwest region was .469, while for the West it was .434 and for the Northeast it was .445.

1.           Incentive amount

To test the hypothesis that the incentive amount sent has an impact on the cooperation rate, several dichotomous independent variables were created to examine the cooperation level of each combination of conditions in the three experiments.  In the combined dataset of the first two experiments, variables were created for five dollars (control), eight dollars and ten dollars.  In the dataset of the third experiment, dichotomous dummy variables were created for ten dollars (control), twelve dollars and fifteen dollars.

		B	S.E.	Wald	Df	Sig.	Exp(B)
Step 1(a)	Number of TV’s	-.082	.009	86.731	1	.000	.921
	AOH Under35	-.633	.030	449.141	1	.000	.531
	AOH 35to49	-.345	.026	180.932	1	.000	.708
	Experiment	.078	.024	10.105	1	.001	1.081
	Listed	.095	.026	12.869	1	.000	1.099
	Matched Address	.147	.031	22.561	1	.000	1.158
	Northeast	-.170	.047	12.972	1	.000	.844
	South	-.182	.026	48.150	1	.000	.833
	Spanish	-.051	.023	4.776	1	.029	.950
	Eight Dollars	.133	.027	24.214	1	.000	1.142
	Ten Dollars	.227	.027	72.603	1	.000	1.255
	Constant	-.454	.063	52.350	1	.000	.635

ANOVA showed that there was a significant main effect (p < .001) in Experiments 1 and 2 associated with the incentive amount, whereby the mean cooperation rate among those sent $5 was .302 compared to a cooperation rate of .331 among those mailed $8 and .350 for those mailed $10.  As with the previous ANOVA, there are significant main effects associated with age and Region of the country and a significant two way interaction effect associated with age and Black or Spanish status.  However, there was a significant main effect associated with Black or Spanish status (p < .05), whereas the previous ANOVA for Test Condition showed a marginal main effect for Black or Spanish status.

The logistic regression data in Table 9 confirm ANOVA results for incentive amount as the cooperation rates for both eight dollars and ten dollars incentives are significantly positive compared to the control of five dollars incentive.  All other variables in the equation are similar to the results for the regression analysis of the incentive test condition. ANOVA showed that there was no significant main effect (p = .113) in Experiment 3 associated with Incentive, whereby the mean cooperation rate among from those sent $10 was .419 compared to a cooperation rate of .418 among those mailed $12 and .440 for those mailed $15.

Of note, table 10 shows that the logistic regression analysis does identify significant negative main effects of $10 (P < .01) and $12 (P < .05) incentives.  All other variables in the equation are similar to the results for the regression analysis of the incentive test condition.

		B	S.E.	Wald	Df	Sig.	Exp (B)
Step 1(a)	Number of TV’s	-.167	.009	316.102	1	.000	.846
	Listed	.262	.035	55.627	1	.000	1.300
	Matched Address	.225	.038	34.920	1	.000	1.253
	Spanish	-.517	.037	197.245	1	.000	.596
	Black Race	-.676	.039	306.575	1	.000	.509
	South	-.109	.026	17.102	1	.000	.897
	Midwest	.073	.030	6.062	1	.014	1.076
	Ten Dollars	-.084	.030	7.931	1	.005	.920
	Twelve Dollars	-.093	.042	4.882	1	.027	.911
	Constant	.014	.042	.112	1	.738	1.014

1.           Number of bills

To test the hypothesis that the number of bills sent has an impact on the cooperation rate, several dichotomous independent variables were created to examine the cooperation level of each combination of conditions in the three experiments.  In the combined dataset of the first two experiments, variables were created for two (two $5 bills), four (one $5 and three $1 bills), five (five $1 bills), six (one $5 and five $1 bills) and eight bills (eight $1 bills).  In the dataset of the third experiment, dichotomous dummy variables were created for two bills (one $10 and one $5 bill), three bills (one $10 and two $1 bills), four bills (two $5 and two $1 bills), six bills control (one $5 bill and five $1 bills), six bills test (one $10 and five $1 bills) and seven bills (two $5 bills and five $1 bills).

For Experiments 1 and 2 (See Table 11), the cooperation rate differences for all of the bill combinations are highly significant compared to the five-bill control condition.  As it is possible that the amount of the incentive is the underlying reason for this difference, an examination of the two combinations of number of bills that receive the same incentive amount, shows that there is no significant difference between the cooperation rates of the two $8 conditions (four bills and eight bills).  However, there is a significant difference of 1.9 P.P. between the two $10 conditions (P < .05, Two bills and six bills).

For Experiment 3, the difference between the control condition of six bills and two bills is highly significant and the difference of 2.1 P.P. between the control and three bills is marginally significant (P = .062) with the control actually showing higher cooperation (See Table 12).  The difference between the three bills and four bill conditions (both $12) of 4.0 P.P. is highly significant (P = .01).  Among three $15 conditions, the two-bill condition shows a non-significant 2.2 P.P. higher cooperation than the six bills (P < .143) and a non-significantly 2.2 P.P. higher cooperation than the seven bills condition (P = .161)

ANOVA showed that there was a significant main effect associated with the number of bills in Experiments 1 and 2, which confirms the finding in Table 11.  The logistic regression analysis (see Table 13) also confirms this, as all four dummy variables for number of bills are statistically significant higher in cooperation than the suppressed reference category of five bills (control, P < .001).  All other variables in the equation are similar to the results for the regression analysis of the incentive test condition.

ANOVA showed that there is no significant main effect associated with the number of bills in experiment 3.  However the logistic regression equation model (See Table 8, Two bills is the same variable as Test condition 5 and three bills is the same as test condition 1) indicates that the two-bill condition is statistically significantly positive (P < .01) and the three bills condition is marginally significantly positive (P < .06).

1.           Denomination of bills

To test the hypothesis that the denomination of bills sent has an impact on the cooperation rate, several independent variables were created to examine the cooperation level of each combination of conditions in the three experiments.  In the combined dataset of the first two experiments, separate variables were created for the presence of and number of one and five dollar bills.  In the dataset of the third experiment, separate dummy variables were created for the presence of and number of, one, five and ten dollar bills.

ANOVA for Experiments 1 and 2 showed that the presence of $5 bills had a significant main effect on the cooperation rate, whereas, the cooperation rate for the group of households sent $5 bills as part of their incentive was .345 and for those that did not receive $5 bills the cooperation rate was .309.  The logistic regression shown in Table 14 confirms this, as the P-value is less than .001.  All other variables in the equation are similar to the results for the regression analysis of the incentive test condition.

For Experiment 3 the ANOVA showed that the denomination of bills did not have a significant main effect, whereas the logistic regression indicates that there is a statistically significant negative impact of having $1 bills included (See Table 15, P < .01).  This appears to be a function of the fact that the highest cooperating test condition did not have $1 bills included and all other conditions did.

Discussion and Conclusions

The three experiments reported on here are important as they used very large sample sizes to test the effects on mail survey response rates of different incentive amounts, number of bills and denominations on hard to reach demographic subgroups.  Furthermore this was conducted across the entire United States[i].  Although previous research has consistently shown that use of a non-contingent incentive in a mail survey questionnaire, there has been very little research that has tested the impacts of these treatments on Blacks, Spanish speaking and under age 35 cohorts.  Additionally there has been very little consideration given in the literature about the impact of the number of bills or denomination of those bills on survey response rates.

In experiments 1 and 2, an interaction effect was found between the Age of the Householder and whether they were Black or Spanish speaking (as these were the only groups included in these two experiments).  As the age of the householder increases, the cooperation increased more among Black race households than among Spanish households, so it appears that older Black households, relative to younger Black households, are more likely to participate in our survey than are older Spanish speaking households relative to younger Spanish households.

In addition to this interaction effect, several significant main effects are noteworthy.  The main effect of age in experiments 1 and 2 highlights the importance of researchers attending to differential response rates among key demographic subgroups.  Younger adults were significantly less likely to respond than the older cohort.  In addition, the region of the country, and the Ethnicity of the Householder had a significant impact on the likelihood of cooperation.  An implication of this is if a researcher is especially concerned about the un-weighted sample demographics, then s/he should seriously consider differential incentives to demographic subgroups least likely to respond.

A practical implication of our findings is that, depending on how much value is placed on increasing response rates and demographic representation in a mailed survey, researchers need to make a reasoned/conscious decision about whether higher levels of incentives are needed for various important demographic sub-groups that may be know to or expected to cooperate a lower level.

The main effects of Incentive Amount and Test Condition generally show that “more is better.”  These findings are consistent with the literature about the impact of incentives on response rate in a mailed survey.

The main effect of the number of bills included in the incentive had mixed results.  The analysis of the first two experiments showed that all bill combinations had significantly higher cooperation than the control.  However, these combinations of bills also were delivered in higher amounts than the control so much of that difference is likely due to the higher incentive.  Since there is a high correlation between the number of bills and the incentive amount, we are unable to determine which factor was the determinant.  If there were a positive impact on the cooperation rate based on a higher or lower number of bills, we would have expected to see a pattern of lower cooperation on one side of the control, as the control with five bills was in the middle of the range of numbers of bills, but this was not the case.

The main effect of the denominations of the bills also had mixed results.  Experiments 1 and 2 results showed that there was a significantly positive main effect involved with the presence of $5 bills.  However, the Test Condition with one $5 bill and five $1 bills had a statistically significantly higher cooperation rate than the Test Condition with two $5 bills.  Experiment 3 showed that there was a significant negative main effect associated with presence of $1 bills.  This is in contrast to previously unpublished Company X research that showed that five $1 bills gained higher cooperation than did one $5 bill.  Interestingly, the highest cooperation level achieved in any of the Test Conditions in Experiment 3 included a combination of two higher denominations (one $5 bill and one $10 bill).  The experimental designs were not set up to test for the effect of denomination.  It is likely that the impact of this effect is more influenced by the amount of the incentive rather than the denominations of bills.  Further research into this relationship would be interesting to determine if indeed there is a causal relationship of note.

One interesting note in Experiment 3 was that the Quality of the Return Rate was significantly higher for those Test Conditions that received $10 bills with their survey packets.  This was not the case with those Test Conditions that received only $5 and $1 bills in their survey packet.

The results presented here show that there is much variation between the cooperation rates of different demographic groups.  It has been well established by Company X that in it’s samples, Blacks, Spanish Speaking and under age 35 AOH households cooperate at a lower level, which in turn means that they are under-represented in the intab sample.  Company X has dealt with this in the past with over sampling of these key demographics combined with weighting the data once collected.

Of note, as a result of the first two experiments, Company X began to send $10 Incentives to its Black and Spanish Speaking survey participants effective with the May 2003 diary measurement.  The same methodology as used in Test Condition 3 is being employed, i.e., one $5 bill and five $1 bills enclosed in the diary survey packet.

In addition Company X plans to conduct a follow up experiment to Experiment 3 in the near future to further investigate the effect on response rates of sending $15 using different combinations of bills to deliver the amount.  It is hoped that we can shed further light on the subject of the impact on the number and denomination of bills.

Appendix

Company X Diary Research Methodology

Company X conducts extensive surveys to measure household television viewing through its National Station Index[ii] (NSI) Diary service, which is conducted several times each year in 210 local market areas throughout the U.S.  This process is commonly referred to as “The Sweeps”.  Respondents in randomly sampled households throughout the U.S. are sent a paper television viewing data collection questionnaire booklet, hereafter referred to as a diary, to record demographic information and all household television viewing for a specified one-week period.  Company X’s metered service is used to produce the overnight ratings that appear in many newspapers.  The diary service is geared toward producing ratings on a market-by-market basis of the local TV viewing habits.

The process begins with the selection of a random-digit dial (RDD) sample of telephone numbers in each of 210 Designated Market Areas[iii], hereafter referred to as markets, across the United States.  These telephone numbers are matched to a database of available addresses and demographic information by a third party vendor.  Records are then processed through address verification software in an effort to improve accuracy.  For households with an address match, the first contact is a pre-recruitment post card announcing the household’s selection and forthcoming phone call from our research interviewer.  The second contact (or first for those with no address associated with their record) is a recruitment call requesting commitment to complete the diary.  Third, all households that (a) agree to participate in the RDD call, or (b) are not contacted[iv] or refuse, but have a valid address available, are sent an advance post card advising them that a diary will be mailed shortly.

Next, a diary packet is sent which includes a cover letter, a question and answer brochure, a small monetary incentive and, if the number of TV sets is known, a diary booklet for each TV (up to 5), or a single diary if the number is not known.  The diary booklet itself is designed to be a self-mailer for return to Company X TV.  Under current Company X procedures, homes with a householder under the age of 50 years are mailed their diary packets via Priority Mail … and age 50 and above homes are mailed their diary packet via first class mail.  The final contacts with each household, which occur during the diary-keeping week, consist of a) two post cards and one phone call or b) one post card and two phone calls, depending on the status of the household[v].  These final contacts are intended to remind the diary keeper to start the diary on Thursday and to return it at the end of the survey period.

Standard Company X incentives and treatments for non-ethnic households range from $1 to $5 and depend on the listed/unlisted status of the RDD phone number of the household, the number of televisions in the household, the age of the householder, and the status of the recruitment call (accept, refuse or no contact).  Five-dollar incentives are sent to all Black households in 88 markets and to all Spanish-speaking households in 52 markets.

References

Armstrong, J.S. (1975) “Monetary Incentives in Mail Surveys” Public Opinion Quarterly, 39, 111 – 116.

Berry, S. H. and D.E. Kanouse (1987) “Physicians Response to a Mailed Survey: An Experiment in Timing of Payment” Public Opinion Quarterly, 51, 102 – 114.

Berk, M. L., Mathiowetz, N. A., Ward, E.P., and White, A. A. (1987) “The Effect of Prepaid and Promised Incentives: Results of a Controlled Experiment” Journal of Official Statistics, 3, 449 – 457.

Blumberg, H. H., Fuller, C. and Hare, A. P. (1974) “Response rates in Postal Surveys” Public Opinion Quarterly, 38, 113 – 123.

Church, A. H. (1993) “Estimating the Effect of Incentives on Mail Survey Response Rates: A Meta-Analysis” Public Opinion Quarterly, 57, 62 – 79.

Dillman, D. A. (2000) “Mail and Internet Surveys: The Tailored Design Method”, John Wiley & Sons, Inc.

Dillman, D. A. (1991) “The Design and Administration of Mail Surveys” Annual Review of Sociology, 17, 225 – 249.

Duncan, W. J. (1979) “Mail Questionnaires in Survey Research: A Review of Response Inducement Techniques” Journal of Management, 5, 39 – 55.

Fox R. J., Crask, M.R., and Kim, J. (1988) “Mail Survey Response Rate: A Meta-Analysis of Selected Techniques for Inducing Response” Public Opinion Quarterly, 52, 467 – 491.

Furse, D. H. and D. W. Stewart (1982) “Monetary Incentives Versus promised Contribution to Charity: New Evidence on Mail Survey Response” Journal of Marketing Research, 19, 375 – 380.

Gunn, W. J. and I.N. Rhodes (1981) “Physician Response Rates to a Telephone Survey: Effects of Monetary Incentive Level” Public Opinion Quarterly, 45, 109 – 115.

Hopkins, K. D. and A.R. Gullickson (1992) “Response Rates in Survey Research: A meta-analysis of the Effects of Monetary Gratuities” Journal of Experimental Education, 61, 52 – 62.

James, J. M. and R. Bolstein, (1990) “The Effect of Monetary Incentives and Follow-up Mailings on the Response Rate and Response Quality on Mail Surveys” Public Opinion Quarterly, 54, 346 – 361.

James, J. M. and R. Bolstein, (1992) “Large Monetary Incentives and Their Effect on Mail Survey Response Rates” Public Opinion Quarterly, 56, 442 – 453.

Kanuck, L. and C. Berenson (1975) “Mail Surveys and Response Rates: A Literature Review” Journal of Marketing Research, 12, 440 – 453.

Kulka, R.A. (1994) “The Use of Incentives to Survey ‘Hard-to-Reach’ Respondents: A Brief Review of Empirical Research and Current Research Practice.”  Paper Presented at the Seminar on New Directions in Statistical Methodology, Council of Professional Agencies on Federal Statistics, Washington, D.C., May 25.

Linsky, A. S. (1975) “Stimulating Responses to Mailed Questionnaires: A Review” Public Opinion Quarterly, 39, 82 – 101.

Peck, J. K. and S. P. Dresch (1981) “Financial Incentives, Survey Response and Sample Representativeness: Does Money Matter?”  Review of Public Data Use, 9, 245 – 266.

Shettle, C and G. Mooney (1999) “Monetary Incentives in U.S. Government Surveys” Journal of Official Statistics, 15, 231 – 250.

Shuttleworth, F. K. (1931) “A Study of Questionnaire Technique” Journal of Educational Psychology, 22, 652 – 658.

Singer, E., Van Hoewyk, J, Gebler, N, Trivellore, T, McGonagle, K.. (1999) “The Effect of Incentives on Response Rates in Interviewer-Mediated Surveys” Journal of Official Statistics, 15, 231 – 250.

Singer, E., Van Hoewyk, J., and Maher, M. P. (2000) “Experiments with Incentives in Telephone Surveys” Public Opinion Quarterly, 64, 189 – 205.

Skinner, S. J., Ferrell, O.C. and Pride, W. M. (1984) “Personal versus Non-personal Incentives in Mail Surveys: Immediate versus delayed inducements” Journal of the Academy of Marketing Science, 12, 106 – 114.

T, N. and M. Bennett (2001)  “Large Scale Experimentation Using Priority Mail® Envelopes to Increase Household Cooperation Rates and Improve Sample Representation” P. 13, Presented at 56th Annual AAPOR conference in Montreal, Canada, May 20.

T, N. and P. J. Lavrakas (2002) “Various Monetary Incentives Experiment in a Mixed Mode Survey”, Presented at 57th Annual AAPOR conference in St. Pete Beach, Florida, May 18.

T, N. and P. J. Lavrakas, (2002) “Reducing Non-Response Among Targeted Demographic Subgroups Using Larger Cash Incentives”, Presented at the Midwest Association for Public Opinion Research annual conference in Chicago, Illinois, November 22.

T, N. and P. J. Lavrakas (2003)  “An Experimental Test of the Effects of Incentive Levels and Type of Mailer on Mail Survey Response in the Post-9/11 Era”, Presented at 58th Annual AAPOR conference, in Nashville Tennessee, May 16.

Warriner, K., Goyder, J., Gjertsen, H., Hohner, P., McSpurren, K. (1996) “Charities, No; Lotteries, No; Cash, Yes: Main Effects and Interactions in a Canadian Incentives Experiment” Public Opinion Quarterly, 60, 542 – 562.

Wotruba, T. R. (1966) “Monetary Inducements and Mail Questionnaire Response” Journal of Marketing Research, 3, 398 – 400.

Yammarino, F. J., Skinner, S. J., and Childers, T. L. (1991) “Understanding Mail Survey Response Behavior: A Meta-analysis.  Public Opinion Quarterly, 55, 613 – 639.

Yu, J., and H. Cooper (1983) “A Quantitative Review of Research Design Effects on Response rates to Questionnaires” Journal of Marketing Research, 20, 36 – 44.