RDD Sampling vs. Listed Sampling
When choosing which sampling method to use, there are many things to consider.
On the top of that list is the trade-off between calling efficiency vs. statistical representation.
Random Digit Samples
A random digit (RDD) sample is a statistically constructed research sample where all telephone households have an equal and representative chance of being interviewed. RDD samples include all households, regardless of whether they are listed in a telephone directory or not. Since unlisted phone rates in most major metro areas typically run in excess of 50%, RDD sampling helps significantly reduce non-coverage bias.
Simply put, RDD samples are completely representative of the sampling universe, or frame, and can be “defended” from a methodological standpoint. RDD samples are typically used when the product/service category under study is in relatively widespread usage, and the researcher wants to ensure that the study results are statistically projectable.
Thus, marketing researchers tend to favor random digit samples for studies when representation and sample accuracy is of key importance.
Although RDD samples are more representative, they are typically less efficient than using listed samples -- especially for very low incidence studies. When cost is of greater consideration -- and the price cannot be adequately adjusted by lowering the sample size, reducing the questionnaire length, or modifying the screening criteria -- some form of listed sample is often used.
Listed samples are lists compiled primarily from telephone directories, but may include other sources as well. In general, only those households “listed” in the phone book will be included in a listed sample – hence the name. Listed samples do not cover the entire sampling frame, and leave out all households that have chosen to remain unlisted.
In general, listed samples can be more tightly targeted than random digit (RDD) samples due to the amount of detailed information available on many households. For instance, with a list, one can identify the exact street address, estimated income, gender, age, ethnicity, or number of children for any given household. In fact, STS offers hundreds of geo-demographic selects from which to choose from.
Listed samples enable very close targeting and, in addition to the highly favorable cost advantages resulting from lower qualifying incidence, may significantly shorten turnaround time for the study. For some projects, the number of qualified households may be so small that listed samples are the only way to conduct the research.
Many years ago, listed samples were essentially the same as RDD samples, in that almost every household was listed in the telephone directory. Nowadays, the cultural and legal emphasis on personal privacy has severely weakened the validity of listed samples. The fact a household may be included as part of a listed sample is now essentially a voluntary act on the part of the person to allow public access to their telephone number.
Unfortunately, the percentage of the population that is willing to have their telephone number published varies considerably by many factors, for example:
- Income level (both high and low income households are more prone to keep their phone unlisted)
- Age (many older people attempt to avoid calls)
- Gender (single female head-of-households are drastically under-represented in the telephone book)
- A host of lifestyle characteristics (including household mobility)
- And -- most importantly -- geography.
The problem for the researcher is that, from a statistical standpoint, there is a strong probability that the general population differs in significant ways from households that can be interviewed via a “listed” sample. In fact, over 50% to 65% of households in many major metropolitan areas choose not to list themselves in telephone directories. This significantly large “unlisted” rate is the primary factor that influences many quantitative research studies to use RDD sampling.
Choosing The Correct Sample Methodology
In order to choose the correct sampling methodology for any research project, it is important to weigh the cost benefits of listed sampling against the accuracy, randomness, and coverage of unlisted households that RDD samples deliver. This is partly a question of how “precise” the researcher needs to be in preparing estimates of the general population. Also, the qualifying incidence for the survey needs to be taken into account.
Some research firms seem to take the attitude that households with “unlisted” telephones are not too different than those with listed telephones. Other research suppliers are convinced that the differences may be great and would never recommend using a listed sample.
For a study in which nearly “everyone” qualifies, the cost advantages of dialing from a listed sample are rather minimal. The RDD sample will tend to have about twice as many “disconnects,” but if the qualifying incidence is high, the cost of dialing a few more disconnects will be the primary difference.
Disconnects are a particular problem in some of the larger markets such as New York, Los Angeles, and San Francisco. Using a cleaning technique often referred to as disconnect screening, the number of disconnects in most RDD samples can be reduced significantly.†
When the thought of using “listed” samples strays from the general population to smaller sub-segments, the potential cost benefits are magnified greatly -- but so is the potential loss of “randomness.” Furthermore, from the standpoint of the sample’s validity, it is generally overlooked that the variables which may be used for “targeting” the sample may be known for only a small percentage of households – thus, hampering list coverage of the population.
On the positive side, with low a qualifying incidence, there is a tremendous cost efficiency to be gained from using a “listed” sample. In fact, at very low incidence levels -- for example when less than 5% or 10% qualify -- it is often the case that the research simply cannot be funded unless a listed sample is used.
For further information, please contact your STS account representative or any member of our sampling team at (800) 944-4-STS.
† Note the STS does not guarantee or warrantee that the STS disconnect screening service will remove/identify a specific number of non-working or other non-productive numbers. Depending upon the geographic area, each region’s operating telephone companies, and other factors beyond our control, the STS disconnect screening service works at different efficiency levels and may find varying levels of non-working numbers.