How to Forecast the Rollout Response of Mailing List from a Sample Test in Direct Mail

Testing new mailing lists is necessary in direct mail because the lists provide access to new and profitable customers, but testing is a costly affair. The high cost of testing can be justified if tests yield enough response to pay for it. The fact is that mailers (list users) often lose money on list testing. Therefore, what the mailers would like to do is to test new lists with minimum cost, and to conduct the test in such a way that the rollout response can be predicted with a reasonable amount of accuracy. Speed is an other concern in list testing. If a list is good, the mailer would like to use all the names right away. Without a satisfactory testing procedure, mailers hesitate to jump from a small test to a full run, no matter what the test results reveal. The procedure often used is something like this: First try 5,000 names; if they work, try another 10,000 names; if they also work, try another 25,000 names, and so on. By so doing, mailers at times miss the opportunity to reap the full benefit of a good list. This is because by the time the mailer is ready to use a large bulk of the list or the balance of it, the dynamics of the list might have changed. As a result, the list is no longer as productive as it was at the time of the initial test. The dynamics of a list can change for various reasons: 1. A competitor got a head start and used the whole list before anyone else. If the competitor has already used it, your response is not going to be as good as your test. This is because the more a list is used for the same offer, the lower will be the response. 2. The composition of the list has changed. The names on the list are no longer as new as they were before. The productivity of a list declines as the names become older. This list may now have customers obtained through other media such as space, TV, and radio, which are generally not as productive as the ones obtained through direct mail. The offer by the list owner might have changed. Previously, the average unit of sales was $50, and now it is $35. Previously, the owner of the list was selling only classic books, and now the owner also sells fiction books. All these can change the composition of a list and consequently the response. The economic environment also affects the response of a list. By the time the mailer decides to use the whole list or a large portion of it, the economic environment might have changed. Therefore, if a list is good, it is to the advantage of a mailer to use it in full as quickly as possible. The objective of this article is to outline a sampling procedure for testing new lists that is cost-effective, and at the same time, predicts the rollout response from a test with a fair amount of accuracy. The effectiveness of the procedure will be validated with real data. LITERATURE REVIEW Before outlining my own sampling procedure for testing new lists, it will be helpful to review the literature. A number of articles and monographs have been written on how to determine the size of a sample, and how to conduct a test and predict the rollout response from a sample test. Doppler, Knowlton, Silverman, and The Alan Drey Company (4,6-8) suggested first obtaining a random sample of a list to be tested. When the list is mailed and responses are in, estimate the permissible sampling error (plus or minus) to predict how the rollout response could vary from a sample test at different levels of confidence. The permissible sample error (e) is computed as follows: where e is the permissible sample error (+/-), n is the sample size, P is the sample test response (or expected response rate in percent), and L is the level of confidence expressed in terms of standard deviation units (i.e., 1 = 68%, 2 = 95%, and 3 = 99%). If n equals 2,500, P equals 2% (or .02 as a decimal), and L equals 1 (68% level of confidence), then This means, then, that the response of a subsequent test or rollout response will fall between 2% +/-0. …