Evolving from the Use of P-Values to Multi-Armed Bandits Algorithms for Marketing Analytics
DOI:
https://doi.org/10.29145/jqm.82.05Keywords:
P-Values, Marketing, Investments, Machine Learning, Quantitative, Artificial Intelligence, Data MiningAbstract
Marketing Analytics is an important application of analytics in modern firms. Being able to evaluate and improve the impact of marketing investments drives sales, acquisition. retention and customer loyalty. The use of hypothesis testing to evaluate the effectiveness of marketing campaigns is an important application of marketing analytics. However, marketing analysts either follow the social sciences gold standard of rejecting the null hypothesis if the probability is less than .05 or are unsure of which p-values to use to evaluate statistical significance. This research takes a critical look at the typical marketing investments and evaluates how the .05 level of significance is inappropriate for these types of investments. Finally, the research suggests a repeat testing approach via multi-armed bandit algorithms which are more suited to the types of investment marketing organizations are making and what is expected from their stakeholders.
Downloads
References
Auer, P., Cesa-Bianchi, N., Fischer, P. (2002) Finite Time Analysis of the Multiarmed Bandit Problem. Machine Learning, 47(2/3), pp. 235–256.
Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E. (2002). The nonstochastic multiarmed bandit problem. SIAM Journal on Computing, 32(1), pp. 48–77
Burnham K., P., Anderson, D. R. (2014). P values are only an index to evidence: 20th- vs. 21st-century statistical science. Ecology, 95(3), pp. 627–630.
Fisher, R., A. (1936). The Design of Experiments. Oliver and Boyd. London, UK
Fogarty, D., J. (2013) Global 500 Test and Learn: The Relationship between Business Experimentation and Explosive Growth in Large Global Firms. International Journal of Business and Management Tomorrow, vol. 3, no. 5, pp. 16-25.
Fogarty, D., J., Harrison, P., Jing, L., Yip, S. (2014) Beyond Sales and Awareness: Using Marketing Analytics for Improved Health Engagement and Outcomes. Applied Marketing Analytics, vol. 1, no. 1, pp. 13-20.
Driscoll, D., L., Appiah-Yeboah, A., Salib, P., Rupert, D., J. (2007) Merging Qualitative and Quantitative Data in Mixed Methods Research: How To and Why Not? Ecological and Environmental Anthropology, vol. 3, no. 1, pp. 19-28.
Gelman, A (2013) Misunderstanding the p-Value. Statistical Modeling, Causal Inference, and Social Science, Retrieved from: https://statmodeling.stat.columbia.edu/2013/03/12/misunderstanding-the-p-value/
Hawton, K., Kingsbury, S., Steinhardt, K., James, A. & Fagg, J. (1999). Op cite.
Johnson, R., B., Onwuegbuzie, A.,J., Turner, L.,A. (2011) Toward a Definition of Mixed Methods Research. Journal of Mixed Methods Research, vol. 1, no. 2, pp. 112-133.
Kaufmann, E., Cappe, O., Garivier, A. (2016). On the Complexity of Best-Arm Identification in Multi-Armed Bandit Models. Journal of Machine Learning Research, vol. 17, no. 1, pp. 1-42.
Koutras, M., V., Bersimis, S., Maravelakis, P., E. 2007) Statistical Process Control Using Shewhart Control Charts with Supplementary Runs Rules. Methodology and Computing in Applied Probability, vol. 9, no. 2, pp. 207-214.
Mannor, S., Tsitsiklis, J.N. (2003). The Sample Complexity of Exploration in the Multi-Armed Bandit Problem. In: Sixteenth Annual Conference on Computational Learning Theory.
McShane, B., Gal, D., Gelman, A., Robert, C., Tackett, J. (2019) Abandon Statistical Significance. The American Statistician, Vol. 17, no 4, pp. 23-34.
Mosteller, F. (1941). Note on application of runs to quality control charts. Annals of Mathematical Statistics, vol. 12, no, 1, pp. 228–231.
Neyman, J. (1950) First Course in Probability and Statistics. Henry Holt, New York.
Owens, D., Horrocks, J. & House, A. (2002). Fatal and non-fatal repetition of self-harm. British Journal of Psychiatry, 181, 193-199.
Page, E., S. (1955). Control charts with warning lines, Biometrics, vol. 42, no.2, pp. 243–257.
Slivkins, A. (2019), "Introduction to Multi-Armed Bandits", Foundations and Trends® in Machine Learning: Vol. 12: No. 1-2, pp 1-286. http://dx.doi.org/10.1561/2200000068
Surmenok, P. (2017). Contexual Bandits and Reinforcement Learning. Towards Data Science. Retrieved from: https://towardsdatascience.com/contextual-bandits-and-reinforcement-learning-6bdfeaece72a
Tourangeau, R., Rips. L., J., Rasinski, K. (2000). The Psychology of Survey Response
Cambridge: Cambridge University Press, Cambridge UK.
Vander Pas, S., L. (2010). Much ado about the p-value: Fisherian hypothesis testing versus an alternative test, with an application to highly-cited clinical research. Unpublished Thesis. Mathematisch Instituut, Universiteit Leiden.
Vermorel J., Mohri M. (2005). Multi-armed Bandit Algorithms and Empirical Evaluation. In: Gama J., Camacho R., Brazdil P.B., Jorge A.M., Torgo L. (eds) Machine Learning: ECML 2005. ECML 2005. Lecture Notes in Computer Science, vol 3720. Springer, Berlin, Heidelberg.
Villar, S., S., Bowdon, J., Wason, J. (2015). Multi-armed Bandit Models for the Optimal Design of Clinical Trials: Benefits and Challenges. Statistical Science, vol. 30, no. 2, pp. 199-215.
Western Electric Corporation, (1956). Western Electric, Statistical Quality Control Handbook, Indianapolis, Ind.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 David Fogarty
This work is licensed under a Creative Commons Attribution 4.0 International License.
JQM follows an open-access publishing policy and full text of all published articles is available free, immediately upon publication of an issue. The journal’s contents are published and distributed under the terms of the Creative Commons Attribution 4.0 International (CC-BY 4.0) license. Thus, the work submitted to the journal implies that it is original, unpublished work of the authors (neither published previously nor accepted/under consideration for publication elsewhere). On acceptance of a manuscript for publication, a corresponding author on the behalf of all co-authors of the manuscript will sign and submit a completed Copyright and Author Consent Form.
