This talk will introduce and discuss statistical issues in artificial intelligence. Those who do science on a computer are not often in a position to perform a rigourous theoretical analysis of the programs that they craft. Furthermore, many processes of interest in computer science involve non-normal data, small sample sizes, repeated comparisons between groups and many other statistical hobgoblins. This talk will begin by pointing out 12 myths of experimental analysis, quickly cover the T Test, then proceed to nonparametric tests, such as the Mann-Whitney-Wilcoxon. We will round out with a discussion of multiple independent comparisons, and how to correct for them with the Bonferroni inequality. We will demonstrate these techniques using Excel to perform the statistics, since it is adequate for many problems and easy to use.

PDF slides