We present a systematic analysis of twenty four performance measures used in the complete spectrum of Machine Learning classification tasks, i.e., binary, multi-class, multi-labelled, and hierarchical. We relate a set of changes in a confusion matrix to specific characteristics of data. Then the analysis concentrates on the type of changes to a confusion matrix that do not change a measure. The result is the measure taxonomy with respect to all relevant label distribution changes in a classification problem. We support the formal analysis with examples of applications where properties of measures lead to a more reliable evaluation of classifiers.