We present a ranking-based evaluation methodology for regression problems with non-uniform costs. In regression it is common to assume that all the prediction errors have the same relevance. Still, there are several relevant numeric prediction tasks (e.g. from ecology or finance), which assign different costs/benefits to each prediction. A sub-class of these regression tasks associates key events with some particular ranges of the target variable that are infrequent, e.g. buy a financial asset if the predicted return is above a certain threshold. For these problems a ranking evaluation of the model regarding the prediction of such events, considering a non-uniform costs setting, can be crucial for a proper model selection decision. We will describe an approach, based on the notion of utility scores of the predictions of the models, that provides a ranking evaluation of these models.