A major advance in machine learning was to exploit the concept of the ‘Wisdom of Crowds’, in order to create an important new research area known as Ensemble Methods. Ensemble methods work by combining the output of trained models to make a decision about a new sample, such as its label or its membership in a cluster. In this thesis proposal, I argue that ensemble methods are a very general concept, and that they can be applied in many more situations than they have been previously. Instead of using them only to combine the output of an algorithm, it is also possible to apply them to decisions made inside the algorithm itself. I called this approach Inner Ensembles, and we discuss the idea in terms of its novelty, significance and feasibility. As for motivation for the work, we discuss using the wisdom of crowds concept in different ways, including robustness to variation, solving comprehensibility, and the simplicity of the resulting models. The main contribution of the thesis proposal will be to demonstrate just how broadly this idea can applied, and its potential impact on all types of algorithms. More specifically, I will show that in the case of classification, the idea can be applied to algorithms such as decision trees and Bayesian network.