Unsupervised grammar induction where a grammar is inferred from example corpora is a challenge that can help us build better natural language parsers and help us understand how humans learn language. Recent results challenge the idea that linguistic competence is categorical and discrete. In this research, we are developing a probabilistic model that tries to make use of word frequency, gradience of categories and morphological productivity. Our model uses probabilistic hidden structures of language by building a probabilistic tree of grammatical elements containing lexical and categorical information instead of relying on simple statistical surface facts of language. We keep track of word dependencies which can be used to parse sentences. A bootstrapping technique is integrated in the learning process by using information from partial parses based on the learned grammar. This enhances the learning and allows the model to learn higher level and more abstract word relationships.