Large volumes of data are currently produced under different application domains such as climate analysis, surveillance and communication networks. However, the rate of collecting information on some real-world systems can exceed our capacity of processing and crucial information to be lost. That is when the novelty detection comes up. Novelty detection for those applications is important for efficiently and automatically extracting important information, monitor unusual events and learn about their trends. Current novelty detection techniques, as most of machine learning techniques, need the manual selection of settings, called hyperparameters, which guide the learning process toward high-quality models. However, the time-varying nature of data streams can quickly turn hyperparameters and models outdated. At the same time, the amount of data prevent exhaustive searches for new values to hyperparameters. In this paper, we propose a new hyperparameter update approach for detecting novelties in data streams. This approach adapts hyperparameters according to variation in amount of information of estimated models. The hyperparameters adaptation is implemented on the SONDE (Self-Organizing Novelty Detection) Neural Network, which is inspired by Growing When Required and Adaptive Resonance Theory Neural Networks. We describe simulations on a well-known time series competition dataset and experiments considering a greenhouse-effect related dataset that consists in daily measurements of Outgoing Longwave Radiation.