A Deep Network with Visual Text Composition Behavior
PRESENTER:
Harry Guo
NRC
ABSTRACT:
While natural languages are compositional, how state-of-the-art neural
models
achieve compositionality is still unclear. We propose a deep
network, which
not only achieves competitive accuracy for text classification,
but also
exhibits compositional behavior. That is, while creating
hierarchical
representations of a piece of text, such as a sentence, the lower
layers of the
network distribute their layer-specific attention weights to
individual words.
In contrast, the higher layers compose meaningful phrases and
clauses, whose
lengths increase as the networks get deeper until fully composing
the
sentence.