Adaptive Input Representation

• Adaptive Input Representations for Neural Language Modeling
• varying the size of input word embeddings for neural language modeling
• improve accuracy while drastically reducing the number of model parameters
• more than twice as fast to train than the popular character input CNN while having a lower number of parameters
• English Wikipedia
• Billion Word