To summarize, this paper proposes contextual string embeddings, a novel type of word embeddings based on character-level language modeling, and their use in a state-of-the-art sequence labeling architecture.
we • illustrate how we extract such representations from a character-level neural language model, and  integrate them into a simplified sequence labeling architecture; • present experiments in which we quantiatively evaluate the usefulness and inherent semantics of the proposed embeddings against previous embeddings and their stacked combinations in downstream tasks; • report a new state-of-the-art on the CONLL03 NER task for English (93.09 F1, ↑0.87 pp vs. previous best) and German (88.33 F1, ↑9.56 pp vs. previous best), and state-of-the-art scores for chunking and PoS; • release all code and pre-trained language models in a simple-to-use framework to the research community, to enable reproduction of these experiments and application of our proposed embeddings to other tasks.