+**********************************************************************
*
*
* Einladung
*
*
*
* Informatik-Oberseminar
*
*
*
+**********************************************************************
Zeit: Freitag, 12. Juli 2019, 10.00 Uhr
Ort: Informatikzentrum, E3, Raum 9222
Referent: Dipl.-Inform. Malte Nuhn
Thema: Unsupervised Training with Applications in Natural Language
Processing//
Abstract:
The state-of-the-art algorithms for various natural language processing
tasks require large amounts of labeled training data. At the same time,
obtaining labeled data of high quality is often the most costly step in
setting up natural language processing systems.Opposed to this,
unlabeled data is much cheaper to obtain and available in larger
amounts.Currently, only few training algorithms make use of unlabeled
data. In practice, training with only unlabeled data is not performed at
all. In this thesis, we study how unlabeled data can be used to train a
variety of models used in natural language processing. In particular, we
study models applicable to solving substitution ciphers, spelling
correction, and machine translation. This thesis lays the groundwork for
unsupervised training by presenting and analyzing the corresponding
models and unsupervised training problems in a consistent manner.We show
that the unsupervised training problem that occurs when breaking
one-to-one substitution ciphers is equivalent to the quadratic
assignment problem (QAP) if a bigram language model is incorporated and
therefore NP-hard. Based on this analysis, we present an effective
algorithm for unsupervised training for deterministic substitutions. In
the case of English one-to-one substitution ciphers, we show that our
novel algorithm achieves results close to human performance, as
presented in [Shannon 49].
Also, with this algorithm, we present, to the best of our knowledge, the
first automatic decipherment of the second part of the Beale
ciphers.Further, for the task of spelling correction, we work out the
details of the EM algorithm [Dempster & Laird + 77] and experimentally
show that the error rates achieved using purely unsupervised training
reach those of supervised training.For handling large vocabularies, we
introduce a novel model initialization as well as multiple training
procedures that significantly speed up training without hurting the
performance of the resulting models significantly.By incorporating an
alignment model, we further extend this model such that it can be
applied to the task of machine translation. We show that the true
lexical and alignment model parameters can be learned without any
labeled data: We experimentally show that the corresponding likelihood
function attains its maximum for the true model parameters if a
sufficient amount of unlabeled data is available. Further, for the
problem of spelling correction with symbol substitutions and local
swaps, we also show experimentally that the performance achieved with
purely unsupervised EM training reaches that of supervised training.
Finally, using the methods developed in this thesis, we present results
on an unsupervised training task for machine translation with a ten
times larger vocabulary than that of tasks investigated in previous work.
Es laden ein: die Dozentinnen und Dozenten der Informatik
_______________________________________________
--
--
Stephanie Jansen
Faculty of Mathematics, Computer Science and Natural Sciences
HLTPR - Human Language Technology and Pattern Recognition
RWTH Aachen University
Ahornstraße 55
D-52074 Aachen
Tel. Frau Jansen: +49 241 80-216 06
Tel. Frau Andersen: +49 241 80-216 01
Fax: +49 241 80-22219
sek(a)i6.informatik.rwth-aachen.de
www.hltpr.rwth-aachen.de
Tel: +49 241 80-216 01/06
Fax: +49 241 80-22219
sek(a)i6.informatik.rwth-aachen.de
www.hltpr.rwth-aachen.de
+**********************************************************************
*
*
* Einladung
*
*
*
* Informatik-Oberseminar
*
*
*
+**********************************************************************
Zeit: Freitag, 08. Mai 2020, 11.00 Uhr
Zoom:
https://rwth.zoom.us/j/94637532047?pwd=aVlweVRLdGU0OWNjdEh4TmJhSktrUT09
Referent: Dipl.-Medieninf. István Koren
Lehrstuhl Informatik 5
Thema: DevOpsUse: Community-Driven Continuous Innovation of Web Information
Infrastructures
Abstract:
The steady evolution of the Web over the last thirty years was shaped by an
interplay of new technologies and innovative applications. The current
challenges are caused by the ongoing digital transformation of whole
societies. In Industry 4.0 for example, these are changing workplace
settings and the adoption of the Internet of Things. Inhibiting the demanded
fast innovation cycles, this may create a disruptive and unstable
environment in which the requirements of heterogeneous professional
communities need to be addressed.
Information systems infrastructure, while only partially visible and thus
hard to grasp, has a strong influence on practices in professional
communities. Therefore, our aim is to stabilize the dichotomies apparent in
the Web by means of an agile information systems development methodology. It
supports the evolution of infrastructure through community-driven and
model-based technologies to guide it on a sustainable path of continuous
innovation. Agile development practices in software engineering, in
particular the already established DevOps approach, promote stronger
cooperation between development and operating teams. Our DevOpsUse
methodology additionally fosters a stronger involvement of end user
communities in software development processes by including them in the
process of infrastructuring, i.e. the appropriation of infrastructure during
its usage.
The developed DevOpsUse methodology has been successfully validated by the
transitions between three generations of technologies: near real-time
peer-to-peer Web architectures, edge computing, and the Internet of Things.
In particular, we were able to demonstrate our methodologys capabilities
through longitudinal studies in several large-scale international
digitalization projects. Beyond Web information systems, the framework and
its open source tools are applicable in further innovative areas like mixed
reality. Its broad adaptability testifies that DevOpsUse has the potential
to unlock sustainable innovation capabilities.
Es laden ein: die Dozentinnen und Dozenten der Informatik
_______________________________
Leany Maaßen
RWTH Aachen University
Lehrstuhl Informatik 5, LuFG Informatik 5
Prof. Dr. Stefan Decker, Prof. Dr. Matthias Jarke,
Prof. Gerhard Lakemeyer Ph.D.
Ahornstrasse 55
D-52074 Aachen
Tel: 0241-80-21509
Fax: 0241-80-22321
E-Mail: maassen(a)dbis.rwth-aachen.de
******************************************************
*
*
* Einladung
*
*
*
* Informatik-Oberseminar
*
*
*
*******************************************************
Zeit: Dienstag, 05. Mai 2020, 14.00 Uhr
Zoom:
https://us02web.zoom.us/j/84813327259?pwd=Y2NydlRMRzE1dkpkcmpERkFwMWZYZz09
Referent: Kazuki Irie, M.Sc.
Thema: Advancing Neural Language Modeling in Automatic Speech Recognition//
Abstract:
Statistical language modeling is one of the fundamental problems in
natural language processing. In the recent years, language modeling has
seen great advances by active research and engineering efforts in
applying artificial neural networks, especially those which are
recurrent. The application of neural language models to speech
recognition has now become well established and ubiquitous. Despite this
impression of some degree of maturity, we claim that the full potential
of the neural network based language modeling is yet to be explored. In
this thesis, we further advance neural language modeling in automatic
speech recognition, by investigating a number of new perspectives. From
the architectural view point, we investigate the newly proposed
Transformer neural networks for language modeling application. The
original model architecture proposed for machine translation is studied
and modified to accommodate the specific task of language modeling.
Particularly deep models with about one hundred layers are developed. We
present an in-depth comparison with the state-of-the-art recurrent
neural network language models based on the long short-term memory.
While scaling up language modeling to larger scale datasets, the
diversity of the data emerges as an opportunity and a challenge. The
current state-of-the-art neural language modeling lacks a mechanism of
handling diverse data from different domains for a single model to
perform well across different domains. In this context, we introduce
domain robust language modeling with neural networks, and propose two
solutions. As a first solution, we propose a new type of adaptive
mixture of experts model which is fully based on neural networks. In the
second approach, we investigate knowledge distillation from multiple
domain expert models, as a solution to the large model size problem seen
in the first approach. Methods for practical applications of knowledge
distillation to large vocabulary language modeling are proposed, and
studied to a large extent.
Finally, we investigate the potential of neural language models to
leverage long-span cross-sentence contexts for cross-utterance speech
recognition. The appropriate training method for such a scenario is
under-explored in the existing works. We carry out systematic
comparisons of the training methods, allowing us to achieve improvements
in cross-utterance speech recognition. In the same context, we study the
sequence length robustness for both recurrent neural networks based on
the long short-term memory and Transformers, because such a robustness
is one of the fundamental properties we wish to have, in neural networks
with the ability to handle variable length contexts. Throughout the
thesis, we tackle these problems through novel perspectives of neural
language modeling, while keeping the traditional spirit of language
modeling in speech recognition.
Es laden ein: die Dozentinnen und Dozenten der Informatik
--
--
Stephanie Jansen
Faculty of Mathematics, Computer Science and Natural Sciences
HLTPR - Human Language Technology and Pattern Recognition
RWTH Aachen University
Ahornstraße 55
D-52074 Aachen
Tel. Frau Jansen: +49 241 80-216 06
Tel. Frau Andersen: +49 241 80-216 01
Fax: +49 241 80-22219
sek(a)i6.informatik.rwth-aachen.de
www.hltpr.rwth-aachen.de
Tel: +49 241 80-216 01/06
Fax: +49 241 80-22219
sek(a)i6.informatik.rwth-aachen.de
www.hltpr.rwth-aachen.de
Hallo,
Matthias Wichtlhuber vom DeCIX hält nächsten Dienstag für die Aachener
Informatik einen Vortrag über Gegenmaßnahmen zu
Distributed-Denial-of-Service (DDoS) attacks am DeCIX und über die
Traffic Veränderungen am DeCIX aufgrund des COVID-19 lockdowns.
Wann: Dienstag 24.4. 11:00
Zoom Meeting ID: 916 2538 0906, Password: 091338
https://rwth.zoom.us/j/91625380906?pwd=cEJCSk01VkVEUUpTSjgzYTlnNk5Odz09
Abstract des Vortrags:
We will start with a short introduction of Internet Exchange Points
(IXPs) and their operations. With this background, we will discuss the
topic of Distributed-Denial-of-Service (DDoS) attacks at IXPs. We
present results of a recent measurement study of DDoS-for-hire Websites
(Booters) and the traffic effects of a seizure operation of 15 Booter
websites by the FBI in late 2018 [1]. Subsequently, we introduce the
concept of Advanced Blackholing, a published and operational mechanism
designed by the DE-CIX research team to defend IXP links against DDoS
attacks [2]. Due to the current COVID-19 situation, we will close the
presentation by showing some preliminary results on the traffic shifts
caused by nation-wide lockdowns in several countries.
Über Matthias Wichtlhuber:
Matthias Wichtlhuber holds a Diploma in Information Systems from
Universität Mannheim and a Ph. D. from Technische Universität Darmstadt.
His Ph. D. thesis focused on optimizing content delivery on the
Internet. During his Ph. D. he worked on numerous EU and nationally
funded research projects as well as industrial projects. After his
studies, he joined DE-CIX, the operator of the largest Internet Exchange
Point (IXP) in the world in Frankfurt. He is a member of the DE-CIX
research team and works on product development, system security, future
network architectures for Internet exchange points, and large-scale
network data analysis.
[1] Kopp, D., Wichtlhuber, M., Poese, I., Santanna, J., Hohlfeld, O., &
Dietzel, C.: “DDoS Hide & Seek: On the Effectiveness of a Booter
Services Takedown”, ACM IMC, 2019.
[2] Dietzel, C., Wichtlhuber, M., Smaragdakis, G., & Feldmann, A.:
“Stellar: Network Attack Mitigation using Advanced Blackholing”, ACM
CoNEXT, 2018.