|
|
Main menu for Browse IS/STAG
Course info
KIV / NLP-E
:
Course description
Department/Unit / Abbreviation
|
KIV
/
NLP-E
|
Academic Year
|
2024/2025
|
Academic Year
|
2024/2025
|
Title
|
Advanced Natural Language Processing
|
Form of course completion
|
Exam
|
Form of course completion
|
Exam
|
Accredited / Credits
|
Yes,
6
Cred.
|
Type of completion
|
Combined
|
Type of completion
|
Combined
|
Time requirements
|
Lecture
2
[Hours/Week]
Tutorial
2
[Hours/Week]
|
Course credit prior to examination
|
Yes
|
Course credit prior to examination
|
Yes
|
Automatic acceptance of credit before examination
|
No
|
Included in study average
|
YES
|
Language of instruction
|
English
|
Occ/max
|
|
|
|
Automatic acceptance of credit before examination
|
No
|
Summer semester
|
0 / -
|
0 / -
|
0 / -
|
Included in study average
|
YES
|
Winter semester
|
0 / -
|
0 / -
|
0 / -
|
Repeated registration
|
NO
|
Repeated registration
|
NO
|
Timetable
|
Yes
|
Semester taught
|
Winter + Summer
|
Semester taught
|
Winter + Summer
|
Minimum (B + C) students
|
5
|
Optional course |
Yes
|
Optional course
|
Yes
|
Language of instruction
|
English
|
Internship duration
|
0
|
No. of hours of on-premise lessons |
|
Evaluation scale |
1|2|3|4 |
Periodicity |
každý rok
|
Evaluation scale for credit before examination |
S|N |
Periodicita upřesnění |
|
Fundamental theoretical course |
No
|
Fundamental course |
Yes
|
Fundamental theoretical course |
No
|
Evaluation scale |
1|2|3|4 |
Evaluation scale for credit before examination |
S|N |
Substituted course
|
None
|
Preclusive courses
|
KIV/ANLP
|
Prerequisite courses
|
N/A
|
Informally recommended courses
|
KIV/SU and KIV/IR
|
Courses depending on this Course
|
KIV/NLSZ
|
Histogram of students' grades over the years:
Graphic PNG
,
XLS
|
Course objectives:
|
The course aims at providing students with advanced knowledge of modern approaches to natural language processing methods. Students obtain practical skill favoring them on the labor market. Besides, students will be able to start their research career in the subject research field.
|
Requirements on student
|
Completion and a defense of a semestral project. Passing an oral exam in front of a committee. Meeting all terms, returning the project before the deadline and an on-time registration for the exam.
|
Content
|
1. Revision: Multi-layer perceptron and Backpropagation.
2. Language models and Word2Vec.
3. Convolutional neural networks.
4. Recurrent neural networks.
5. LSTM, GRU, tagging.
6. Encoder-decoder architecture, machine translation.
7. Attention principle.
8. Transformer architecture.
9. BERT and similar models.
10. Fine-tuning and pre-trained model application.
11. Generative models.
12. Adversarial training in NLP.
13. Deep Learning Frameworks for Text.
|
Activities
|
|
Fields of study
|
|
Guarantors and lecturers
|
|
Literature
|
-
Basic:
Jurafsky, Daniel; Martin, James H. Speech and language processing : an introduction to natural language processing, computational linguistics, and speech recognition. 2nd ed. Upper Saddle River : Pearson/Prentice Hall, 2009. ISBN 978-0-13-504196-3.
-
Recommended:
A Primer on Neural Network Models for Natural Language Processing
(Yoav Goldberg)
-
Recommended:
Ian Goodfellow, Yoshua Bengio, Aaron Courville. Deep Learning (Adaptive Computation and Machine Learning series). MIT press, 2016. ISBN 9780262035613.
-
Recommended:
François Chollet. Deep Learning with Python. 2017. ISBN 9781617294433.
-
Recommended:
Christopher D. Manning and Hinrich Schütze. Foundations of Statistical Natural Language Processing. MIT Press, Cambridge, MA, USA, 1999.
-
Recommended:
Aurélien Géron. Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. O'Reilly Media, 2017. ISBN 1491962291.
-
Recommended:
Jacob Eisenstein. Introduction to Natural Language Processing (Adaptive Computation and Machine Learning series). MIT Press, 2019. ISBN 0262042843.
-
Recommended:
Delip Rao, Brian McMahan. Natural Language Processing with PyTorch: Build Intelligent Language Applications Using Deep Learning. ISBN 1491978236.
-
On-line library catalogues
|
Time requirements
|
All forms of study
|
Activities
|
Time requirements for activity [h]
|
Practical training (number of hours)
|
26
|
Contact hours
|
26
|
Preparation for an examination (30-60)
|
40
|
Preparation for formative assessments (2-20)
|
10
|
Individual project (40)
|
60
|
Total
|
162
|
|
Prerequisites
|
Knowledge - students are expected to possess the following knowledge before the course commences to finish it successfully: |
having an overview of basic methods of probability and statistics |
having an overview of basic methodshaving an overview of basic methods of probability and statistics |
solving computer tasks at the level of Bachelor degree in Computer Science or a similar field |
Skills - students are expected to possess the following skills before the course commences to finish it successfully: |
decompose tasks into simpler units |
implement more advanced programs in an imperative programming language |
solve linear algebra problems |
Competences - students are expected to possess the following competences before the course commences to finish it successfully: |
N/A |
|
Learning outcomes
|
Knowledge - knowledge resulting from the course: |
be familiar with basic text summarization methods |
be familiar with evaluating the success of natural language processing methods |
be familiar with multilingual text processing |
describe the principles of natural language processing and text data retrieval |
Skills - skills resulting from the course: |
apply machine learning to natural language processing |
create algorithms for automatic evaluation of semantic similarity of words sentences and documents |
create algorithms for sentence parsing |
create machine learning algorithms |
create named entity recognition algorithms |
train language models |
Competences - competences resulting from the course: |
N/A |
N/A |
|
Assessment methods
|
Knowledge - knowledge achieved by taking this course are verified by the following means: |
Oral exam |
Test |
Skills - skills achieved by taking this course are verified by the following means: |
Seminar work |
Competences - competence achieved by taking this course are verified by the following means: |
Oral exam |
|
Teaching methods
|
Knowledge - the following training methods are used to achieve the required knowledge: |
Discussion |
E-learning |
Interactive lecture |
Lecture supplemented with a discussion |
One-to-One tutorial |
Practicum |
Self-study of literature |
Multimedia supported teaching |
Skills - the following training methods are used to achieve the required skills: |
Individual study |
Competences - the following training methods are used to achieve the required competences: |
Interactive lecture |
|
|
|
|