This is an old revision of the document!
Table of Contents
Course Outline
The course outline is a guideline to topics that will be discussed in the course, and when they will be discussed:
Weeks 1 and 2
Math Background: probabilities; Bayes theorem; statistics; estimation; regression; hypothesis testing; entropy; mutual information; decision tree; optimization theory and convex optimization; matrix decomposition
Week 3
Data, Feature and Model: Feature Engineering, Feature Extraction (PCA, LDA, etc), Data Virtualization
Week 4
Machine Learning (ML) Basics: Learnability; Some basic ML Concepts; Bayesian decision theory
Week 5
Generative Models (I): model estimation: maximum likelihood, Bayesian learning, multivariate Gaussian, Multinomial, Markov Chain model, etc.
Week 6
Discriminative Models (I) : Linear discriminant functions; support vector machine (SVM); large margin classifiers; sparse kernel machines;
Week 7
Discriminative Models (II) : Neural Networks and Deep Learning
Week 8
Generative Models (II): EM algorithm; Finite Mixture models, e.g. Gaussian mixture model
Week 9
Generative Models (III):
Week 10
Hidden Markov Model (HMM): HMM vs. Markov chains; HMM concepts; Three algorithms: forward-backward; Viterbi decoding; Baum-Welch learning.
midterm presentation
Week 8
Automatic Speech Recognition (ASR) (I): Acoustic and Language Modeling: HMM for ASR; ASR as an example of pattern classification; Acoustic modeling: HMM learning (ML, MAP, DT); parameter tying (decision tree based state tying); n-gram models: smoothing, learning, perplexity, class-based.
Week 9
Automatic Speech Recognition (ASR) (II): Search - why search; Viterbi decoding in a large HMM; beam search; tree-based lexicon; dynamic decoding; static decoding; weighted finite state transducer (WFST)
Week 10
Spoken Language Processing (I): text categorization classify text documents: call/email routing, topic detection, etc. vector-based approach, Naïve Bayes classifier; Bayesian networks, etc. (2) HMM applications: Statistical Part-of-Speech (POS) tagging; Language understanding: hidden concept models.
Week 11
Spoken Language Processing (II): statistical machine translation IBM’s models for machine translation: lexicon model, alignment model, language model training process, generation & search
Week 12
student presentation