COURSE OUTLINE: MH3512

Intended Learning Outcomes

Upon successfully completing this course, you should be able to:

1. Compute hitting probabilities and mean hitting times for discrete and continuous-time Markov chains.
2. Apply the technique of first step analysis to estimate key quantities in discrete and continuous-time random systems.
3. Classify the state properties of Markovian random systems.
4. Evaluate the long-run behavior of random systems via their limiting and stationary distributions.
5. Construct stochastic models for common random systems appearing in real-life applications.

Course Content

Introduction and motivations

Probability background

Gambling problems

Random walks

Discrete-time Markov chains

First step analysis (hitting probabilities)

First step analysis (mean hitting times)

Classification of states

Long-run behavior of Markov chains

Branching processes

Continuous-time Markov chains (definitions and properties)

Continuous-time Markov chains (examples and first step analysis)

Revisions and additional tutorial exercises

Assessment

Formative Feedback

Midterm: Written and verbal feedback will be given from the lecturer and graders on your midterm test results, including through in-class discussion.

You will also receive formative feedback for all learning outcomes during weekly tutorial classes from Week 2-Week 13.

Final Examination, summative group feedback on the exam will be given following the conclusion of the module.

Learning and Teaching Approach

Reading and References

N. Privault. Understanding Markov Chains - Examples and Applications. Second Edition, Springer Undergraduate Mathematics Series, 2018. ISBN 978-981-13-0659-4

Course Policies and Student Responsibilities

(1) General

You are expected to complete all assigned class readings and activities, attend classes punctually and take all scheduled assignments and tests by due dates. You are expected to take responsibility to follow up with course notes and assignments, and to participate in tutorial discussions and activities.

(2) Absenteeism

Absence from class without a valid reason will affect your overall course grade. Valid reasons include falling sick supported by a medical certificate and participation in NTU’s approved activities supported by an excuse letter from the relevant bodies.

If you miss a lecture, you must inform the course instructor via email prior to the start of the class.

(3) Absence Due to Medical or Other Reasons

If you are sick and not able to attend a quiz or midterm, you have to submit the original Medical Certificate (or another relevant document) to the administration to obtain official leave. In this case, the missed assessment component will not be counted towards the final grade.

Academic Integrity

Good academic work depends on honesty and ethical behaviour. The quality of your work as a student relies on adhering to the principles of academic integrity and to the NTU Honour Code, a set of values shared by the whole university community. Truth, Trust and Justice are at the core of NTU’s shared values.

As a student, it is important that you recognize your responsibilities in understanding and applying the principles of academic integrity in all the work you do at NTU. Not knowing what is involved in maintaining academic integrity does not excuse academic dishonesty. You need to actively equip yourself with strategies to avoid all forms of academic dishonesty, including plagiarism, academic fraud, collusion and cheating. If you are uncertain of the definitions of any of these terms, you should go to the Academic Integrity website for more information. Consult your instructor(s) if you need any clarification about the requirements of academic integrity in the course.

Course Instructors

Planned Weekly Schedule