COURSE OUTLINE: MH3512

Course Title

Stochastic Processes

Course Code

MH3512

Offered Study Year 3, Semester 1
Course Coordinator Ariel Neufeld (Asst Prof) ariel.neufeld@ntu.edu.sg +65 65921799
Pre-requisites MH2500
AU 4
Contact hours Lectures: 39, Tutorials: 12
Approved for delivery from AY 2019/20 semester 1
Last revised 16 May 2019, 14:45

Course Aims

This course will enable you to analyze random systems and their long-run behavior by the computation of key quantities such as hitting probabilities and mean hitting times. On a more global scale, it aims at training undergraduates to design stochastic models with potential applications in various fields, such as biology, economics, finance, physics, analytics and data science.

Intended Learning Outcomes

Upon successfully completing this course, you should be able to:

  1. Compute hitting probabilities and mean hitting times for discrete and continuous-time Markov chains.
  2. Apply the technique of first step analysis to estimate key quantities in discrete and continuous-time random systems.
  3. Classify the state properties of Markovian random systems.
  4. Evaluate the long-run behavior of random systems via their limiting and stationary distributions.
  5. Construct stochastic models for common random systems appearing in real-life applications.

Course Content

Introduction and motivations

Probability background

Gambling problems

Random walks

Discrete-time Markov chains

First step analysis (hitting probabilities)

First step analysis (mean hitting times)

Classification of states

Long-run behavior of Markov chains

Branching processes

Continuous-time Markov chains (definitions and properties)

Continuous-time Markov chains (examples and first step analysis)

Revisions and additional tutorial exercises

Assessment

Component Course ILOs tested SPMS-MAS Graduate Attributes tested Weighting Team / Individual Assessment Rubrics
Continuous Assessment
Tutorials
Assignment 1, 2, 3, 4 1. a, b, c, d
2. a, b, c
4. a
5. a
25 individual See Appendix for rubric
Mid-semester Quiz
Short Answer Questions 1, 2, 3, 4, 5 1. a, b, c, d
2. a, b, c
4. a
25 individual See Appendix for rubric
Examination (2 hours)
Short Answer Questions 1, 2, 3, 4, 5 1. a, b, c
2. a, b, c
5. a
50 individual See Appendix for rubric
Total 100%

These are the relevant SPMS-MAS Graduate Attributes.

1. Competence

a. Independently process and interpret mathematical theories and methodologies, and apply them to solve problems

b. Formulate mathematical statements precisely using rigorous mathematical language

c. Discover patterns by abstraction from examples

d. Use computer technology to solve problems, and to communicate mathematical ideas

2. Creativity

a. Critically assess the applicability of mathematical tools in the workplace

b. Build on the connection between subfields of mathematics to tackle new problems

c. Develop new applications of existing techniques

4. Civic-mindedness

a. Develop and communicate mathematical ideas and concepts relevant in everyday life for the benefits of society

5. Character

a. Act in socially responsible and ethical ways in line with the societal expectations of a mathematics professional, particularly in relation to analysis of data, computer security, numerical computations and algorithms

Formative Feedback

Midterm: Written and verbal feedback will be given from the lecturer and graders on your midterm test results, including through in-class discussion.

You will also receive formative feedback for all learning outcomes during weekly tutorial classes from Week 2-Week 13.

Final Examination, summative group feedback on the exam will be given following the conclusion of the module.

Learning and Teaching Approach

Lectures
(39 hours)

Derivations and proofs:
Helps you improve your problem solving skills.

Peer instruction:
Develops communication and presentation skills and deepens understanding.

Self-practice:
Self-practice through solved exercises helps to build fundamental techniques and to develop your individual learning abilities.

Tutorials
(12 hours)

Derivations and proofs:
Helps you improve your problem solving skills.

Peer instruction:
Develops communication and presentation skills and deepens understanding.

Self-practice:
Self-practice through solved exercises helps to build fundamental techniques and to develop your individual learning abilities.

Reading and References

N. Privault. Understanding Markov Chains - Examples and Applications. Second Edition, Springer Undergraduate Mathematics Series, 2018. ISBN 978-981-13-0659-4

Course Policies and Student Responsibilities

(1) General

You are expected to complete all assigned class readings and activities, attend classes punctually and take all scheduled assignments and tests by due dates. You are expected to take responsibility to follow up with course notes and assignments, and to participate in tutorial discussions and activities.

(2) Absenteeism

Absence from class without a valid reason will affect your overall course grade. Valid reasons include falling sick supported by a medical certificate and participation in NTU’s approved activities supported by an excuse letter from the relevant bodies.

If you miss a lecture, you must inform the course instructor via email prior to the start of the class.

(3) Absence Due to Medical or Other Reasons

If you are sick and not able to attend a quiz or midterm, you have to submit the original Medical Certificate (or another relevant document) to the administration to obtain official leave. In this case, the missed assessment component will not be counted towards the final grade.

Academic Integrity

Good academic work depends on honesty and ethical behaviour. The quality of your work as a student relies on adhering to the principles of academic integrity and to the NTU Honour Code, a set of values shared by the whole university community. Truth, Trust and Justice are at the core of NTU’s shared values.

As a student, it is important that you recognize your responsibilities in understanding and applying the principles of academic integrity in all the work you do at NTU. Not knowing what is involved in maintaining academic integrity does not excuse academic dishonesty. You need to actively equip yourself with strategies to avoid all forms of academic dishonesty, including plagiarism, academic fraud, collusion and cheating. If you are uncertain of the definitions of any of these terms, you should go to the Academic Integrity website for more information. Consult your instructor(s) if you need any clarification about the requirements of academic integrity in the course.

Course Instructors

Instructor Office Location Phone Email
Ariel Neufeld (Asst Prof) SPMS-MAS 05 02 +65 65921799 ariel.neufeld@ntu.edu.sg

Planned Weekly Schedule

Week Topic Course ILO Readings/ Activities
1

Introduction and motivations

1, 2, 3, 4, 5

Lecture notes

2

Probability background

1, 2, 3, 4, 5

Lecture notes

3

Gambling problems

1, 2, 5

Lecture notes, assignments and simulations

4

Random walks

1, 2, 5

Lecture notes, assignments and simulations

5

Discrete-time Markov chains

5

Lecture notes, assignments and simulations

6

First step analysis (hitting probabilities)

1, 2, 5

Lecture notes, assignments and simulations

7

First step analysis (mean hitting times)

1, 2, 5

Lecture notes, assignments and simulations

8

Classification of states

3

Lecture notes and assignments

9

Long-run behavior of Markov chains

4, 5

Lecture notes, assignments and simulations

10

Branching processes

1, 2, 3, 4, 5

Lecture notes, assignments and simulations

11

Continuous-time Markov chains (definitions and properties)

1, 2, 3, 4, 5

Lecture notes, assignments and simulations

12

Continuous-time Markov chains (examples and first step analysis)

1, 2, 3, 4, 5

Lecture notes, assignments and simulations

13

Revisions and additional tutorial exercises

1, 2, 3, 4, 5

Lecture notes

Appendix 1: Assessment Rubrics

Rubric for Tutorials: Assignment (25%)

Point-based marking (not rubrics based)

Rubric for Mid-semester Quiz: Short Answer Questions (25%)

Point-based marking (not rubrics based)

Rubric for Examination: Short Answer Questions (50%)

Point-based marking (not rubrics based)