Course Title | ## Stochastic Processes | ||

Course Code | ## MH3512 | ||

Offered | Study Year 3, Semester 1 | ||

Course Coordinator | Ariel Neufeld (Asst Prof) | ariel.neufeld@ntu.edu.sg | +65 65921799 |

Pre-requisites | MH2500 | ||

AU | 4 | ||

Contact hours | Lectures: 39, Tutorials: 12 | ||

Approved for delivery from | AY 2019/20 semester 1 | ||

Last revised | 16 May 2019, 14:45 |

This course will enable you to analyze random systems and their long-run behavior by the computation of key quantities such as hitting probabilities and mean hitting times. On a more global scale, it aims at training undergraduates to design stochastic models with potential applications in various fields, such as biology, economics, finance, physics, analytics and data science.

Upon successfully completing this course, you should be able to:

- Compute hitting probabilities and mean hitting times for discrete and continuous-time Markov chains.
- Apply the technique of first step analysis to estimate key quantities in discrete and continuous-time random systems.
- Classify the state properties of Markovian random systems.
- Evaluate the long-run behavior of random systems via their limiting and stationary distributions.
- Construct stochastic models for common random systems appearing in real-life applications.

Introduction and motivations

Probability background

Gambling problems

Random walks

Discrete-time Markov chains

First step analysis (hitting probabilities)

First step analysis (mean hitting times)

Classification of states

Long-run behavior of Markov chains

Branching processes

Continuous-time Markov chains (definitions and properties)

Continuous-time Markov chains (examples and first step analysis)

Revisions and additional tutorial exercises

Component | Course ILOs tested | SPMS-MAS Graduate Attributes tested | Weighting | Team / Individual | Assessment Rubrics |
---|---|---|---|---|---|

Continuous Assessment | |||||

Tutorials | |||||

Assignment | 1, 2, 3, 4 | 1. a, b, c, d2. a, b, c4. a5. a | 25 | individual | See Appendix for rubric |

Mid-semester Quiz | |||||

Short Answer Questions | 1, 2, 3, 4, 5 | 1. a, b, c, d2. a, b, c4. a | 25 | individual | See Appendix for rubric |

Examination (2 hours) | |||||

Short Answer Questions | 1, 2, 3, 4, 5 | 1. a, b, c2. a, b, c5. a | 50 | individual | See Appendix for rubric |

Total | 100% |

These are the relevant SPMS-MAS Graduate Attributes.

## 1. Competence

a. Independently process and interpret mathematical theories and methodologies, and apply them to solve problems

b. Formulate mathematical statements precisely using rigorous mathematical language

c. Discover patterns by abstraction from examples

d. Use computer technology to solve problems, and to communicate mathematical ideas

## 2. Creativity

a. Critically assess the applicability of mathematical tools in the workplace

b. Build on the connection between subfields of mathematics to tackle new problems

c. Develop new applications of existing techniques

## 4. Civic-mindedness

a. Develop and communicate mathematical ideas and concepts relevant in everyday life for the benefits of society

## 5. Character

a. Act in socially responsible and ethical ways in line with the societal expectations of a mathematics professional, particularly in relation to analysis of data, computer security, numerical computations and algorithms

Midterm: Written and verbal feedback will be given from the lecturer and graders on your midterm test results, including through in-class discussion.

You will also receive formative feedback for all learning outcomes during weekly tutorial classes from Week 2-Week 13.

Final Examination, summative group feedback on the exam will be given following the conclusion of the module.

Lectures (39 hours) | Derivations and proofs: Peer instruction: Self-practice: |

Tutorials (12 hours) | Derivations and proofs: Peer instruction: Self-practice: |

N. Privault. Understanding Markov Chains - Examples and Applications. Second Edition, Springer Undergraduate Mathematics Series, 2018. ISBN 978-981-13-0659-4

(1) General

You are expected to complete all assigned class readings and activities, attend classes punctually and take all scheduled assignments and tests by due dates. You are expected to take responsibility to follow up with course notes and assignments, and to participate in tutorial discussions and activities.

(2) Absenteeism

Absence from class without a valid reason will affect your overall course grade. Valid reasons include falling sick supported by a medical certificate and participation in NTU’s approved activities supported by an excuse letter from the relevant bodies.

If you miss a lecture, you must inform the course instructor via email prior to the start of the class.

(3) Absence Due to Medical or Other Reasons

If you are sick and not able to attend a quiz or midterm, you have to submit the original Medical Certificate (or another relevant document) to the administration to obtain official leave. In this case, the missed assessment component will not be counted towards the final grade.

Good academic work depends on honesty and ethical behaviour. The quality of your work as a student relies on adhering to the principles of academic integrity and to the NTU Honour Code, a set of values shared by the whole university community. Truth, Trust and Justice are at the core of NTU’s shared values.

As a student, it is important that you recognize your responsibilities in understanding and applying the principles of academic integrity in all the work you do at NTU. Not knowing what is involved in maintaining academic integrity does not excuse academic dishonesty. You need to actively equip yourself with strategies to avoid all forms of academic dishonesty, including plagiarism, academic fraud, collusion and cheating. If you are uncertain of the definitions of any of these terms, you should go to the Academic Integrity website for more information. Consult your instructor(s) if you need any clarification about the requirements of academic integrity in the course.

Instructor | Office Location | Phone | |
---|---|---|---|

Ariel Neufeld (Asst Prof) | SPMS-MAS 05 02 | +65 65921799 | ariel.neufeld@ntu.edu.sg |

Week | Topic | Course ILO | Readings/ Activities |
---|---|---|---|

1 | Introduction and motivations | 1, 2, 3, 4, 5 | Lecture notes |

2 | Probability background | 1, 2, 3, 4, 5 | Lecture notes |

3 | Gambling problems | 1, 2, 5 | Lecture notes, assignments and simulations |

4 | Random walks | 1, 2, 5 | Lecture notes, assignments and simulations |

5 | Discrete-time Markov chains | 5 | Lecture notes, assignments and simulations |

6 | First step analysis (hitting probabilities) | 1, 2, 5 | Lecture notes, assignments and simulations |

7 | First step analysis (mean hitting times) | 1, 2, 5 | Lecture notes, assignments and simulations |

8 | Classification of states | 3 | Lecture notes and assignments |

9 | Long-run behavior of Markov chains | 4, 5 | Lecture notes, assignments and simulations |

10 | Branching processes | 1, 2, 3, 4, 5 | Lecture notes, assignments and simulations |

11 | Continuous-time Markov chains (definitions and properties) | 1, 2, 3, 4, 5 | Lecture notes, assignments and simulations |

12 | Continuous-time Markov chains (examples and first step analysis) | 1, 2, 3, 4, 5 | Lecture notes, assignments and simulations |

13 | Revisions and additional tutorial exercises | 1, 2, 3, 4, 5 | Lecture notes |