Elements Of Stochastic Processes
STAT/MA 532, Spring 2013

Lecture: MWF, 8:30 AM -- 9:20 AM, in UNIV 003
(STAT 53200-001; Banner CRN 43058)
(MA 53200-001; Banner CRN 22175)

Professor: Mark Daniel Ward
Email: mdw@purdue.edu
Office: MATH 540
Phone: 765-496-9563

Office hours: Dr. Ward is always happy to meet with students.
Dr. Ward is available for walk-in or scheduled appointments anytime, throughout the week.
He is also always guaranteed to be available MWF, 7:30 AM -- 8:20 AM, in MATH 540.

Grader: Kelvin Ma
Email: kkma@purdue.edu

Course policy: click here

Midterm exam dates: Friday, February 8; Friday, March 8; Friday, April 12

Final exam date/time/location: to be announced

Plan to be present at all exams. Plan to be present for class every day.

Homework: Homework solutions will be collected in class on the due date. Homework solutions will be distributed in class.
Outline of Topics
Week 1: Mon, Jan 7
Introduction to the course,
discussion of policy, dates of exams,
other administrative issues.
Introduction to Markov chains.
Problem Set 1 assigned,
due on Wednesday, Jan 23, at 8:30 AM
Wed, Jan 9
Introduction to stationary distributions,
Gambler's ruin, hitting times,
recurrent and transient states
Fri, Jan 11
discussion of Theorems in
Ch 1 of Hoel, Port, Stone
Week 2: Mon, Jan 14
proof of Theorems in
Ch 1 of Hoel, Port, Stone
Wed, Jan 16
Example 1.14 from Durrett
extension of ideas:
recurrent vs transient
expected values, long-run averages;
start of discussion of birth-death chains
Fri, Jan 18
more discussion of birth-death chains
Week 3: Mon, Jan 21 (no lecture)
Martin Luther King Jr. Day
Wed, Jan 23
Problem Set 1 due today at 8:30 AM
Problem Set 2 assigned,
due on Wednesday, Feb 6, at 8:30 AM
Fri, Jan 25
branching processes:
introduction and examples
Week 4: Mon, Jan 28
when are branching processes
transient or recurrent? (with proofs)
Wed, Jan 30
queuing processes:
introduction and examples
Fri, Feb 1
when are queuing processes
transient or recurrent? (with proofs)
Week 5: Mon, Feb 4
overview of steady states
and connections to renewal theory
Wed, Feb 6
discussion about expected proportion
of the time a chain is in a state
Problem Set 2 due today at 8:30 AM
Fri, Feb 8
Midterm Exam 1
Week 6: Mon, Feb 11
proofs about expected proportion
of the time a chain is in a state
Problem Set 3 assigned,
due on Monday, Feb 25, at 8:30 AM
Wed, Feb 13
stationary distributions
Fri, Feb 15
stationary distributions
Week 7: Mon, Feb 18
examples about expected
return time in a Markov chain
(examples of positive vs null recurrence)
Wed, Feb 20
in-depth discussion of expected
return time in queuing processes
Fri, Feb 22
introduction to
Poisson Processes
Week 8: Mon, Feb 25
more about equivalent definitions
of Poisson processes;
comparison to order statistics
of uniformly distributed random variables
Problem Set 3 due today at 8:30 AM
Problem Set 4 assigned,
due on Wed, Mar 6, at 8:30 AM
Wed, Feb 27
interarrival times
in a Poisson process
Fri, Mar 1
conditional distribution of
arrival times in a Poisson process
Week 9: Mon, Mar 4
discussion of M/G/1 busy period
including 3 lemmas needed for the proof
Wed, Mar 6
Problem Set 4 due today at 8:30 AM
end of the proof for M/G/1 busy period
Introduction to nonhomogeneous
Poisson processes
Fri, Mar 8
Midterm Exam 2
Spring Break: Mon, Mar 11 (no lecture)
Spring Break
Mon, Mar 13 (no lecture)
Spring Break
Mon, Mar 15 (no lecture)
Spring Break
Week 10: Mon, Mar 18
Problem Set 5 assigned,
due on Mon, Apr 1, at 8:30 AM
Introduction to renewal processes
Wed, Mar 20
Elementary Renewal Theorem
Fri, Mar 22
Blackwell's Theorem
and the Key Renewal Theorem
Week 11: Mon, Mar 25
Examples of the Key Renewal Theorem
Wed, Mar 27
alternating renewal processes;
limiting mean excess
Fri, Mar 29
age-dependent branching processes
Week 12: Mon, Apr 1
Problem Set 5 due today at 8:30 AM
Problem Set 6 assigned,
due on Wednesday, Apr 10, at 8:30 AM
continuous time birth and death processes
Wed, Apr 3
Kolmogorov's forward and backward equations
for continuous time Markov chains
Fri, Apr 5
Q-matrices for continuous time Markov chains
Week 13: Mon, Apr 8
introduction to martingales
Wed, Apr 10
Problem Set 6 due today at 8:30 AM
stopping times
Fri, Apr 12
Midterm Exam 3
Week 14: Mon, Apr 15
Problem Set 6 due today at 8:30 AM
Problem Set 7 assigned,
due on Friday, Apr 15, at 8:30 AM
examples of martingales
Wed, Apr 17
introduction to Brownian Motion
and Gaussian processes
Fri, Apr 19
more about Brownian Motion;
expected values, covariances;
Brownian bridges
Week 15: Mon, Apr 22
4 variations on Brownian motion
Wed, Apr 24
Brownian motion with drift
Fri, Apr 26
Problem Set 7 due today at 8:30 AM
review for Final Exam
Final exam date/time/location: Tuesday, April 30, 7 PM to 9 PM, ME 1009