Discrete Stochastic Processes, Spring 2011 (M-I-T)

• 1. Introduction and Probability Review (M-I-T)
• 2. More Review; The Bernoulli Process (M-I-T)
• 3. Law of Large Numbers, Convergence (M-I-T)
• 4. Poisson (the Perfect Arrival Process) (M-I-T)
• 6. From Poisson to Markov (M-I-T)
• 7. Finite-state Markov Chains; The Matrix Approach (M-I-T)
• 8. Markov Eigenvalues and Eigenvectors (M-I-T)
• 9. Markov Rewards and Dynamic Programming (M-I-T)
• 10. Renewals and the Strong Law of Large Numbers (M-I-T)
• 11. Renewals: Strong Law and Rewards (M-I-T)
• 12. Renewal Rewards, Stopping Trials, and Wald's Inequality (M-I-T)
• 13. Little, M/G/1, Ensemble Averages (M-I-T)
• 14. Review (M-I-T)
• 15. The Last Renewal (M-I-T)
• 16. Renewals and Countable-state Markov (M-I-T)
• 17. Countable-state Markov Chains (M-I-T)
• 18. Countable-state Markov Chains and Processes (M-I-T)
• 19. Countable-state Markov Processes (M-I-T)
• 20. Markov Processes and Random Walks (M-I-T)
• 21. Hypothesis Testing and Random Walks (M-I-T)
• 22. Random Walks and Thresholds (M-I-T)
• 23. Martingales (Plain, Sub, and Super) (M-I-T)
• 24. Martingales: Stopping and Converging (M-I-T)
• 25. Putting It All Together (M-I-T)