Get Discrete-time Markov Chains: Two-time-scale Methods and PDF

By G. George Yin, Qing Zhang

ISBN-10: 038721948X

ISBN-13: 9780387219486

This booklet specializes in two-time-scale Markov chains in discrete time. Our motivation stems from present and rising functions in optimization and keep watch over of advanced platforms in production, instant communique, and ?nancial engineering. a lot of our e?ort during this booklet is dedicated to designing procedure versions coming up from quite a few functions, examining them through analytic and probabilistic recommendations, and constructing possible compu- tionalschemes. Ourmainconcernistoreducetheinherentsystemcompl- ity. even supposing all the purposes has its personal detailed features, them all are heavily comparable during the modeling of uncertainty because of bounce or switching random tactics. Oneofthesalientfeaturesofthisbookistheuseofmulti-timescalesin Markovprocessesandtheirapplications. Intuitively,notallpartsorcom- nents of a large-scale approach evolve on the comparable price. a few of them switch speedily and others range slowly. The di?erent charges of diversifications let us decrease complexity through decomposition and aggregation. it'd be perfect if lets divide a wide method into its smallest irreducible subsystems thoroughly separable from each other and deal with every one subsystem indep- dently. even though, this can be infeasible in fact as a result of numerous actual constraints and different issues. therefore, we need to care for occasions during which the structures are just approximately decomposable within the feel that there are susceptible hyperlinks one of the irreducible subsystems, which dictate the oc- sional regime alterations of the procedure. An e?ective solution to deal with such close to decomposability is time-scale separation. that's, we arrange the structures as though there have been time scales, quick vs. sluggish. xii Preface Followingthetime-scaleseparation,weusesingularperturbationmeth- ology to regard the underlying platforms.

Show description

Read Online or Download Discrete-time Markov Chains: Two-time-scale Methods and Applications PDF

Best mathematicsematical statistics books

A First Course in Statistics for Signal Analysis - download pdf or read online

This article serves as a superb creation to statistical data for sign research. remember that it emphasizes conception over numerical tools - and that it really is dense. If one isn't searching for long causes yet as a substitute desires to get to the purpose fast this publication should be for them.

Michael J. Campbell's Statistics at Square Two: Understanding Modern Statistical PDF

Up-to-date better half quantity to the ever renowned data at sq. One (SS1) data at sq. , moment version, is helping you review the numerous statistical tools in present use. Going past the fundamentals of SS1, it covers refined tools and highlights misunderstandings. effortless to learn, it comprises annotated computing device outputs and retains formulation to a minimal.

Extra info for Discrete-time Markov Chains: Two-time-scale Methods and Applications

Example text

M0 } and generator Q. Let f (·, ·) : Rr × M → Rr , and g(·, ·) : Rr × M → Rr×r be appropriate functions. 12) where w(·) is an r-dimensional standard Brownian motion, and f (·) and g(·) satisfy certain regularity conditions. Assume that the Markov chain α(·) and the Brownian motion w(·) are independent. Assume also that the 16 1. 12) has a unique solution in distribution for each initial condition. 12) can only be solved numerically, which makes appropriate discretization and numerical algorithms necessary.

S1m1 } ∪ · · · ∪ {sl0 1 , . . 6), the state space M is decomposable into the following form M = M1 ∪ · · · ∪ Ml0 ∪ M∗ = {s11 , . . , s1m1 } ∪ · · · ∪ {sl1 , . . , slml0 } ∪ {s∗1 , . . 8) with m0 = m1 + m2 + · · · + ml0 + m∗ . The subspaces Mi for i = 1, . . , l0 consist of recurrent states belonging to l0 different ergodic classes, and the subspace M∗ consists of transient states. 2 Asymptotic Expansions For future use, for an appropriate function f : R1×m0 → R1×m0 , define an operator Lε as Lε f (k) = f (k + 1) − f (k)Pkε .

Under suitable conditions and using asymptotic results of the two-time-scale Markov chains to be presented in this book, we can derive mean squares error bounds for the tracking error sequence θn . The mean squares bounds will assist us further to obtain a weak convergence result of a suitably scaled sequence. In addition, we will be able to find probabilistic bounds on P (|θn − θn | > η) for η > 0. 7. This example is concerned with a numerical scheme for the approximation of regime-switching diffusions.

Download PDF sample

Discrete-time Markov Chains: Two-time-scale Methods and Applications by G. George Yin, Qing Zhang

by John

Rated 4.55 of 5 – based on 35 votes