How Deep Are Deep Gaussian Processes?

Recent research has shown the potential utility of probability distributions designed through hierarchical constructions which are conditionally Gaussian. This body of work is placed in a common framework and, through recursion, several classes of deep Gaussian processes are defined. The resulting samples have a Markovian structure with respect to the depth parameter and the effective depth of the process is interpreted in terms of the ergodicity, or non-ergodicity, of the resulting Markov chain.