What is high redshift? I'm curious about the difference between low redshift and high redshift universe. Is there any defined limit of redshift beyond which we call things high redshifted?
There definitely is no definition of what is low and what is high redshift — it is always seen in relation to something. An astronomer working in, say, the interstellar medium of the Milky Way ($z\sim0$) might talk about the ISM of a $z\sim2$ galaxy as being high redshift, whereas another astronomer working the reionization of the Universe (which took place around $z\sim10$) would consider $z\sim2$ to be low redshift. A cosmologist working in the CMB or inflation, however, would probably call $z\sim10$ low redshift. It all depends on the context. 

Nevertheless I think that most people would consider $z\lesssim0.1$ to be low redshift.