Is the universe moving through infinite space time as it expands? What I mean by this, is imagine a person blowing up a balloon. As the balloon expands, the person blowing it up is also walking. At a much grander scale, apply this concept to the universe. As the universe expands, is it also moving like the balloon through infinite space time?
I think what you're asking is a very fundamental question that doesn't really have a definitive answer. I interpreted your question slightly differently that Sir Cumference, so I'll answer it differently.

For people just learning about the expansion of the Universe, they are often taught about the analogy of the expanding balloon. The concept here is that you have a deflated balloon that you draw two dots on. These dots may represent two galaxies in the Universe defined by the surface of the balloon. The important concept here is that the surface of the balloon represents the entire universe in this scenario. The balloon is then blown up, and you notice that because of the "expansion" of the universe, the dots (i.e., galaxies) are moving farther apart. This is shown in the diagram below.



This concept is usually used to teach an important fact about our universe, namely that there is no center. At no point on the balloon can you saw that you are the center of the expansion because the expansion is happening everywhere in equal magnitude.

What you seem to be doing though, is trying to take this analogy one step further. Even though the balloon universe (ballooniverse?) is defined on the surface of the balloon and is thus two dimensional, there still exists a three dimensional space that the 2D ballooniverse is expanding into. So you question, what does that say about our 3D universe? Are we expanding into some 4D space?

That, in my opinion, is a distillation of the fundamental concepts behind your question. Assuming I'm interpreting your question correctly, then I would have to say the answer is I don't know.

There is a somewhat (unsatisfying) tautological answer in that the Universe is, by definition, everything that exists. Therefore, there can't be a 3D Universe and also a 4D space into which our 3D Universe is expanding into, because that 4D space would necessarily be a part of our Universe, by definition.

A more reasoned approach to this answer is to look at it from a mathematical perspective. If you begin to learn higher math (and by higher I mean upper undergraduate/graduate level math) you'll learn about manifolds. Manifolds are basically mathematical objects that can define things like the surface of a balloon. What you'll find is that the definition for the surface of a balloon, using the concept of manifolds, does not require that the surface exist in 3D space, even if that manifold is expanding. It is perfectly possible to define a 2D manifold that has the same topology (fancy word for structure) as an expanding balloon, without "embedding" it in 3D space. This concept makes your question kind of moot. It allows for an expanding 3D universe, without requiring the existence of some 4D space to expand into. So ultimately, I'm claiming that it is possible for a 3D Universe to exist which both has no center and is expanding, just not into a 4th dimension.