Escape a viewport

I am stuck in a viewport.  In paper/layout space, was zoomed in beyond the viewports borders, and now zooming only applies to the viewport that I'm already in.  How do I get out of the viewport, back to layout space?