I've moved several times in my career. Usually these moves are inspired by external event or circumstance. I have moved for partners, for a spouse, to be nearer my parents. Sometimes for science: I spent two years in Australia working on marsupials. But I've learned a lot from each move. The most consistent things I've learned is: Damned if I am going to move again.
Many people don't move, for all sorts of reasons. Some settle down, have a family and are reluctant to move that family. Some feel they have reached an absorbing boundary: why would I want to leave Harvard/Caltech/BSD-R01 University? Some would like to leave, and don't have the chops to do so.
Here's the quote:
"Interestingly, koi, when put in a fish bowl, will only grow up to three inches. When this same fish is placed in a large tank, it will grow to about nine inches long. In a pond koi can reach lengths of eighteen inches. Amazingly, when placed in a lake, koi can grow to three feet long. The metaphor is obvious. You are limited by how you see the world." - Vince Poscente
Who the heck is Vince Poscente? He's a motivational speaker and "best selling author". Hmmm... Let's just use the quote at face value and not worry about Vince.
One might think the message is you are limited by the size of your pond. And that's probably true to some extent. Big universities have more resources (mostly) and can provide their faculty with either more opportunities (mostly) or fewer limits on their growth (sometimes). It's important to remember that size isn't everything. Or, as my one good friend sez: Size is nothing. And there are multiple ways of measuring excellence and good-fit. And, of course, sometimes its not in academics at all.
One of the things I have learned from moving a lot is that there is not a single scale and metric of what works for me, let alone for someone else. There are lots of ingredients of happiness, and size/quality/supurbity of place is only one of them.
As is true for most things: people are the same everywhere, and there are wonderful unique people to meet in each place. But that character with pony tail and torn tshirt & work boots who always interrupts the dept meetings to tell questionable jokes? Yup there's one of him or her in every department. plus or minus.
At everyplace I've been, there are a few lifers. People who've been in one place since their post-doc or residency. And no matter an external assessment of the place, they are so steeped in the traditions that they become blinded to nuance and the much of what goes on in the wider world.
If indeed, its "how you see the world" and not "how big is your pond", then the best thing that one can do, as best one can, is make sure to see more than one's own pond. I had a partner who did not believe in scientific meetings, and referred to them as "ego-fests". I think there is nothing like going to a meeting and sitting in sessions once or twice removed from your work. Or go to SfN and just wander a brand new poster aisle. Go to ExBio and go to other society posters. or sessions. Hit the plenary talks by not necessary the Big Dogs, but by the folks who are supposed to give Great Talk. You can push the boundaries of your own little pond.