The inmates do not necessarily feel trapped, because their silos are often so big that they can spend their entire careers in one without feeling any constraint to do "their thing." Take IBM for example. One can spend their entire career in sales and never deal with any other part of the firm and do just fine. Within a sales division, they could also spend years within one geographic region, and again do just fine. A bigger problem is that there is so much work to be done within a silo that even if one wanted to poke their nose outside their silo, there might not be enough hours in the day to do that. So size of silos is something we should recognize as a factor in "silo centric" behavior.
The reason in business, at least, that silos get a bad rap is because collaboration across silos holds out the promise of optimization of costs and mission, innovation, better performance, lower operating costs, etc.--an assumption that can be both true and also absolutely false, depending on circumstances.
Then there is culture. Take a university, for example. You can have a history department occupy the fourth floor of a building, and the economics department on the third, and sociology on the second, and have a situation where (a) nobody knows each other across those three departments, (b) that is OK with them and (c) no collaboration in teaching cross-disciplinary classes, collaborations in research and publishing. That might even apply to a small college, such as the one I attended, which at the time had only about 875 students and a tiny faculty to match.
This is a fascinating subject you raise. Having spent 38 years at IBM at various levels or management and in numerous jobs (including doing strategy and work and managerial optimization consulting) I can assure you the questions you raise are profoundly important to literally millions of managers around the world, and the subject of extensive discussion in the business literature, although often without using the word "silo," but clearly in the minds of the authors.