Very good question. And the previous reply does well to address real world differences. In the Traveller universe there are planets which are relatively inaccessible due to politics, planetary conditions, cultural differences, distance from nearest main, and a whole host of other reasons.
The short and RPG oriented answer is that it makes things more interesting to play. If you look at the TL stats for the Solomani Rim you might notice that there is a more even layer of TL and prosperity, as well as higher population levels.
This reflects a longer settled area with more even trade and technology distribution.
The Vland Sectors as well have a long settlement period with steady trade, even during parts of the Long Night.
To look at a real world history and analysis of how a frontier can become a central part of "civilization" read
Nature's Metropolis: Chicago and the Great West by William Cronon
To pull the editorial review from Amazon.com: "Cronon's history of 19th-century Chicago is in fact the history of the widespread effects of a single city on millions of square miles of ecological, cultural, and economic frontier. Cronon combines archival accuracy, ecological evaluation, and a sweeping understanding of the impact of railroads, stockyards, catalog companies, and patterns of property on the design of development of the entire inland United States to this date. Although focused on Chicago and the U.S., the general lessons it teaches are of global significance, and a rich source of metaphors for the ways in which colonization of physical space operates differently from, and similarly to, colonization of cyberspace. This is a compelling, wise, thorough--and thoroughly accessible--masterpiece of history writ large. Very Highest Recommendation."
I find it easy to substitute the word city for world and railroad for starship and you get an idea of how some well meaning people and entities can succeed wildly while some others fail miserably.
When you add the general Imperial attitude of "look out for yourself" as far as planet development goes, the varying levels of TL, starport and trade are understandable.
Compared to Hiver or Zhodani space, which seem to have much more interventionist governments/philosophies, the Imperium is very uneven in weatlh/tech/trade distribution.