• Open borders create a 'race to the bottom' in terms of social benefits and a social safety net. Imagine that borders are completely open and it's near costless to immigrate from one country to the other. Then the country with the best social services and benefits, (especially health benefits and unemployment benefits) will draw the World's poorest people, creating a strong Adverse Selection effect. That country will quickly lose its ability to fund those services and they will be significantly downscaled. This will continue until all developed countries will offer relatively low levels (compared to today) of benefits. Obviously in reality there is friction on immigration so this process wouldn't be immediate, but the dynamics described will still be a powerful force acting against meaningful social benefits.
• Open borders severely limits the ability of a country to chose its cultural identity. If you're strong believer in multiculturalism this may not seem like a drawback, but if you do believe some cultures are better than others, then this is a huge minus for open borders.
• Open borders cause wars and catastrophes to create global ripple effects that are hard to control. You can already see some of this with the current refugee crisis in Syria. While in the past if a refugee crisis wasn't at a World War scale it would typically mostly affect the countries directly involved in conflict and their neighbors. Open borders will cause a population affected by war or catastrophe to be willing to immigrate earlier and further, since the cost of migrating is lower and the benefits are higher. This makes prospering countries more vulnerable (in terms of the strength of the population they need to support).
• Open borders can undermine the rule of law in places where it is well established. If the population of a city can jump dramatically overnight due to truly open borders, then there's very little guaranteeing that the rule of law can be maintained, even in places that are essentially crime free. The rule of law isn't maintained just through policing and a legal framework, but also through a shared cultural understanding of expected norms and day to day behavior. Sudden changes in the makeup of a population can completely undermine this cultural aspect.

Now that you've read those points, think of it this way -

No nation should. No nation does, actually.

A nation is defined by four necessary things: a people, a land with borders, a law within that land, and a ruler -- lawgiver or administrator. Lose one element and you cease to be a nation. Weaken one, you weaken the whole.

Strong nations are built off of these four strong pillars. Weaken one, and you weaken the nation. This much should be obvious.

But there are those who do not like borders, are not interested in laws unless they are the ones in control, and are not supportive of the ruler unless he is one of theirs. Their's is a partisan USA, and they are only proud of it when they run things.

If you want to decline as a nation, stop enforcing the law equally. That is what some of the South American countries did, and they have become a cautionary tale. Yet even they still enforce borders because it's only when we do that we know who the laws and taxes apply to.

From a Libertarian Standpoint - It comes down to the fundamental question of how a libertarian society is going to address people who do not agree with/follow libertarian ideals. If everyone in the entire world were to abide completely by the NAP, there would be no need for countries or borders or governments (anyone singing John Lennon, right now?). But the fact of the matter is that there are people who will commit aggressive acts against others, and libertarians need a reasonable way to address this.

The first requirement is to have a country and a government, though with severe restrictions on what the government does. A country is defined by borders - people on one side of the line are countrymen and their guests, people on the other side are either not welcome or don’t have an interest in crossing the line.

What principles should be followed to distinguish the welcome guest from the persona non grata?

• Those with a history of aggression/violence against others (regardless of where it occurred) should be heavily scrutinized before being permitted into a libertarian state.
• Those who are known to frequently/positively associate with individuals/groups that act in aggression against the libertarian state should also be regarded with a wary eye.
• Those who are admitted to a libertarian state and subsequently commit acts of aggression/violence should be subject to the same punishments as a citizen, with the added possibility of expulsion from the country with no opportunity to return.

The focus of these concepts is identifying the problematic individual, because libertarianism at its core is about the rights of individuals. Following the same logic, certain generalities should not enter into the immigration calculus: race, religion, nationality, gender, etc.