Since some weeks some internet providers start using encryption on their international links. My own provider backbone.is does this standard on all links by default so this is nothing new to me. However, it takes more than just checking a box "encrypted" to do so. And some big providers have not taken that into consideration.
On a normal ethernet, the maximum packet size is usually 1500 bytes. This is called the MTU (maximum transmissible unit). If you encapsulate the IP packet into a tunnel such as PPTP or L2TP as it is the case for VPN's then the MTU drops to slightly lower values. IP is made the way that this always should work. but there are scenarios where this isn't the case.
Lets assume you have a path which looks like this:
SENDER ----> ROUTER-A ----> Router B ----> Router-C ----> Router D ----> RECEIVER
If SENDER uses an MTU of 1500 and sends a packet to the server and all along the way, the MTU is 1500 or more, then everything works fine.
If SENDER uses an MTU of 1400 and sends a pacekt to the server, the client machine will make smaller than needed packets but all works. The RECEIVER will send replies back to the client in 1400 byte chunks because when the connection was opened the webserver was told the client can not accept anything bigger than 1400 (the tcp mss field).
Where it gets tricky is if both sides have a MTU of 1500 but somewhere in the middle there's a smaller one. Lets assume between B and C we now have an MTU of 1300. When the SENDER sends a packet of 1500 bytes and this packet hits router B, router B has two options to deal with it. He can either drop the packet or make 2 packets out of it and forward it to C. The second is called "fragmenting" and is not recommended as it produces inefficient use of the media and it also produces unnecessary CPU load. So most routers will drop the packet and send back a message to the caller telling it that the packet was dropped and the maximum size it could use. This is also known as MTU path discovery. SENDER would then retry with smaller packets based on this info message. This info message is a so called ICMP message, also used for PING.
Now here comes the catch: Firewalls
There are many firewall administrators out there in the field who have no clue and they block ICMP completely because they fear it being abused for something. In fact in 1995 there was a vulnerability in windows NT which allowed a server to crash by sending it a few well crafted ICMP messages (so called Ping-o-Death). however those days are long gone and ICMP has a purpose.
Think of this scenario:
You open a browser. You type www.networksolutions.com. Your request travels to the webserver. The webserver answers with a big huge picture which is larger than 1500 bytes. The answer travels the way back to your browser but gets dropped on the way because the MTU is slightly smaller. A ICMP message gets sent back to www.networksolutions.com but that ICMP message never hits the webserver. So the webserver continues to retry with MTU 1500 and the messages get dropped again and again. In other word,the webbrowser will never get to see anything.
This is what we have seen since Level3 has started encrypting its transatlantic link between Amsterdam and the USA. Same error with many other websites delivered from content delivery networks which also blocked ICMP. Another example was www.kickstarter.com.
The workaround is to adapt the tcp mss value on the router for messages going through so the initial request will instruct the other side to use smaller packets. But that was too much asked for level3