Maybe your ISP has a connection to the internet backbone. This is shared by all the customers of your ISP. Another possibility is that the server computer that you are communicating or its ISP may be overloaded and limit your download speeds