In the Network Settings pane of your System Preferences, you may have noticed that you can manually set the DNS servers your connection will use. There are a few reasons for doing this, namely speeding up the time it takes to look up any given website, but also to bypass some annoyances in your ISP's (or IT department's) default name server. Such annoyances could include domain blocking, censorship and other things you may or may not know are even happening. For the most part, though, you'd change your DNS settings to make sure you were using the fastest possible server from your current location. Read on to find out how!
Finding that fastest servers is really easy these days, thanks to a free utility called namebench. Grab a copy and load it up. It will automatically include whatever you have set as your current domain name servers, which you can leave alone for now. You can experiment with the settings, but I've found that my best results come from checking all three of the main checkboxes (global DNS, regional DNS and censorship checks), selecting my primary browser as the source, and using the automatic test selection mode. I run 220 queries, and usually do two runs. When it's finished, it will load up a web page of results (from your local machine). Then you just have to modify your DNS settings based on the results.
To modify your DNS servers, open System Preferences and go to "Network." The easiest thing to do here is just to go to your primary interface (usually Ethernet or Airport, depending on how you connect) and enter a comma-separated list of IP addresses in the DNS server field. If you want a nicer interface, click on the "Advanced" button and choose the DNS tab in the tab bar. There, you can use the plus and minus buttons to add and remove IP addresses from the list.
Generally, the top three IP addresses that namebench provides can just be inserted in one of these two ways and, depending on what your current servers were, you may notice a sizeable difference in the time it takes to initially locate web pages. Give it a shot! Note that your optimum settings will change over time, and especially from location to location. Setting up a few Network Locations for networks you use often will allow you to easily change DNS settings when you change location.
Addendum: As noted over at APC, using US-based DNS providers when outside of the US can actually be detrimental to your download speeds. If you're surfing outside of the US, turn off the "Use global DNS providers" checkbox for better results!
I found using OpenDNS to be the cause of my SLOW downloads on my Apple TV. I've since switched back to the ISP DNS and things are working better. I am in the US.