We built out our own infrastructure in 2008, after using co-location from 1995-2007. It IS expensive, but we have never looked back. Yes, we have to maintain our own equipment. Yes, we have to go onsite to replace what breaks, but, as a whole, we have had far fewer issues with our own build-out. It is not for the timid, and certainly not for every hosting company, but if you are growing, take a look at that option. The worst part of the experience will come in 2 parts, shopping Data Center options (often highly aggregated through special sales channels); and making wise decisions on initial purchases. Start small, Good public switch - 10G, Good Private Switch(es) - 25G and smart choices on servers. Do not buy bargain servers of poor quality, or you will grow to hate the site of them. DO consider used equipment or recently "off-lease" equipment. You will find excellent deals, just be sure to replace hard-drives/SSD on servers, and check switches for known vulnerabilities.
Note: Name-brand servers such as Dell, HP, SuperMicro, Lenovo and ASUS fill data centers for a reason. They just keep working, year after year after year. Check for pricing on used servers. Many times they were cold or warm racked and saw no actual production usage.
If you ever do rent a rack, be sure to understand power commits, 120 v 208 (in the US), bandwidth. You also need to consider redundancy on power (you need A/B power in EVERY rack) and redundancy on network connectivity. Once you expand past 1 data center, you also need to deeply consider private connectivity between DC.
