July, 2007
"First of all I would like to tell you that my position with the Idaho Industrial Commission does not normally deal with this type of network problem. We are currently without a Network Analyst who would usually be the point of contact.
I would like to compliment your staff on the help we received with the outage of our field offices. Matt, Alex and Tom were extremely helpful in getting the situation resolved. Matt kept track of the problem until he left work and then turned it over to Alex. Alex went above and beyond what you would expect someone to do in trying to get the situation resolved. He kept us up to date on what was happening and continued after his normal work hours. Although I personally did not speak with Tom, as I had turned the situation over to IT Manager by the time he came on the job; from the emails I received from my Manager this morning, Tom also deserves to be acknowledged for his efforts on behalf of our agency.
What a great team - it was a pleasure working with your staff; good technicians are rare in the majority of the places I normally deal with. Thank you very much for your help and support."
Elena Knighten Information Systems Technician Sr Idaho Industrial Commission
"On Friday the tenth of June at approximately 4:30 PM CST pagers started to buzz across Vanderbilt University Medical Centers Network Services department. At that moment it appeared we had lost three different networking components between our two data centers. Upon further investigation it was discovered that we had lost an entire fiber route. At 5PM a string of blasting caps and wire were discovered on the street above our primary fiber route; a very pronounced mound in the center of the street served as the smoking gun pointing to the root cause of the failure.
We immediately called AFS local support, Mark Stacey, to advise him of our situation. Mark and the AFS organization immediately moved into action, within less than two hours men and materials were stationed over the bulging street. A construction crew, blasting the surrounding bedrock in preparation for a water and sewer upgrades had not only blasted immediately on top of the AFS/VUMC fiber path, they had bored through the cable in at least two places.
The AFS crew worked through the evening until the work site was shut down by the metro bomb squad for safety reasons. The bomb squad had been called in when suspect blasting caps were found on the jobsite; they surveyed the site and released it at 7AM the following morning. Work resumed on Saturday morning as huge sections of the street were excavated to locate and repair the damaged fiber. Ultimately the entire fiber route including the aerial portions of the run were replaced as recovery efforts continued through the evening hours of Saturday.
At 3AM Sunday morning service restoration was complete and we began the process of re-routing network traffic back onto the repaired pathway. This was an amazing and dramatic event and AFS performed exemplary in their efforts. Two years earlier untold hours were spent in negotiation and planning with AFS representatives as we prepared for the construction of our high availability data center scheduled to go live that year. Although it could not be imagined at that time, this event was exactly the type of situation that had been discussed and plans were included for escalation protocols as a part of our agreements.
Our data centers operate 7x24x365 employing a fully redundant DWDM and Gigabit Ethernet network configurations in support of Mainframe, Mid-range, and SAN services. All of this is transported over dark fibers provided by AFS on their metro fiber ring. This event was a true test of the escalation processes and AFS’ ability to meet our ongoing and business continuity needs. I am grateful and appreciative of all of their combined efforts during that weekend and for the continued service provided daily.
As a result of this event we have recently expanded our service agreement with AFS to include monitoring of the ‘last mile’ of fiber between the AFS and VUMC demarcation points. This was the point of failure for the blasting event and it highlighted a weakness in our monitoring plan that warranted a solution."
David L. Cochran
Manager, Health Systems Network
Vanderbilt University Medical Center
"I wanted to take this opportunity to update you on our relationship with AFS. WOW! The dark fiber ring project was a complete success, including hitting our tight timeline, and helping out when we had facilities based questions in some of our buildings. I want to thank AFS for helping to make sure we hit our marks. The network is running great. AFS is without a doubt “metro experts”.
Regarding the burstable lit services on your IP network, once again WOW! We originally intended to use it only as a backup in the case of a failover from one of our primary providers but as it turned out they missed their deadline numerous times on the delivery of the upgrade we ordered. So when our customers launched several large web initiatives over the summer, it was AFS to the rescue. The project was a complete success, including hitting our tight timeline.
Also, any time we experienced difficulties with our network hardware AFS was on the phone letting me know about the issue at the same time our pagers were going off internally. What great service!"
Robin Greenhagen
President
GSI
"I have to say, it's been a pleasure working with American Fiber for this move. You guys have been on the ball every step of the way. Now if only Bellsouth worked like that… Thanks for everything!"
Jonathan Black
Alacrity Services
"The T1 move is complete. Please pass on my thanks for a job well done. Our office was out for all of 5 seconds."
Walt Henley
Chief Operating Officer Camvera Networks
Want to hear more about what are customers are saying?
Request an invitation to our full Reference & Testimonial library.