Cisco IT designed AI-ready infrastructure with Cisco compute, best-in-class NVIDIA GPUs, and Cisco networking that helps AI mannequin coaching and inferencing throughout dozens of use instances for Cisco product and engineering groups. 

It’s no secret that the stress to implement AI throughout the enterprise presents challenges for IT groups. It challenges us to deploy new know-how sooner than ever earlier than and rethink how knowledge facilities are constructed to satisfy growing calls for throughout compute, networking, and storage. Whereas the tempo of innovation and enterprise development is exhilarating, it may well additionally really feel daunting.  

How do you rapidly construct the info middle infrastructure wanted to energy AI workloads and sustain with vital enterprise wants? That is precisely what our staff, Cisco IT, was going through. 

The ask from the enterprise

We have been approached by a product staff that wanted a technique to run AI workloads which could be used to develop and check new AI capabilities for Cisco merchandise. It would finally help mannequin coaching and inferencing for a number of groups and dozens of use instances throughout the enterprise. And they wanted it performed rapidly. want for the product groups to get improvements to our clients as rapidly as doable, we needed to ship the new surroundings in simply three months.  

The know-how necessities

We started by mapping out the necessities for the brand new AI infrastructure. A non-blocking, lossless community was important with the AI compute cloth to make sure dependable, predictable, and high-performance knowledge transmission throughout the AI cluster. Ethernet was the first-class alternative. Different necessities included: 

• Clever buffering, low latency: Like several good knowledge middle, these are important for sustaining clean knowledge stream and minimizing delays, in addition to enhancing the responsiveness of the AI cloth. 
• Dynamic congestion avoidance for varied workloads: AI workloads can fluctuate considerably of their calls for on community and compute sources. Dynamic congestion avoidance would make sure that sources have been allotted effectively, stop efficiency degradation throughout peak utilization, keep constant service ranges, and forestall bottlenecks that would disrupt operations. 
• Devoted front-end and back-end networks, non-blocking cloth: With a purpose to construct scalable infrastructure, a non-blocking cloth would guarantee ample bandwidth for knowledge to stream freely, in addition to allow a high-speed knowledge switch — which is essential for dealing with massive knowledge volumes typical with AI purposes. By segregating our front-end and back-end networks, we might improve safety, efficiency, and reliability. 
• Automation for Day 0 to Day 2 operations: From the day we deployed, configured, and tackled ongoing administration, we needed to scale back any guide intervention to maintain processes fast and reduce human error. 
• Telemetry and visibility: Collectively, these capabilities would offer insights into system efficiency and well being, which might permit for proactive administration and troubleshooting. 

The plan – with a couple of challenges to beat

With the necessities in place, we started determining the place the cluster may very well be constructed. The present knowledge middle services weren’t designed to help AI workloads. We knew that constructing from scratch with a full knowledge middle refresh would take 18-24 months – which was not an possibility. We wanted to ship an operational AI infrastructure in a matter of weeks, so we leveraged an present facility with minor modifications to cabling and system distribution to accommodate. 

Our subsequent issues have been across the knowledge getting used to coach fashions. Since a few of that knowledge wouldn’t be saved domestically in the identical facility as our AI infrastructure, we determined to duplicate knowledge from different knowledge facilities into our AI infrastructure storage techniques to keep away from efficiency points associated to community latency. Our community staff had to make sure ample community capability to deal with this knowledge replication into the AI infrastructure.

Now, attending to the precise infrastructure. We designed the guts of the AI infrastructure with Cisco compute, best-in-class GPUs from NVIDIA, and Cisco networking. On the networking facet, we constructed a front-end ethernet community and back-end lossless ethernet community. With this mannequin, we have been assured that we might rapidly deploy superior AI capabilities in any surroundings and proceed so as to add them as we introduced extra services on-line.

Merchandise: 

Supporting a rising surroundings

After making the preliminary infrastructure out there, the enterprise added extra use instances every week and we added extra AI clusters to help them. We wanted a technique to make all of it simpler to handle, together with managing the swap configurations and monitoring for packet loss. We used Cisco Nexus Dashboard, which dramatically streamlined operations and ensured we might develop and scale for the longer term. We have been already utilizing it in different components of our knowledge middle operations, so it was straightforward to increase it to our AI infrastructure and didn’t require the staff to be taught an extra device. 

The outcomes

Our staff was in a position to transfer quick and overcome a number of hurdles in designing the answer. We have been in a position to design and deploy the backend of the AI cloth in beneath three hours and deploy the complete AI cluster and materials in 3 months, which was 80% sooner than the choice rebuild.  

Right now, the surroundings helps greater than 25 use instances throughout the enterprise, with extra added every week. This consists of:

• Webex Audio: Enhancing codec growth for noise cancellation and decrease bandwidth knowledge prediction
• Webex Video: Mannequin coaching for background alternative, gesture recognition, and face landmarks
• Customized LLM coaching for cybersecurity merchandise and capabilities

Not solely have been we in a position to help the wants of the enterprise in the present day, however we’re designing how our knowledge facilities must evolve for the longer term. We’re actively constructing out extra clusters and can share extra particulars on our journey in future blogs. The modularity and suppleness of Cisco’s networking, compute, and safety offers us confidence that we are able to hold scaling with the enterprise. 

Further sources:

Share: