Image Credits:Niv-AI
Niv-AI Emerges from Stealth to Enhance GPU Power Efficiency
Electricity plays a crucial role in the realm of artificial intelligence, yet the rapid evolution of processing techniques is outpacing the capacity of data center operators to effectively manage their relationship with the power grid. Consequently, these operators are often compelled to throttle their GPU usage by as much as 30%.
“There is so much power squandered in these AI factories,” stated Nvidia CEO Jensen Huang at the company’s annual GTC customer conference. “Every unused watt is revenue lost,” he emphasized during his keynote address.
In a bid to tackle this pressing issue, the Tel Aviv-based startup Niv-AI has officially emerged from stealth mode, securing $12 million in seed funding. The company’s mission is to enhance GPU power efficiency through the precise measurement of GPU power consumption using innovative sensors and the development of management tools.
Niv-AI was founded last year by CEO Tomer Timor and CTO Edward Kizis, and has garnered support from several investors, including Glilot Capital, Grove Ventures, Arc VC, Encoded VC, Leap Forward, and Aurora. Although the company has not disclosed its valuation, its goals are ambitious.
As cutting-edge labs operate thousands of GPUs in unison to train and execute advanced models, they experience frequent power demand surges that occur in milliseconds as processors transition between different computation tasks and coordinate with other GPUs. These surges complicate power management for data centers, making it challenging to draw sufficient electricity from the grid. To mitigate this risk, data centers either invest in temporary energy storage solutions or throttle their GPU usage, both of which diminish the return on investment in costly hardware.
“We just can’t continue building data centers the way we build them now,” remarked Lior Handelsman, a partner at Grove Ventures and a board member at Niv.
Niv’s initial focus involves deploying rack-level sensors capable of detecting GPU power usage at the millisecond level. The company plans to utilize these sensors on GPUs it owns, as well as those belonging to design partners. The objective is to comprehend the specific power profiles associated with various deep learning tasks and develop strategies to unlock additional capacity in data centers.
In the long run, Niv-AI aims to leverage the data collected to train an AI model that can predict and synchronize power loads across data centers, acting as a “copilot” for engineers managing these facilities.
The startup anticipates having an operational system in several U.S. data centers within the next six to eight months. This development is particularly appealing as hyperscalers encounter challenges related to land use and supply chain logistics while attempting to establish new data centers. Ultimately, the founders envision their product as a vital “intelligence layer” that bridges the gap between data centers and the electrical grid.
“The grid is actually afraid of the data center consuming too much power at a specific time,” Timor explained. “Our goal is to assist data centers in utilizing more GPUs, maximizing the power they are already paying for, while also fostering responsible power profiles between the data centers and the grid.”
Source: TechCrunch News