Ziroh Labs, an AI startup based in India, has developed a new platform named Kompact AI in collaboration with the Indian Institute of Technology (IIT) Madras. The system is designed to operate large AI models without the need for high-end GPU hardware, instead leveraging widely available CPU technology.

Overcoming GPU Dependency for AI Inference

Kompact AI primarily addresses the inference stage of AI, where models are used after training. Unlike traditional systems that depend on expensive GPUs, this platform allows models to function efficiently on standard CPUs, such as Intel Xeon processors found in regular laptops and desktops.

Demonstrated Compatibility with Leading AI Models

In a recent demonstration, Kompact AI successfully ran Meta’s Llama 2 and Alibaba’s Qwen2.5 models on a shelf-bought CPU-based laptop. Ziroh Labs emphasized that the system delivers high-quality outputs despite not using specialized hardware.

Industry Validation and Broader Impact

Kompact AI has been tested by major chipmakers including Intel and AMD. William Raduchel, former strategy chief at Sun Microsystems and current advisor to the startup, described the technology as having “profound market impact” in future years.

Bridging the AI Accessibility Gap in India

Due to the limited availability and high cost of Nvidia GPUs in India, many local developers face challenges in AI development. Kompact AI offers an alternative path that could help democratize access to powerful AI tools in emerging markets.

IIT Madras: Championing Inclusive AI Innovation

Professor V. Kamakoti, Director of IIT Madras, commented on the broader implications of the project, highlighting the opportunity to narrow the “AI divide” caused by unequal access to computing infrastructure.

Read More: Ziroh Labs Unveils Kompact AI to Run Large Models Without GPUs