For a few thousand dollars a month, you can now reserve the capacity of a single Nvidia HGX H100 GPU through a company called CoreWeave. The H100 is the successor to the A100, the GPU that was instrumental in shaping ChatGPT into LLMs (Large Language Models). Prices start at €2.33 per hour, or €56 per day or around €20,000 per year; for comparison, a single HGX H100 costs around €28,000 on the open market (NVH100TCGPU-KIT) and less wholesale.
You'll pay more for spot prices ($4,76 per hour) and while there's a cheaper SKU available (HGX H100 PCIe, as opposed to the NVLINK model), you still can't order it. A valid GPU instance configuration must include at least one GPU, at least one vCPU, and at least 2 GB of RAM. When deploying a virtual private server (VPS), the GPU instance configuration must also include at least 40 GB of NVMe-grade storage on the root disk.
The news comes after a series of announcements at Nvidia's GTC 2023, where generative AI was left, right, and center. Technology uses an LLM education that allows for creative work, including writing academic papers, a stand-up comedy routine, or a sonnet; design artwork from a block of text; and in the case of NovelAI, one of the first CoreWeave clients, which composes literature.
You can of course check out what our folks at Tomshardware have done and run a ChatGPT alternative on your local PC.
Nvidia monopoly
Jensen Huang, founder and CEO of NVIDIA, oversaw the launch of several GPUs aimed at specific segments of the growing AI content market: the L4 for AI video, the L40 for imaging, and the H100 NVL (essentially two H100s in an SLI-esque setup). Nvidia, which turns 30 in April 2023, wants to capture as much of the market as possible by offering hardware and software to implement its own hardware-as-a-service.
It unveiled a cloud version of its DGX H100 server, which contains eight H100 cards and can be leased for just under €37,000 from Oracle with Microsoft and Google coming soon. While that sounds expensive, just keep in mind that the DGX H100 costs upwards of €500,000 from enterprise vendor Insight, and that excludes the actual cost of running the device (maintenance, placement, utilities, etc.).
Nvidia's sudden interest in becoming its own service provider may make its partners a little uncomfortable. The president of TIEA (Taiwan Internet and E-Commerce Association), which brings together some of the biggest names in the technology hardware industry, was lucid enough to say yesterday that the company will "compete cooperatively" with major vendors. cloud services (CSP). ), and will likely speed up the search for an alternative to Nvidia, to strike a balance.
All eyes are on AMD (with its Instinct MI300 GPU) and Intel, but lurking in the shadows is a list of challengers (Graphcore, Cerebras, Kneron, IBM, and others) that will want a piece of a growing pie.