Nvidia may see a 'significant' opportunity from generative AI, but challenges remain
BING-JHEN HONG

Generative artificial intelligence, and in particular ChatGPT, has taken the investment world by storm in recent months and companies like Microsoft (MSFT) and Google (GOOG) (GOOGL) have noticed.
While some of the benefits (and pitfalls) of the technology may go to the world's largest tech companies, advanced semiconductor companies such as Nvidia (NASDAQ:NVDA) may also benefit from the trend, investment firm KeyBanc Capital Markets said.
In a research note, analyst John Vinh said Nvidia (NVDA) is the "clear outsized beneficiary" among semiconductor companies for generative AI thanks to the technology's heavy compute needs. 
Specifically for OpenAI's ChatGPT - which has some issues, as evidenced by Microsoft's (MSFT) recent defense of its chatbot for its new Bing search engine that is based on ChatGPT technology - Vinh believes the training cluster to help train the transformer language model is running roughly 10,000 A100 graphics processing unit, or GPUs, from the Jensen Huang-led company
"At $10K in [average selling prices], this would represent ~$100M in revenues," Vinh wrote in a note to clients. 
Vinh added that ChatGPT is likely also using GPUs for AI inferencing as well, though the number needed for this task is believed to be "significantly less" than the need for the training cluster.
Despite these opportunities, not everything is rosy for GPU usage when it comes to generative AI.
Some companies, such as Google (GOOG) (GOOGL), may opt to use tensor processing units, or TPUs, for its own training. Vinh added that Google is likely using TPUs for Bard, based off conversations. However, the analyst noted there is not likely a "significant operational cost advantage" for using TPUs compared to GPUs.
Earlier this month, Google (GOOG) (GOOGL) unveiled its Bard chatbot in a rushed response to the Microsoft (MSFT)-OpenAI deal, as well as an underwhelming event held in Paris showing off several AI-related updates that failed to impress analysts.
In addition, there are cost concerns, with the estimated cost of running a ChatGPT query believed to be between 30 cents and $1, Vinh added, well below the cost of running a traditional Google search query, which he estimates at one hundredth of a cent. Other analysts have estimated Google spends roughly one-fifth of a cent on a search query.
Nvidia's (NVDA) A100 GPU has an average selling price of approximately $10,000, while the H100, unveiled last March, has an ASP of around $25,000.
As such, companies may look to keep costs for training large language models, or LLMs, as low as they can, using alternative chips, perhaps such as AMD's (AMD) MI300.
In addition to AMD (AMD), Vinh added Broadcom (AVGO), Intel (INTC), Micron Technology (MU) and Marvell Technology (MRVL) are also likely to be beneficiaries of generative AI.
Other companies may also look to eschew GPUs and focus on custom silicon to keep costs low, as Amazon (AMZN) has done with its Graviton ARM CPU and other ARM-based server projects.
However, the pendulum may be swinging back in the direction of Nvidia (NVDA), Vinh suggested.
"[W]ith cloud providers increasing looking to cut Capex budgets, widespread layoffs in tech, and rising interest rates, there is increasing uncertainty in how aggressive these cloud projects will continue to be funded," Vinh explained. "We believe these trends could mark a shift back toward higher usage of merchant silicon," while providing a benefit for companies like Nvidia (NVDA).
Despite all the hype surrounding it, it is still early days for the rise of generative AI and its use cases beyond chatbots, Vinh noted. Nonetheless, "there is increasing pressure for generative AI initiatives to significantly lower the cost disparity with search."
For Nvidia (NVDA), the long-term total addressable market may be a "sizable opportunity" and with the company's continued advancements in AI, it's clear the company is focusing on the space as a way to drive growth.