M.B.A. Students vs. ChatGPT: Who Comes Up With More Innovative Ideas?
We put humans and AI to the test. The results weren’t even close.
By CHRISTIAN TERWIESCH
Thu, Sep 14, 2023 9:01am
4 min
4 min
How good is AI in generating new ideas?
The conventional wisdom has been not very good. Identifying opportunities for new ventures, generating a solution for an unmet need, or naming a new company are unstructured tasks that seem ill-suited for algorithms. Yet recent advances in AI, and specifically the advent of large language models like ChatGPT, are challenging these assumptions.
We have taught innovation, entrepreneurship and product design for many years. For the first assignment in our innovation courses at the Wharton School, we ask students to generate a dozen or so ideas for a new product or service. As a result, we have heard several thousand new venture ideas pitched by undergraduate students, M.B.A. students and seasoned executives. Some of these ideas are awesome, some are awful, and, as you would expect, most are somewhere in the middle.
The library of ideas, though, allowed us to set up a simple competition to judge who is better at generating innovative ideas: the human or the machine.
In this competition, which we ran together with our colleagues Lennart Meincke and Karan Girotra, humanity was represented by a pool of 200 randomly selected ideas from our Wharton students. The machines were represented by ChatGPT4, which we instructed to generate 100 ideas with otherwise identical instructions as given to the students: “generate an idea for a new product or service appealing to college students that could be made available for $50 or less.”
In addition to this vanilla prompt, we also asked ChatGPT for another 100 ideas after providing a handful of examples of successful ideas from past courses (in other words, a trained GPT group), providing us with a total sample of 400 ideas.
Collapsible laundry hamper, dorm-room chef kit, ergonomic cushion for hard classroom seats, and hundreds more ideas miraculously spewed from a laptop.
How to compare
The academic literature on ideation postulates three dimensions of creative performance: the quantity of ideas, the average quality of ideas, and the number of truly exceptional ideas.
First, on the number of ideas per unit of time: Not surprisingly, ChatGPT easily outperforms us humans on that dimension. Generating 200 ideas the old-fashioned way requires days of human work, while ChatGPT can spit out 200 ideas with about an hour of supervision.
Next, to assess the quality of the ideas, we market tested them. Specifically, we took each of the 400 ideas and put them in front of a survey panel of customers in the target market via an online purchase-intent survey. The question we asked was: “How likely would you be to purchase based on this concept if it were available to you?” The possible responses ranged from definitely wouldn’t purchase to definitely would purchase.
The responses can be translated into a purchase probability using simple market-research techniques. The average purchase probability of a human-generated idea was 40%, that of vanilla GPT-4 was 47%, and that of GPT-4 seeded with good ideas was 49%. In short, ChatGPT isn’t only faster but also on average better at idea generation.
Still, when you’re looking for great ideas, averages can be misleading. In innovation, it’s the exceptional ideas that matter: Most managers would prefer one idea that is brilliant and nine ideas that are flops over 10 decent ideas, even if the average quality of the latter option might be higher. To capture this perspective, we investigated only the subset of the best ideas in our pool—specifically the top 10%. Of these 40 ideas, five were generated by students and 35 were created by ChatGPT (15 from the vanilla ChatGPT set and 20 from the pre trained ChatGPT set). Once again, ChatGPT came out on top.
What it means
We believe that the 35-to-5 victory of the machine in generating exceptional ideas (not to mention the dramatically lower production costs) has substantial implications for how we think about creativity and innovation.
First, generative AI has brought a new source of ideas to the world. Not using this source would be a sin. It doesn’t matter if you are working on a pitch for your local business-plan competition or if you are seeking a cure for cancer—every innovator should develop the habit of complementing his or her own ideas with the ones created by technology. Ideation will always have an element of randomness to it, and so we cannot guarantee that your idea will get an A+, but there is no excuse left if you get a C.
Second, the bottleneck for the early phases of the innovation process in organisations now shifts from generating ideas to evaluating ideas. Using a large language model, an innovator can produce a spreadsheet articulating hundreds of ideas, which likely include a few blockbusters. This abundance then demands an effective selection mechanism to find the needles in the haystack.
To date, these models appear to perform no better than any single expert in their ability to predict commercial viability. Using a sample of a dozen or so independent evaluations from potential customers in the target market—a wisdom of crowds approach—remains the best strategy. Fortunately, screening ideas using a purchase intent survey of customers in the target market is relatively fast and cheap.
Finally, rather than thinking about a competition between humans and machines, we should find a way in which the two work together. This approach in which AI takes on the role of a co-pilot has already emerged in software development. For example, our human (pilot) innovator might identify an open problem. The AI (co-pilot) might then report what is known about the problem, followed by an effort in which the human and AI independently explore possible solutions, virtually guaranteeing a thorough consideration of opportunities.
The human decision maker is likely ultimately responsible for the outcome, and so will likely make the screening and selection decisions, informed by customer research and possibly by the opinion of the AI co-pilot. We predict such a human-machine collaboration will deliver better products and services to the market, and improved solutions for whatever society needs in the future.
Christian Terwiesch and Karl Ulrich are professors of operations, information and decisions at the Wharton School of the University of Pennsylvania, where Terwiesch also co-directs the Mack Institute for Innovation Management.
Copyright 2020, Dow Jones & Company, Inc. All Rights Reserved Worldwide. LEARN MORE
MOST POPULAR
The Australian leather house has opened an immersive four-day pop-up in Manhattan, unveiling its Bloom Collection and redefining what a product launch can look like.
Following the successful launch of its Palais Collection, MAISON de SABRÉ has unveiled a new modular handbag system offering more than 720 styling combinations.
Gold miners are emerging as a compelling way to navigate market uncertainty, with analysts pointing to strong cash flows, attractive valuations and rising profit margins. As gold prices stabilize above US$4,000 an ounce, mining stocks could offer investors both downside protection and long-term upside.
By Paul R. La Monica
Thu, Aug 6, 2026 3 min
3 min
Gold is one of the market’s go-to hedges in rocky times. Don’t forget that gold miners’ stocks are too.
The stock market’s gains in 2026 belie the rocky macroeconomic picture: elevated inflation, heightened geopolitical tensions, and jitters about the artificial-intelligence trade. That backdrop, in theory, should be the time for gold to shine. Instead, the price of the yellow metal has tumbled more than 5% so far, after last year’s blistering 65% rally. In part, the U.S. dollar’s recovery has stymied gold, which benefited from the greenback’s weakness in 2025.
Even with the precious metal’s recent weakness, gold mining stocks could be the best way to profit from this year’s uncertainty.
Gold miners “are a valuable hedge against macro risks that would likely be damaging for equities,” BCA Research’s Noah Weisberger and Rishabh Shah wrote this week.
Concerns about the Federal Reserve’s next moves to tackle inflation, the increasingly crowded AI trade, and steep valuations for tech stocks are just some of the drivers that could help gold’s price get on even footing— and lead to even bigger gains for miner stocks.
These stocks’ prices tend to outpace gold’s moves, because the companies have fixed operational costs. So when gold’s price rallies, their profit margins soar, and vice versa. For instance, the VanEck Gold Miners GDX +7.39% exchange-traded fund has fallen 11% this year as the metal has slumped.
Now, gold’s price just needs to stabilize to help miners’ stocks take off, and that seems to be happening. The precious metal has recently found support above the $4,000 level, and has stuck in a narrow range since the end of June. But its price rose ever so slightly in July, ending a four-month losing streak for the metal. Technical analysis also suggests that gold is due for a comeback.
Barron’s recently wrote that the pullbacks for both gold miners and the metal itself are overdone. Senior technical analyst Doug Busch noted that the VanEck ETF is on the “verge of a breakout” and has the potential to hit $11o in early 2027, up more than 40% from its current price.
Gold miners also have more than their role as a market hedge going for them. Their fundamentals are solid, too, says Chris Mancini, portfolio co-manager of the Gabelli Gold Fund.
“Precious metals miners are generating substantial amounts of free cash flow given profit margins of over $2,000 per ounce, and are returning this cash to shareholders through buybacks and dividends,” he said in an email.
“Buying the miners is a cheap way to get exposure to the price of gold,” he added. His fund owns Newmont NEM +6.71%, a Barron’s stock pick last year, and Agnico Eagle Mines as top holdings, as well as miners Northern Star Resources, Endeavour Mining, and Kinross Gold K+8.59%.
Miners are better businesses than they used to be, the BCA team added.
“Capex is more disciplined, margins are high and rising…and they are largely independent of the AI story,” Weisberger, BCA’s head of equities, and Shah, a senior analyst, wrote.
That last part is key. AI is disrupting the software industry and many other services and information-oriented businesses, and investors have piled into AI stocks. But ChatGPT, Claude, Grok, and other large-language models aren’t going to replace the need to mine for metals.
“Equity portfolios can benefit from exposure to quality that is uncorrelated to AI risk, and gold miners fit the bill,” the BCA team said.
They recommend that investors buy the VanEck Gold Miners ETF, which owns top miners such as Agnico, Barrick Mining ABX +7.24%, and Newmont.
An important bonus for big gold miners’ stocks is that their valuations are attractive after the gold’s pullback, too. The VanEck ETF is now trading at just a little more than nine times next year’s earnings estimates. That’s a big discount to its five-year average price-to-earnings ratio of 14, according to FactSet.
What’s more, the ETF is currently valued at a more than 50% discount to the S&P 500 SPX -0.17%, which is trading for about 19 times earnings estimates for 2027. Mining stocks have typically traded at just a 25% discount to the broader market over the past five years. So there is significant upside for the group if valuations move back toward normal levels.
One factor that complicates mining stocks as a market hedge, of course, is if stocks bounce back, which has been the case so far in August.
But both the market and economic outlooks remain cloudy, and investors remain nervous about the Fed’s next moves and AI stocks. Gold miners should do just fine, even if the anxious mood on Wall Street persists.
