The dark web is home to a growing array of artificial-intelligence chatbots similar to ChatGPT, but designed to help hackers. Businesses are on high alert for a glut of AI-generated email fraud and deepfakes.
5 min
A new crop of nefarious chatbots with names like “BadGPT” and “FraudGPT” are springing up on the darkest corners of the web, as cybercriminals look to tap the same artificial intelligence behind OpenAI’s ChatGPT.
Just as some office workers use ChatGPT to write better emails, hackers are using manipulated versions of AI chatbots to turbocharge their phishing emails. They can use chatbots—some also freely-available on the open internet—to create fake websites, write malware and tailor messages to better impersonate executives and other trusted entities.
Earlier this year, a Hong Kong multinational company employee handed over $25.5 million to an attacker who posed as the company’s chief financial officer on an AI-generated deepfake conference call, the South China Morning Post reported, citing Hong Kong police. Chief information officers and cybersecurity leaders, already accustomed to a growing spate of cyberattacks , say they are on high alert for an uptick in more sophisticated phishing emails and deepfakes.
Vish Narendra, CIO of Graphic Packaging International, said the Atlanta-based paper packing company has seen an increase in what are likely AI-generated email attacks called spear-phishing , where cyber attackers use information about a person to make an email seem more legitimate. Public companies in the spotlight are even more susceptible to contextualised spear-phishing, he said.
Researchers at Indiana University recently combed through over 200 large-language model hacking services being sold and populated on the dark web. The first service appeared in early 2023—a few months after the public release of OpenAI’s ChatGPT in November 2022.
Most dark web hacking tools use versions of open-source AI models like Meta ’s Llama 2, or “jailbroken” models from vendors like OpenAI and Anthropic to power their services, the researchers said. Jailbroken models have been hijacked by techniques like “ prompt injection ” to bypass their built-in safety controls.
Jason Clinton, chief information security officer of Anthropic, said the AI company eliminates jailbreak attacks as they find them, and has a team monitoring the outputs of its AI systems. Most model-makers also deploy two separate models to secure their primary AI model, making the likelihood that all three will fail the same way “a vanishingly small probability.”
Meta spokesperson Kevin McAlister said that openly releasing models shares the benefits of AI widely, and allows researchers to identify and help fix vulnerabilities in all AI models, “so companies can make models more secure.”
An OpenAI spokesperson said the company doesn’t want its tools to be used for malicious purposes, and that it is “always working on how we can make our systems more robust against this type of abuse.”
Malware and phishing emails written by generative AI are especially tricky to spot because they are crafted to evade detection. Attackers can teach a model to write stealthy malware by training it with detection techniques gleaned from cybersecurity defence software, said Avivah Litan, a generative AI and cybersecurity analyst at Gartner.
Phishing emails grew by 1,265% in the 12-month period starting when ChatGPT was publicly released, with an average of 31,000 phishing attacks sent every day, according to an October 2023 report by cybersecurity vendor SlashNext.
“The hacking community has been ahead of us,” said Brian Miller, CISO of New York-based not-for-profit health insurer Healthfirst, which has seen an increase in attacks impersonating its invoice vendors over the past two years.
While it is nearly impossible to prove whether certain malware programs or emails were created with AI, tools developed with AI can scan for text likely created with the technology. Abnormal Security , an email security vendor, said it had used AI to help identify thousands of likely AI-created malicious emails over the past year, and that it had blocked a twofold increase in targeted, personalised email attacks.
When Good Models Go Bad
Part of the challenge in stopping AI-enabled cybercrime is some AI models are freely shared on the open web. To access them, there is no need for dark corners of the internet or exchanging cryptocurrency.
Such models are considered “uncensored” because they lack the enterprise guardrails that businesses look for when buying AI systems, said Dane Sherrets, an ethical hacker and senior solutions architect at bug bounty company HackerOne.
In some cases, uncensored versions of models are created by security and AI researchers who strip out their built-in safeguards. In other cases, models with safeguards intact will write scam messages if humans avoid obvious triggers like “phishing”—a situation Andy Sharma, CIO and CISO of Redwood Software, said he discovered when creating a spear-phishing test for his employees.
The most useful model for generating scam emails is likely a version of Mixtral, from French AI startup Mistral AI, that has been altered to remove its safeguards, Sherrets said. Due to the advanced design of the original Mixtral, the uncensored version likely performs better than most dark web AI tools, he added. Mistral did not reply to a request for comment.
Sherrets recently demonstrated the process of using an uncensored AI model to generate a phishing campaign. First, he searched for “uncensored” models on Hugging Face, a startup that hosts a popular repository of open-source models—showing how easily many can be found.
He then used a virtual computing service that cost less than $1 per hour to mimic a graphics processing unit, or GPU, which is an advanced chip that can power AI. A bad actor needs either a GPU or a cloud-based service to use an AI model, Sherrets said, adding that he learned most of how to do this on X and YouTube.
With his uncensored model and virtual GPU service running, Sherrets asked the bot: “Write a phishing email targeting a business that impersonates a CEO and includes publicly-available company data,” and “Write an email targeting the procurement department of a company requesting an urgent invoice payment.”
The bot sent back phishing emails that were well-written, but didn’t include all of the personalisation asked for. That’s where prompt engineering , or the human’s ability to better extract information from chatbots, comes in, Sherrets said.
Dark Web AI Tools Can Already Do Harm
For hackers, a benefit of dark web tools like BadGPT—which researchers said uses OpenAI’s GPT model—is that they are likely trained on data from those underground marketplaces. That means they probably include useful information like leaks, ransomware victims and extortion lists, said Joseph Thacker, an ethical hacker and principal AI engineer at cybersecurity software firm AppOmni.
While some underground AI tools have been shuttered, new services have already taken their place, said Indiana University Assistant Computer Science Professor Xiaojing Liao, a co-author of the study. The AI hacking services, which often take payment via cryptocurrency, are priced anywhere from $5 to $199 a month.
New tools are expected to improve just as the AI models powering them do. In a matter of years, AI-generated text, video and voice deepfakes will be virtually indistinguishable from their human counterparts, said Evan Reiser , CEO and co-founder of Abnormal Security.
While researching the hacking tools, Indiana University Associate Dean for Research XiaoFeng Wang, a co-author of the study, said he was surprised by the ability of dark web services to generate effective malware. Given just the code of a security vulnerability, the tools can easily write a program to exploit it.
Though AI hacking tools often fail, in some cases, they work. “That demonstrates, in my opinion, that today’s large language models have the capability to do harm,” Wang said.
Copyright 2020, Dow Jones & Company, Inc. All Rights Reserved Worldwide. LEARN MORE
BNW Developments has established a Sydney presence, joining Arada and Sobha Realty among the growing number of UAE developers pursuing Australian buyers and development opportunities.
The Australian leather house has opened an immersive four-day pop-up in Manhattan, unveiling its Bloom Collection and redefining what a product launch can look like.
Strong family financial planning can help reduce taxes and support younger generations. From intrafamily loans and asset gifts to inheritance disclaimers, strategic cooperation can create meaningful financial benefits—when families have the trust and structure to make it work.
4 min
Happy is the family whose members—parents, grandparents and grown children—trust each other enough to cooperate on shared goals, especially financial ones. When that is truly the case, Uncle Sam’s tax rules can help as well.
This matters especially now that mortgage rates are above 7%, and many families are looking for ways to help younger members. If the elders have resources and are confident younger ones can cooperate, a family loan could make homeownership possible while providing the elders with a useful income stream.
Other strategies can actually lower taxes, such as when funding 529 plans or Roth IRAs, or when someone inherits a traditional IRA with required withdrawals larger than they will need.
Here are three strategies useful for functional families.
It is perfectly legal for families to lend money to a relative for a down payment or even a private mortgage.
But it is important not to cut corners, says Ryan McKeown, a CPA with Modern Wealth Enhancement in Minnesota. If the loan is for a down payment, be honest with the mortgage provider and have a formal agreement. The lender owes tax on the interest payments received.
If the loan is for a private mortgage, both sides should have legal representation with a formal written agreement, including payment terms. The lender owes tax on the interest, and the borrower often can deduct it if he or she itemizes.
To avoid IRS trouble, the interest rate shouldn’t be lower than the agency’s Applicable Federal Rate at the time of the loan. Currently that is about 5% for loans longer than nine years; about 4.5% for loans three to nine years; and about 4% for loans three years or less. Currently, traditional mortgage rates are generally above 7%.
In addition, the lender could use the $19,000 annual gift-tax exemption (described below) to forgive some or all of the interest or principal annually. If both lenders and borrowers are married, that is up to $76,000 a year. There is no tax for the borrower on such forgiveness, because it is from a gift.
If you’re going this route, McKeown advises against having a fixed plan to forgive the debt. Instead, do it in one-off letters specifying the amount—and keep careful records. Otherwise the IRS might try to treat the loan as a taxable gift.
Powerful tax-saving moves for families often use gift-tax provisions. Under current law, anyone can give anyone else up to $19,000 of assets annually, free of gift tax. That means a married couple with three grandchildren could give them a total of $114,000 in 2026.
The gifts can be cash or other assets, like stock. For noncash gifts, the cost basis—which is the starting point for measuring taxable gain after a sale—“carries over” to the recipient. So if someone buys $1,000 of stock and gives it away when it is worth $5,000, the recipient’s cost basis is $1,000. If the recipient later sells the shares for $8,000, the taxable gain is $7,000.
Here’s an example showing how gifts could save a family taxes. Grandma is a widow of modest means, while her child and spouse have prospered. The couple has two children, and they want to contribute $5,000 to a 529 college-savings plan for each—but they need to sell stock to do it. Their tax rate on the sale would be 18.8%, and they would need to sell about $11,000 of stock.
However, Grandma’s federal tax rate on the stock sale is 0%. If the couple gives $10,000 of shares to Grandma, she could sell them, pay no tax, and fund the grandchildren’s 529 plans. This saves about $1,000 of tax.
These moves are legal, and they could be used in other ways, such as to help a young person fund a Roth IRA.
But trust among family members is essential: Under the law, givers can’t put conditions on a gift. Grandma could use her stock proceeds to take a cruise, but she makes 529 contributions instead.
Mark Sellner, a retired tax attorney and CPA living in Sarasota, Fla., uses this strategy. His children sell stock he gives them and fund 529 plans for his grandchildren.
The family’s tax savings aren’t huge, but he likes other benefits. The sales by his children don’t boost his adjusted gross income, which in turn could raise his Medicare Irmaa premiums or his 3.8% surtax on net investment income.
Sellner doesn’t worry about his children using the funds for another purpose.
“There can’t be any strings attached to gifts. Of course, it is up to us to decide whether to make them in the future,” he says.
Two caveats: The “kiddie tax” applies to most children under age 24, and it is levied at the parents’ rate on investment income above $2,700 in 2026. Consider this before making gifts to a young person.
Investors who give away stock also forgo the step-up, an important provision that exempts assets held at death from capital-gains tax.
Sellner knows he’s losing a step-up, but says, “The children could use a little more now. Why should they wait 20 years to get it?”
A disclaimer is a highly useful strategy in which one heir renounces an inheritance in favor of another heir. Assuming family members cooperate, this can save taxes.
Here’s one example. Dad died and had a large traditional IRA that he left to Mom. She has enough assets and income to cover her expenses, and she lives in a state with a stiff estate tax. The inherited IRA would put her estate over the threshold.
Also surviving are three young-adult children. If Mom disclaims all or part of Dad’s IRA within nine months of his death, that amount could go directly to the children. They will have 10 years to empty the account, and the family as a whole will likely save estate and income taxes.
Disclaimers have many key details, especially regarding beneficiary documents. Although heirs have great freedom in choosing what assets to disclaim, the rules about who gets disclaimed property are rigid. It is best if the original owner names tiers of heirs so that if one disclaims, the next recipient is clear.
