Neil KumarThe Dark Side of Generative AI
How ChatGPT and Other Tech Can Foster Fraud

By Neil Kumar, CAMS, CGSS, Assistant Vice President, Compliance

April 5, 2023


Have you heard of generative artificial intelligence (AI)? How about ChatGPT? While these technological breakthroughs have great potential and practical applications, there is cause for concern about their potential to foster fraud.

What is generative AI exactly? Per McKinsey & Company, generative AI systems fall under the broad category of machine learning – a type of artificial intelligence that enables machinery to learn from data patterns without human direction or intervention. It’s highly adaptable and customizable as it can create new content based on inputs and public data. Plus, it evolves over time based on prior interactions to improve its response. ChatGPT is an example of one such machine learning system. McKinsey & Company argues, “The unmanageably huge volume and complexity of data (unmanageable by humans, anyway) that is now being generated has increased the potential of machine learning, as well as the need for it.” So, while generative AI can simplify communication and improve efficiency, it's also essential to recognize its possible misuse.

At Alloya, our compliance professionals have been reviewing the ways that scammers could use generative AI to enhance their fraud communications with victims. We've been amazed to see how well the AI responds to simple requests, yet troubled about the implications of the technology's adaptability.