Curbing the Generative AI Spam Machine
September 26, 2023

Kapil Tandon
Perforce

Share this

Efficiency. It's the latest buzz word that enterprise leaders everywhere are focused on this year, driven mainly by the promise of generative AI to help reduce the amount of time spent on the mundane and increase the workforce's productivity. It's these promised benefits that have driven increased interest in AI investment, and while technology possesses the opportunity to transform the way we work, just like anything else in history, it also has its shortcomings as well.

. The more dependent the workforce becomes on generative AI, despite its inaccuracies, the more mistakes will be made that can end up being costly. Until AI advances and its accuracy increases, all the AI investments organizations are currently investing in may not pay off after all.

What Does Generative AI Spam Look Like?

When people hear the word spam, they're typically thinking of mass marketing emails or unsolicited sales messages. However, in the case of generative AI in the enterprise, the spam it can produce looks a little different.

One tactic generative AI will be used for is to increase the number of scams shared through spam bots which can be leveraged for things like phishing emails. With AI tools available to the average citizen, any person regardless of their technical ability can now use AI to craft human-like messages and research better tactics for cyberattacks.

Typically, one method recipients can use to identify a spam email that may be a phishing scam is through incorrect grammar and spelling, but with generative AI chatbots like ChatGPT, bad actors can now leverage these tools to make their emails appear more legitimate. These actors can prompt ChatGPT or other large language models (LLMs) to draft emails with concise grammar and correct spelling, making it indistinguishable for consumers.

Another form of spam that will be produced is that overwhelming amount of data that end-users will flood generative AI models with. As more prompts are put in, and training data remains outdated or stagnant, the more likely it is for systems to produce copious amounts of incorrect information. With both the machine and humans using it. inputting and outputting massive amounts of irrelevant, inaccurate data, the less efficient AI will be for organizational productivity.

How Will This Impact the Bottom Line?

The rise in spam will coincide with the decrease in productivity gains that decision-makers were predicting AI systems would bring, therefore decreasing the value of using these tools. The democratization of AI, while great for education and awareness around technology, will only result in everyday users accidentally engaging with spam bots or phishing emails more frequently, in turn increasing the number of cyber incidents. With nearly half of cybersecurity leaders predicted to change jobs by 2025 due to elevated levels of stress and burnout, already overworked cyber teams will become extremely overburdened as cyberattacks due to AI rise and talent shortages continue.

With data revealing 96% of organizations address supply chain security problems on an adhoc basis, but only half have a formalized DevOps supply chain security strategy in place, it's expected AI will only exasperate software supply chain attacks and draw light to these gaps in strategy. The increase in security incidents from AI will lead to more organizations eventually adopting infrastructure state management and continuous compliance. This is critical to meet security standards and address growing concerns. However, until these mitigations are in place, overburdened security teams will continue to face challenges regarding DevOps supply chain security.

Additionally, as generative AI outputs begin to decrease in accuracy the older the model gets, the more time employees will have to spend sifting through garbage outputs to find valuable insights. When employees are assured that the outputs are accurate and that generative AI increases productivity, they will begin to become overdependent on antiquated models and work off incorrect data. If the generative AI model employees rely on is outdated and requires new training data, it will take away from the promised productivity gains and organizational efficiency. Not only will employees have to sift through responses, but developers will now have to dedicate more time to system maintenance and finding new training data.

Stopping the Generative AI Spam Machine

Even though AI adoption is inevitable and has been for a long time, organizations must be prepared to deal with the implications of outdated training data when the day comes. To prevent generative AI spam, developer teams should focus on continual data upkeep, making incremental additions where applicable to ensure the system is trained on the most up-to-date information. As more of the workforce becomes educated and comfortable with AI, the learning curve will be less steep, and users will begin to understand what a correct vs incorrect output is.

Similarly, use cases for AI will begin to narrow and become more specific to streamline the actual benefits. Decision-makers will weed out areas where AI is not adding real value and focus efforts on the areas where AI can increase operational efficiency. As the technology becomes more advanced, it will be easier to identify where leveraging AI may not be the best fit. AI may not be ready to make higher level decisions or compute substantial amounts of code, but if organizations focus on the right use cases and ensuring data is up to date, this will one day be a potential benefit for leaders in all industries.

While individual organizations may not be able to limit potential cyber criminals from leveraging chatbots and learning from AI, they can provide the correct education for employees. Teaching employees other ways to spot spam vs real requests, as well as general AI education, will help combat the wave of cyberattacks to come. Eventually, regulations will be introduced to handle these issues, especially around AI spam bots on social media, but in the meantime, educating employees is critical for organizational security.

Don't Fall for the Generative AI Spam Machine Trap

AI holds tremendous opportunities to upend and improve our daily lives. Whether it is being used to help brainstorm content, provide quick insights on research topics, and more, the current state of AI does hold benefits for many. However, beyond basic assistance tasks, generative AI is not in the place where its promise has been fully realized.

When making investments in AI, be sure to evaluate the areas of the business that may not be equipped to handle integrating AI into their workflows. When leaders focus on the tried-and-true use cases that have shown proven benefits, they can devote time to innovating for the AI model of tomorrow. Like any new advent in history, AI will display both the good and bad, and it is up to leaders to protect their enterprise and navigate the issues of today.

Kapil Tandon is VP of Product Management at Perforce
Share this

The Latest

April 25, 2024

The use of hybrid multicloud models is forecasted to double over the next one to three years as IT decision makers are facing new pressures to modernize IT infrastructures because of drivers like AI, security, and sustainability, according to the Enterprise Cloud Index (ECI) report from Nutanix ...

April 24, 2024

Over the last 20 years Digital Employee Experience has become a necessity for companies committed to digital transformation and improving IT experiences. In fact, by 2025, more than 50% of IT organizations will use digital employee experience to prioritize and measure digital initiative success ...

April 23, 2024

While most companies are now deploying cloud-based technologies, the 2024 Secure Cloud Networking Field Report from Aviatrix found that there is a silent struggle to maximize value from those investments. Many of the challenges organizations have faced over the past several years have evolved, but continue today ...

April 22, 2024

In our latest research, Cisco's The App Attention Index 2023: Beware the Application Generation, 62% of consumers report their expectations for digital experiences are far higher than they were two years ago, and 64% state they are less forgiving of poor digital services than they were just 12 months ago ...

April 19, 2024

In MEAN TIME TO INSIGHT Episode 5, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses the network source of truth ...

April 18, 2024

A vast majority (89%) of organizations have rapidly expanded their technology in the past few years and three quarters (76%) say it's brought with it increased "chaos" that they have to manage, according to Situation Report 2024: Managing Technology Chaos from Software AG ...

April 17, 2024

In 2024 the number one challenge facing IT teams is a lack of skilled workers, and many are turning to automation as an answer, according to IT Trends: 2024 Industry Report ...

April 16, 2024

Organizations are continuing to embrace multicloud environments and cloud-native architectures to enable rapid transformation and deliver secure innovation. However, despite the speed, scale, and agility enabled by these modern cloud ecosystems, organizations are struggling to manage the explosion of data they create, according to The state of observability 2024: Overcoming complexity through AI-driven analytics and automation strategies, a report from Dynatrace ...

April 15, 2024

Organizations recognize the value of observability, but only 10% of them are actually practicing full observability of their applications and infrastructure. This is among the key findings from the recently completed Logz.io 2024 Observability Pulse Survey and Report ...

April 11, 2024

Businesses must adopt a comprehensive Internet Performance Monitoring (IPM) strategy, says Enterprise Management Associates (EMA), a leading IT analyst research firm. This strategy is crucial to bridge the significant observability gap within today's complex IT infrastructures. The recommendation is particularly timely, given that 99% of enterprises are expanding their use of the Internet as a primary connectivity conduit while facing challenges due to the inefficiency of multiple, disjointed monitoring tools, according to Modern Enterprises Must Boost Observability with Internet Performance Monitoring, a new report from EMA and Catchpoint ...