Skip to main content

Curbing the Generative AI Spam Machine

Kapil Tandon
Perforce

Efficiency. It's the latest buzz word that enterprise leaders everywhere are focused on this year, driven mainly by the promise of generative AI to help reduce the amount of time spent on the mundane and increase the workforce's productivity. It's these promised benefits that have driven increased interest in AI investment, and while technology possesses the opportunity to transform the way we work, just like anything else in history, it also has its shortcomings as well.

. The more dependent the workforce becomes on generative AI, despite its inaccuracies, the more mistakes will be made that can end up being costly. Until AI advances and its accuracy increases, all the AI investments organizations are currently investing in may not pay off after all.

What Does Generative AI Spam Look Like?

When people hear the word spam, they're typically thinking of mass marketing emails or unsolicited sales messages. However, in the case of generative AI in the enterprise, the spam it can produce looks a little different.

One tactic generative AI will be used for is to increase the number of scams shared through spam bots which can be leveraged for things like phishing emails. With AI tools available to the average citizen, any person regardless of their technical ability can now use AI to craft human-like messages and research better tactics for cyberattacks.

Typically, one method recipients can use to identify a spam email that may be a phishing scam is through incorrect grammar and spelling, but with generative AI chatbots like ChatGPT, bad actors can now leverage these tools to make their emails appear more legitimate. These actors can prompt ChatGPT or other large language models (LLMs) to draft emails with concise grammar and correct spelling, making it indistinguishable for consumers.

Another form of spam that will be produced is that overwhelming amount of data that end-users will flood generative AI models with. As more prompts are put in, and training data remains outdated or stagnant, the more likely it is for systems to produce copious amounts of incorrect information. With both the machine and humans using it. inputting and outputting massive amounts of irrelevant, inaccurate data, the less efficient AI will be for organizational productivity.

How Will This Impact the Bottom Line?

The rise in spam will coincide with the decrease in productivity gains that decision-makers were predicting AI systems would bring, therefore decreasing the value of using these tools. The democratization of AI, while great for education and awareness around technology, will only result in everyday users accidentally engaging with spam bots or phishing emails more frequently, in turn increasing the number of cyber incidents. With nearly half of cybersecurity leaders predicted to change jobs by 2025 due to elevated levels of stress and burnout, already overworked cyber teams will become extremely overburdened as cyberattacks due to AI rise and talent shortages continue.

With data revealing 96% of organizations address supply chain security problems on an adhoc basis, but only half have a formalized DevOps supply chain security strategy in place, it's expected AI will only exasperate software supply chain attacks and draw light to these gaps in strategy. The increase in security incidents from AI will lead to more organizations eventually adopting infrastructure state management and continuous compliance. This is critical to meet security standards and address growing concerns. However, until these mitigations are in place, overburdened security teams will continue to face challenges regarding DevOps supply chain security.

Additionally, as generative AI outputs begin to decrease in accuracy the older the model gets, the more time employees will have to spend sifting through garbage outputs to find valuable insights. When employees are assured that the outputs are accurate and that generative AI increases productivity, they will begin to become overdependent on antiquated models and work off incorrect data. If the generative AI model employees rely on is outdated and requires new training data, it will take away from the promised productivity gains and organizational efficiency. Not only will employees have to sift through responses, but developers will now have to dedicate more time to system maintenance and finding new training data.

Stopping the Generative AI Spam Machine

Even though AI adoption is inevitable and has been for a long time, organizations must be prepared to deal with the implications of outdated training data when the day comes. To prevent generative AI spam, developer teams should focus on continual data upkeep, making incremental additions where applicable to ensure the system is trained on the most up-to-date information. As more of the workforce becomes educated and comfortable with AI, the learning curve will be less steep, and users will begin to understand what a correct vs incorrect output is.

Similarly, use cases for AI will begin to narrow and become more specific to streamline the actual benefits. Decision-makers will weed out areas where AI is not adding real value and focus efforts on the areas where AI can increase operational efficiency. As the technology becomes more advanced, it will be easier to identify where leveraging AI may not be the best fit. AI may not be ready to make higher level decisions or compute substantial amounts of code, but if organizations focus on the right use cases and ensuring data is up to date, this will one day be a potential benefit for leaders in all industries.

While individual organizations may not be able to limit potential cyber criminals from leveraging chatbots and learning from AI, they can provide the correct education for employees. Teaching employees other ways to spot spam vs real requests, as well as general AI education, will help combat the wave of cyberattacks to come. Eventually, regulations will be introduced to handle these issues, especially around AI spam bots on social media, but in the meantime, educating employees is critical for organizational security.

Don't Fall for the Generative AI Spam Machine Trap

AI holds tremendous opportunities to upend and improve our daily lives. Whether it is being used to help brainstorm content, provide quick insights on research topics, and more, the current state of AI does hold benefits for many. However, beyond basic assistance tasks, generative AI is not in the place where its promise has been fully realized.

When making investments in AI, be sure to evaluate the areas of the business that may not be equipped to handle integrating AI into their workflows. When leaders focus on the tried-and-true use cases that have shown proven benefits, they can devote time to innovating for the AI model of tomorrow. Like any new advent in history, AI will display both the good and bad, and it is up to leaders to protect their enterprise and navigate the issues of today.

Kapil Tandon is VP of Product Management at Perforce

Hot Topics

The Latest

I've spent a lot of time in the channel, and one thing I keep coming back to is this: a partner program is only as good as what it looks like in the field. Many programs look great on paper, but when a partner is in front of a customer navigating a complex hybrid environment or trying to make the case for AI-powered observability, the gap between what a vendor promises and what it actually delivers becomes very clear, very fast ...

Enterprises today operate in a real-time environment where uninterrupted access to trusted data has become a baseline expectation for users, applications and automated systems. Traditional DataOps models, built on manual effort and human triage, cannot keep pace with this always active demand. AI agents are emerging as the operational backbone, ensuring consistent data availability, reinforcing trustworthiness and enabling a level of scale that manual processes cannot achieve ...

For decades, trust in the digital workplace rested on familiar signals. We trusted faces on video calls, voices on the phone, and emails that appeared to come from people we knew. These cues felt human and intuitive. They anchored how decisions were made, approvals were granted, and access was authorized. AI-powered deepfakes have quietly broken that model ...

Cloud migration was supposed to be a one-way door. For most enterprises, it turns out it isn't. Cloud data repatriation is a real and growing trend. A new survey ... finds that 89% of organizations plan to expand their on-premises infrastructure footprint over the next two years — and 75% have already moved at least some workloads back from public cloud in the past 24 months. The findings point to a broad rethinking of where data belongs ...

Over the past few years, large language models (LLMs) have revolutionized the software industry. Given their ability to excel at multi-step reasoning, LLMs have helped enterprises streamline workflows and adapt to the unknown. However, employing such models comes with sky-high costs, latency issues, and limited flexibility. In the realm of IT operations, it is generally wiser to employ smaller, domain-specific models instead ...

For years, DevOps teams operated under a simple assumption: collect enough telemetry, and you can find and fix any problem. That assumption is breaking down. Modern enterprises now operate across microservices, hybrid cloud environments, APIs, Kubernetes, and highly automated delivery pipelines. Releases happen continuously, dependencies shift constantly, and failures spread faster than teams can diagnose them ...

New Relic surveyed IT and engineering leaders from the media and entertainment (M&E) sector to understand what's working — and where challenges persist with their observability practices. The findings reveal how M&E organizations are navigating rising platform complexity, audience expectations, and AI-driven change. Below are five takeaways that stand out ...

Let me start with something I've seen play out more times than I can count. A team hits a wall with the cloud. Costs creep up, then spike. Performance starts to feel inconsistent. Someone in finance asks a simple question like "why did this double?" and nobody has a clean answer ... Maybe this isn't the right place for everything. That realization feels like a breakthrough, like you've identified the problem. In reality, you've just identified the starting line ...

In MEAN TIME TO INSIGHT Episode 24, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses network observability tool sprawl ... 

In cloud-native systems, scaling is often as simple as moving a slider. For on-premise databases, the stakes are different. Over-provisioning hardware is expensive. Under-provisioning leads to performance bottlenecks that are difficult to fix once the equipment is in the rack ...

Curbing the Generative AI Spam Machine

Kapil Tandon
Perforce

Efficiency. It's the latest buzz word that enterprise leaders everywhere are focused on this year, driven mainly by the promise of generative AI to help reduce the amount of time spent on the mundane and increase the workforce's productivity. It's these promised benefits that have driven increased interest in AI investment, and while technology possesses the opportunity to transform the way we work, just like anything else in history, it also has its shortcomings as well.

. The more dependent the workforce becomes on generative AI, despite its inaccuracies, the more mistakes will be made that can end up being costly. Until AI advances and its accuracy increases, all the AI investments organizations are currently investing in may not pay off after all.

What Does Generative AI Spam Look Like?

When people hear the word spam, they're typically thinking of mass marketing emails or unsolicited sales messages. However, in the case of generative AI in the enterprise, the spam it can produce looks a little different.

One tactic generative AI will be used for is to increase the number of scams shared through spam bots which can be leveraged for things like phishing emails. With AI tools available to the average citizen, any person regardless of their technical ability can now use AI to craft human-like messages and research better tactics for cyberattacks.

Typically, one method recipients can use to identify a spam email that may be a phishing scam is through incorrect grammar and spelling, but with generative AI chatbots like ChatGPT, bad actors can now leverage these tools to make their emails appear more legitimate. These actors can prompt ChatGPT or other large language models (LLMs) to draft emails with concise grammar and correct spelling, making it indistinguishable for consumers.

Another form of spam that will be produced is that overwhelming amount of data that end-users will flood generative AI models with. As more prompts are put in, and training data remains outdated or stagnant, the more likely it is for systems to produce copious amounts of incorrect information. With both the machine and humans using it. inputting and outputting massive amounts of irrelevant, inaccurate data, the less efficient AI will be for organizational productivity.

How Will This Impact the Bottom Line?

The rise in spam will coincide with the decrease in productivity gains that decision-makers were predicting AI systems would bring, therefore decreasing the value of using these tools. The democratization of AI, while great for education and awareness around technology, will only result in everyday users accidentally engaging with spam bots or phishing emails more frequently, in turn increasing the number of cyber incidents. With nearly half of cybersecurity leaders predicted to change jobs by 2025 due to elevated levels of stress and burnout, already overworked cyber teams will become extremely overburdened as cyberattacks due to AI rise and talent shortages continue.

With data revealing 96% of organizations address supply chain security problems on an adhoc basis, but only half have a formalized DevOps supply chain security strategy in place, it's expected AI will only exasperate software supply chain attacks and draw light to these gaps in strategy. The increase in security incidents from AI will lead to more organizations eventually adopting infrastructure state management and continuous compliance. This is critical to meet security standards and address growing concerns. However, until these mitigations are in place, overburdened security teams will continue to face challenges regarding DevOps supply chain security.

Additionally, as generative AI outputs begin to decrease in accuracy the older the model gets, the more time employees will have to spend sifting through garbage outputs to find valuable insights. When employees are assured that the outputs are accurate and that generative AI increases productivity, they will begin to become overdependent on antiquated models and work off incorrect data. If the generative AI model employees rely on is outdated and requires new training data, it will take away from the promised productivity gains and organizational efficiency. Not only will employees have to sift through responses, but developers will now have to dedicate more time to system maintenance and finding new training data.

Stopping the Generative AI Spam Machine

Even though AI adoption is inevitable and has been for a long time, organizations must be prepared to deal with the implications of outdated training data when the day comes. To prevent generative AI spam, developer teams should focus on continual data upkeep, making incremental additions where applicable to ensure the system is trained on the most up-to-date information. As more of the workforce becomes educated and comfortable with AI, the learning curve will be less steep, and users will begin to understand what a correct vs incorrect output is.

Similarly, use cases for AI will begin to narrow and become more specific to streamline the actual benefits. Decision-makers will weed out areas where AI is not adding real value and focus efforts on the areas where AI can increase operational efficiency. As the technology becomes more advanced, it will be easier to identify where leveraging AI may not be the best fit. AI may not be ready to make higher level decisions or compute substantial amounts of code, but if organizations focus on the right use cases and ensuring data is up to date, this will one day be a potential benefit for leaders in all industries.

While individual organizations may not be able to limit potential cyber criminals from leveraging chatbots and learning from AI, they can provide the correct education for employees. Teaching employees other ways to spot spam vs real requests, as well as general AI education, will help combat the wave of cyberattacks to come. Eventually, regulations will be introduced to handle these issues, especially around AI spam bots on social media, but in the meantime, educating employees is critical for organizational security.

Don't Fall for the Generative AI Spam Machine Trap

AI holds tremendous opportunities to upend and improve our daily lives. Whether it is being used to help brainstorm content, provide quick insights on research topics, and more, the current state of AI does hold benefits for many. However, beyond basic assistance tasks, generative AI is not in the place where its promise has been fully realized.

When making investments in AI, be sure to evaluate the areas of the business that may not be equipped to handle integrating AI into their workflows. When leaders focus on the tried-and-true use cases that have shown proven benefits, they can devote time to innovating for the AI model of tomorrow. Like any new advent in history, AI will display both the good and bad, and it is up to leaders to protect their enterprise and navigate the issues of today.

Kapil Tandon is VP of Product Management at Perforce

Hot Topics

The Latest

I've spent a lot of time in the channel, and one thing I keep coming back to is this: a partner program is only as good as what it looks like in the field. Many programs look great on paper, but when a partner is in front of a customer navigating a complex hybrid environment or trying to make the case for AI-powered observability, the gap between what a vendor promises and what it actually delivers becomes very clear, very fast ...

Enterprises today operate in a real-time environment where uninterrupted access to trusted data has become a baseline expectation for users, applications and automated systems. Traditional DataOps models, built on manual effort and human triage, cannot keep pace with this always active demand. AI agents are emerging as the operational backbone, ensuring consistent data availability, reinforcing trustworthiness and enabling a level of scale that manual processes cannot achieve ...

For decades, trust in the digital workplace rested on familiar signals. We trusted faces on video calls, voices on the phone, and emails that appeared to come from people we knew. These cues felt human and intuitive. They anchored how decisions were made, approvals were granted, and access was authorized. AI-powered deepfakes have quietly broken that model ...

Cloud migration was supposed to be a one-way door. For most enterprises, it turns out it isn't. Cloud data repatriation is a real and growing trend. A new survey ... finds that 89% of organizations plan to expand their on-premises infrastructure footprint over the next two years — and 75% have already moved at least some workloads back from public cloud in the past 24 months. The findings point to a broad rethinking of where data belongs ...

Over the past few years, large language models (LLMs) have revolutionized the software industry. Given their ability to excel at multi-step reasoning, LLMs have helped enterprises streamline workflows and adapt to the unknown. However, employing such models comes with sky-high costs, latency issues, and limited flexibility. In the realm of IT operations, it is generally wiser to employ smaller, domain-specific models instead ...

For years, DevOps teams operated under a simple assumption: collect enough telemetry, and you can find and fix any problem. That assumption is breaking down. Modern enterprises now operate across microservices, hybrid cloud environments, APIs, Kubernetes, and highly automated delivery pipelines. Releases happen continuously, dependencies shift constantly, and failures spread faster than teams can diagnose them ...

New Relic surveyed IT and engineering leaders from the media and entertainment (M&E) sector to understand what's working — and where challenges persist with their observability practices. The findings reveal how M&E organizations are navigating rising platform complexity, audience expectations, and AI-driven change. Below are five takeaways that stand out ...

Let me start with something I've seen play out more times than I can count. A team hits a wall with the cloud. Costs creep up, then spike. Performance starts to feel inconsistent. Someone in finance asks a simple question like "why did this double?" and nobody has a clean answer ... Maybe this isn't the right place for everything. That realization feels like a breakthrough, like you've identified the problem. In reality, you've just identified the starting line ...

In MEAN TIME TO INSIGHT Episode 24, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses network observability tool sprawl ... 

In cloud-native systems, scaling is often as simple as moving a slider. For on-premise databases, the stakes are different. Over-provisioning hardware is expensive. Under-provisioning leads to performance bottlenecks that are difficult to fix once the equipment is in the rack ...