Skip to main content

Curbing the Generative AI Spam Machine

Kapil Tandon
Perforce

Efficiency. It's the latest buzz word that enterprise leaders everywhere are focused on this year, driven mainly by the promise of generative AI to help reduce the amount of time spent on the mundane and increase the workforce's productivity. It's these promised benefits that have driven increased interest in AI investment, and while technology possesses the opportunity to transform the way we work, just like anything else in history, it also has its shortcomings as well.

. The more dependent the workforce becomes on generative AI, despite its inaccuracies, the more mistakes will be made that can end up being costly. Until AI advances and its accuracy increases, all the AI investments organizations are currently investing in may not pay off after all.

What Does Generative AI Spam Look Like?

When people hear the word spam, they're typically thinking of mass marketing emails or unsolicited sales messages. However, in the case of generative AI in the enterprise, the spam it can produce looks a little different.

One tactic generative AI will be used for is to increase the number of scams shared through spam bots which can be leveraged for things like phishing emails. With AI tools available to the average citizen, any person regardless of their technical ability can now use AI to craft human-like messages and research better tactics for cyberattacks.

Typically, one method recipients can use to identify a spam email that may be a phishing scam is through incorrect grammar and spelling, but with generative AI chatbots like ChatGPT, bad actors can now leverage these tools to make their emails appear more legitimate. These actors can prompt ChatGPT or other large language models (LLMs) to draft emails with concise grammar and correct spelling, making it indistinguishable for consumers.

Another form of spam that will be produced is that overwhelming amount of data that end-users will flood generative AI models with. As more prompts are put in, and training data remains outdated or stagnant, the more likely it is for systems to produce copious amounts of incorrect information. With both the machine and humans using it. inputting and outputting massive amounts of irrelevant, inaccurate data, the less efficient AI will be for organizational productivity.

How Will This Impact the Bottom Line?

The rise in spam will coincide with the decrease in productivity gains that decision-makers were predicting AI systems would bring, therefore decreasing the value of using these tools. The democratization of AI, while great for education and awareness around technology, will only result in everyday users accidentally engaging with spam bots or phishing emails more frequently, in turn increasing the number of cyber incidents. With nearly half of cybersecurity leaders predicted to change jobs by 2025 due to elevated levels of stress and burnout, already overworked cyber teams will become extremely overburdened as cyberattacks due to AI rise and talent shortages continue.

With data revealing 96% of organizations address supply chain security problems on an adhoc basis, but only half have a formalized DevOps supply chain security strategy in place, it's expected AI will only exasperate software supply chain attacks and draw light to these gaps in strategy. The increase in security incidents from AI will lead to more organizations eventually adopting infrastructure state management and continuous compliance. This is critical to meet security standards and address growing concerns. However, until these mitigations are in place, overburdened security teams will continue to face challenges regarding DevOps supply chain security.

Additionally, as generative AI outputs begin to decrease in accuracy the older the model gets, the more time employees will have to spend sifting through garbage outputs to find valuable insights. When employees are assured that the outputs are accurate and that generative AI increases productivity, they will begin to become overdependent on antiquated models and work off incorrect data. If the generative AI model employees rely on is outdated and requires new training data, it will take away from the promised productivity gains and organizational efficiency. Not only will employees have to sift through responses, but developers will now have to dedicate more time to system maintenance and finding new training data.

Stopping the Generative AI Spam Machine

Even though AI adoption is inevitable and has been for a long time, organizations must be prepared to deal with the implications of outdated training data when the day comes. To prevent generative AI spam, developer teams should focus on continual data upkeep, making incremental additions where applicable to ensure the system is trained on the most up-to-date information. As more of the workforce becomes educated and comfortable with AI, the learning curve will be less steep, and users will begin to understand what a correct vs incorrect output is.

Similarly, use cases for AI will begin to narrow and become more specific to streamline the actual benefits. Decision-makers will weed out areas where AI is not adding real value and focus efforts on the areas where AI can increase operational efficiency. As the technology becomes more advanced, it will be easier to identify where leveraging AI may not be the best fit. AI may not be ready to make higher level decisions or compute substantial amounts of code, but if organizations focus on the right use cases and ensuring data is up to date, this will one day be a potential benefit for leaders in all industries.

While individual organizations may not be able to limit potential cyber criminals from leveraging chatbots and learning from AI, they can provide the correct education for employees. Teaching employees other ways to spot spam vs real requests, as well as general AI education, will help combat the wave of cyberattacks to come. Eventually, regulations will be introduced to handle these issues, especially around AI spam bots on social media, but in the meantime, educating employees is critical for organizational security.

Don't Fall for the Generative AI Spam Machine Trap

AI holds tremendous opportunities to upend and improve our daily lives. Whether it is being used to help brainstorm content, provide quick insights on research topics, and more, the current state of AI does hold benefits for many. However, beyond basic assistance tasks, generative AI is not in the place where its promise has been fully realized.

When making investments in AI, be sure to evaluate the areas of the business that may not be equipped to handle integrating AI into their workflows. When leaders focus on the tried-and-true use cases that have shown proven benefits, they can devote time to innovating for the AI model of tomorrow. Like any new advent in history, AI will display both the good and bad, and it is up to leaders to protect their enterprise and navigate the issues of today.

Kapil Tandon is VP of Product Management at Perforce

Hot Topics

The Latest

Ask any senior SRE or platform engineer what keeps them up at night, and the answer probably isn't the monitoring tool — it's the data feeding it. The proliferation of APM, observability, and AIOps platforms has created a telemetry sprawl problem that most teams manage reactively rather than architect proactively. Metrics are going to one platform. Traces routed somewhere else. Logs duplicated across multiple backends because nobody wants to be caught without them when something breaks. Every redundant stream costs money ...

80% of respondents agree that the IT role is shifting from operators to orchestrators, according to the 2026 IT Trends Report: The Human Side of Autonomous IT from SolarWinds ...

40% of organizations deploying AI will implement dedicated AI observability tools by 2028 to monitor model performance, bias and outputs, according to Gartner ...

Until AI-powered engineering tools have live visibility of how code behaves at runtime, they cannot be trusted to autonomously ensure reliable systems, according to the State of AI-Powered Engineering Report 2026 report from Lightrun. The report reveals that a major volume of manual work is required when AI-generated code is deployed: 43% of AI-generated code requires manual debugging in production, even after passing QA or staging tests. Furthermore, an average of three manual redeploy cycles are required to verify a single AI-suggested code fix in production ...

Many organizations describe AI as strategic, but they do not manage it strategically. When AI plans are disconnected from strategy, detached from organizational learning, and protected from serious assumptions testing, the problem is no longer technical immaturity; it is a failure of management discipline ... Executives too often tell organizations to "use AI" before they define what AI is supposed to change. The problem deepens in organizations where strategy isn't well articulated in the first place ...

Across the enterprise technology landscape, a quiet crisis is playing out. Organizations have run hundreds, sometimes thousands, of generative AI pilots. Leadership has celebrated the proof of concept (POCs) ... Industry experience points to a sobering reality: only 5-10% of AI POCs that progress to the pilot stage successfully reach scaled production. The remaining 90% fail because the enterprise environment around them was never ready to absorb them, not the AI models ...

Today's modern systems are not what they once were. Organizations now rely on distributed systems, event-driven workflows, hybrid and multi-cloud environments and continuous delivery pipelines. While each adds flexibility, it also introduces new, often invisible failures. Development speed is no longer the primary bottleneck of innovation. Reliability is ...

Seeing is believing, or in this case, seeing is understanding, according to New Relic's 2025 Observability Forecast for Retail and eCommerce report. Retailers who want to provide exceptional customer experiences while improving IT operations efficiency are leaning on observability ... Here are five key takeaways from the report ...

Technology leaders across the federal landscape are facing, and will continue to face, an uphill battle when it comes to fortifying their digital environments against hostile and persistent threat actors. On one hand, they are being asked to push digital transformation ... On the other hand, they are facing the fiscal uncertainty of continuing resolutions (CR) and government shutdowns looming near and far. In the face of these challenges, CIOs, CTOs, and CISOs must figure out how to modernize legacy systems and infrastructure while doing more with less and still defending against external and internal threats ...

Reliability is no longer proven by uptime alone, according to the The SRE Report 2026 from LogicMonitor. In the AI era, it is experienced through speed, consistency, and user trust, and increasingly judged by business impact. As digital services grow more complex and AI systems move into production, traditional monitoring approaches are struggling to keep pace, increasing the need for AI-first observability that spans applications, infrastructure, and the Internet ...

Curbing the Generative AI Spam Machine

Kapil Tandon
Perforce

Efficiency. It's the latest buzz word that enterprise leaders everywhere are focused on this year, driven mainly by the promise of generative AI to help reduce the amount of time spent on the mundane and increase the workforce's productivity. It's these promised benefits that have driven increased interest in AI investment, and while technology possesses the opportunity to transform the way we work, just like anything else in history, it also has its shortcomings as well.

. The more dependent the workforce becomes on generative AI, despite its inaccuracies, the more mistakes will be made that can end up being costly. Until AI advances and its accuracy increases, all the AI investments organizations are currently investing in may not pay off after all.

What Does Generative AI Spam Look Like?

When people hear the word spam, they're typically thinking of mass marketing emails or unsolicited sales messages. However, in the case of generative AI in the enterprise, the spam it can produce looks a little different.

One tactic generative AI will be used for is to increase the number of scams shared through spam bots which can be leveraged for things like phishing emails. With AI tools available to the average citizen, any person regardless of their technical ability can now use AI to craft human-like messages and research better tactics for cyberattacks.

Typically, one method recipients can use to identify a spam email that may be a phishing scam is through incorrect grammar and spelling, but with generative AI chatbots like ChatGPT, bad actors can now leverage these tools to make their emails appear more legitimate. These actors can prompt ChatGPT or other large language models (LLMs) to draft emails with concise grammar and correct spelling, making it indistinguishable for consumers.

Another form of spam that will be produced is that overwhelming amount of data that end-users will flood generative AI models with. As more prompts are put in, and training data remains outdated or stagnant, the more likely it is for systems to produce copious amounts of incorrect information. With both the machine and humans using it. inputting and outputting massive amounts of irrelevant, inaccurate data, the less efficient AI will be for organizational productivity.

How Will This Impact the Bottom Line?

The rise in spam will coincide with the decrease in productivity gains that decision-makers were predicting AI systems would bring, therefore decreasing the value of using these tools. The democratization of AI, while great for education and awareness around technology, will only result in everyday users accidentally engaging with spam bots or phishing emails more frequently, in turn increasing the number of cyber incidents. With nearly half of cybersecurity leaders predicted to change jobs by 2025 due to elevated levels of stress and burnout, already overworked cyber teams will become extremely overburdened as cyberattacks due to AI rise and talent shortages continue.

With data revealing 96% of organizations address supply chain security problems on an adhoc basis, but only half have a formalized DevOps supply chain security strategy in place, it's expected AI will only exasperate software supply chain attacks and draw light to these gaps in strategy. The increase in security incidents from AI will lead to more organizations eventually adopting infrastructure state management and continuous compliance. This is critical to meet security standards and address growing concerns. However, until these mitigations are in place, overburdened security teams will continue to face challenges regarding DevOps supply chain security.

Additionally, as generative AI outputs begin to decrease in accuracy the older the model gets, the more time employees will have to spend sifting through garbage outputs to find valuable insights. When employees are assured that the outputs are accurate and that generative AI increases productivity, they will begin to become overdependent on antiquated models and work off incorrect data. If the generative AI model employees rely on is outdated and requires new training data, it will take away from the promised productivity gains and organizational efficiency. Not only will employees have to sift through responses, but developers will now have to dedicate more time to system maintenance and finding new training data.

Stopping the Generative AI Spam Machine

Even though AI adoption is inevitable and has been for a long time, organizations must be prepared to deal with the implications of outdated training data when the day comes. To prevent generative AI spam, developer teams should focus on continual data upkeep, making incremental additions where applicable to ensure the system is trained on the most up-to-date information. As more of the workforce becomes educated and comfortable with AI, the learning curve will be less steep, and users will begin to understand what a correct vs incorrect output is.

Similarly, use cases for AI will begin to narrow and become more specific to streamline the actual benefits. Decision-makers will weed out areas where AI is not adding real value and focus efforts on the areas where AI can increase operational efficiency. As the technology becomes more advanced, it will be easier to identify where leveraging AI may not be the best fit. AI may not be ready to make higher level decisions or compute substantial amounts of code, but if organizations focus on the right use cases and ensuring data is up to date, this will one day be a potential benefit for leaders in all industries.

While individual organizations may not be able to limit potential cyber criminals from leveraging chatbots and learning from AI, they can provide the correct education for employees. Teaching employees other ways to spot spam vs real requests, as well as general AI education, will help combat the wave of cyberattacks to come. Eventually, regulations will be introduced to handle these issues, especially around AI spam bots on social media, but in the meantime, educating employees is critical for organizational security.

Don't Fall for the Generative AI Spam Machine Trap

AI holds tremendous opportunities to upend and improve our daily lives. Whether it is being used to help brainstorm content, provide quick insights on research topics, and more, the current state of AI does hold benefits for many. However, beyond basic assistance tasks, generative AI is not in the place where its promise has been fully realized.

When making investments in AI, be sure to evaluate the areas of the business that may not be equipped to handle integrating AI into their workflows. When leaders focus on the tried-and-true use cases that have shown proven benefits, they can devote time to innovating for the AI model of tomorrow. Like any new advent in history, AI will display both the good and bad, and it is up to leaders to protect their enterprise and navigate the issues of today.

Kapil Tandon is VP of Product Management at Perforce

Hot Topics

The Latest

Ask any senior SRE or platform engineer what keeps them up at night, and the answer probably isn't the monitoring tool — it's the data feeding it. The proliferation of APM, observability, and AIOps platforms has created a telemetry sprawl problem that most teams manage reactively rather than architect proactively. Metrics are going to one platform. Traces routed somewhere else. Logs duplicated across multiple backends because nobody wants to be caught without them when something breaks. Every redundant stream costs money ...

80% of respondents agree that the IT role is shifting from operators to orchestrators, according to the 2026 IT Trends Report: The Human Side of Autonomous IT from SolarWinds ...

40% of organizations deploying AI will implement dedicated AI observability tools by 2028 to monitor model performance, bias and outputs, according to Gartner ...

Until AI-powered engineering tools have live visibility of how code behaves at runtime, they cannot be trusted to autonomously ensure reliable systems, according to the State of AI-Powered Engineering Report 2026 report from Lightrun. The report reveals that a major volume of manual work is required when AI-generated code is deployed: 43% of AI-generated code requires manual debugging in production, even after passing QA or staging tests. Furthermore, an average of three manual redeploy cycles are required to verify a single AI-suggested code fix in production ...

Many organizations describe AI as strategic, but they do not manage it strategically. When AI plans are disconnected from strategy, detached from organizational learning, and protected from serious assumptions testing, the problem is no longer technical immaturity; it is a failure of management discipline ... Executives too often tell organizations to "use AI" before they define what AI is supposed to change. The problem deepens in organizations where strategy isn't well articulated in the first place ...

Across the enterprise technology landscape, a quiet crisis is playing out. Organizations have run hundreds, sometimes thousands, of generative AI pilots. Leadership has celebrated the proof of concept (POCs) ... Industry experience points to a sobering reality: only 5-10% of AI POCs that progress to the pilot stage successfully reach scaled production. The remaining 90% fail because the enterprise environment around them was never ready to absorb them, not the AI models ...

Today's modern systems are not what they once were. Organizations now rely on distributed systems, event-driven workflows, hybrid and multi-cloud environments and continuous delivery pipelines. While each adds flexibility, it also introduces new, often invisible failures. Development speed is no longer the primary bottleneck of innovation. Reliability is ...

Seeing is believing, or in this case, seeing is understanding, according to New Relic's 2025 Observability Forecast for Retail and eCommerce report. Retailers who want to provide exceptional customer experiences while improving IT operations efficiency are leaning on observability ... Here are five key takeaways from the report ...

Technology leaders across the federal landscape are facing, and will continue to face, an uphill battle when it comes to fortifying their digital environments against hostile and persistent threat actors. On one hand, they are being asked to push digital transformation ... On the other hand, they are facing the fiscal uncertainty of continuing resolutions (CR) and government shutdowns looming near and far. In the face of these challenges, CIOs, CTOs, and CISOs must figure out how to modernize legacy systems and infrastructure while doing more with less and still defending against external and internal threats ...

Reliability is no longer proven by uptime alone, according to the The SRE Report 2026 from LogicMonitor. In the AI era, it is experienced through speed, consistency, and user trust, and increasingly judged by business impact. As digital services grow more complex and AI systems move into production, traditional monitoring approaches are struggling to keep pace, increasing the need for AI-first observability that spans applications, infrastructure, and the Internet ...