Curbing the Generative AI Spam Machine
September 26, 2023

Kapil Tandon
Perforce

Share this

Efficiency. It's the latest buzz word that enterprise leaders everywhere are focused on this year, driven mainly by the promise of generative AI to help reduce the amount of time spent on the mundane and increase the workforce's productivity. It's these promised benefits that have driven increased interest in AI investment, and while technology possesses the opportunity to transform the way we work, just like anything else in history, it also has its shortcomings as well.

. The more dependent the workforce becomes on generative AI, despite its inaccuracies, the more mistakes will be made that can end up being costly. Until AI advances and its accuracy increases, all the AI investments organizations are currently investing in may not pay off after all.

What Does Generative AI Spam Look Like?

When people hear the word spam, they're typically thinking of mass marketing emails or unsolicited sales messages. However, in the case of generative AI in the enterprise, the spam it can produce looks a little different.

One tactic generative AI will be used for is to increase the number of scams shared through spam bots which can be leveraged for things like phishing emails. With AI tools available to the average citizen, any person regardless of their technical ability can now use AI to craft human-like messages and research better tactics for cyberattacks.

Typically, one method recipients can use to identify a spam email that may be a phishing scam is through incorrect grammar and spelling, but with generative AI chatbots like ChatGPT, bad actors can now leverage these tools to make their emails appear more legitimate. These actors can prompt ChatGPT or other large language models (LLMs) to draft emails with concise grammar and correct spelling, making it indistinguishable for consumers.

Another form of spam that will be produced is that overwhelming amount of data that end-users will flood generative AI models with. As more prompts are put in, and training data remains outdated or stagnant, the more likely it is for systems to produce copious amounts of incorrect information. With both the machine and humans using it. inputting and outputting massive amounts of irrelevant, inaccurate data, the less efficient AI will be for organizational productivity.

How Will This Impact the Bottom Line?

The rise in spam will coincide with the decrease in productivity gains that decision-makers were predicting AI systems would bring, therefore decreasing the value of using these tools. The democratization of AI, while great for education and awareness around technology, will only result in everyday users accidentally engaging with spam bots or phishing emails more frequently, in turn increasing the number of cyber incidents. With nearly half of cybersecurity leaders predicted to change jobs by 2025 due to elevated levels of stress and burnout, already overworked cyber teams will become extremely overburdened as cyberattacks due to AI rise and talent shortages continue.

With data revealing 96% of organizations address supply chain security problems on an adhoc basis, but only half have a formalized DevOps supply chain security strategy in place, it's expected AI will only exasperate software supply chain attacks and draw light to these gaps in strategy. The increase in security incidents from AI will lead to more organizations eventually adopting infrastructure state management and continuous compliance. This is critical to meet security standards and address growing concerns. However, until these mitigations are in place, overburdened security teams will continue to face challenges regarding DevOps supply chain security.

Additionally, as generative AI outputs begin to decrease in accuracy the older the model gets, the more time employees will have to spend sifting through garbage outputs to find valuable insights. When employees are assured that the outputs are accurate and that generative AI increases productivity, they will begin to become overdependent on antiquated models and work off incorrect data. If the generative AI model employees rely on is outdated and requires new training data, it will take away from the promised productivity gains and organizational efficiency. Not only will employees have to sift through responses, but developers will now have to dedicate more time to system maintenance and finding new training data.

Stopping the Generative AI Spam Machine

Even though AI adoption is inevitable and has been for a long time, organizations must be prepared to deal with the implications of outdated training data when the day comes. To prevent generative AI spam, developer teams should focus on continual data upkeep, making incremental additions where applicable to ensure the system is trained on the most up-to-date information. As more of the workforce becomes educated and comfortable with AI, the learning curve will be less steep, and users will begin to understand what a correct vs incorrect output is.

Similarly, use cases for AI will begin to narrow and become more specific to streamline the actual benefits. Decision-makers will weed out areas where AI is not adding real value and focus efforts on the areas where AI can increase operational efficiency. As the technology becomes more advanced, it will be easier to identify where leveraging AI may not be the best fit. AI may not be ready to make higher level decisions or compute substantial amounts of code, but if organizations focus on the right use cases and ensuring data is up to date, this will one day be a potential benefit for leaders in all industries.

While individual organizations may not be able to limit potential cyber criminals from leveraging chatbots and learning from AI, they can provide the correct education for employees. Teaching employees other ways to spot spam vs real requests, as well as general AI education, will help combat the wave of cyberattacks to come. Eventually, regulations will be introduced to handle these issues, especially around AI spam bots on social media, but in the meantime, educating employees is critical for organizational security.

Don't Fall for the Generative AI Spam Machine Trap

AI holds tremendous opportunities to upend and improve our daily lives. Whether it is being used to help brainstorm content, provide quick insights on research topics, and more, the current state of AI does hold benefits for many. However, beyond basic assistance tasks, generative AI is not in the place where its promise has been fully realized.

When making investments in AI, be sure to evaluate the areas of the business that may not be equipped to handle integrating AI into their workflows. When leaders focus on the tried-and-true use cases that have shown proven benefits, they can devote time to innovating for the AI model of tomorrow. Like any new advent in history, AI will display both the good and bad, and it is up to leaders to protect their enterprise and navigate the issues of today.

Kapil Tandon is VP of Product Management at Perforce
Share this

The Latest

November 29, 2023
The past few years have presented numerous challenges for businesses: a pandemic, rising interest rates, supply chain disruptions, and geopolitical conflict that sent shockwaves across the global economy. But change may finally be on the horizon. According to a recent report by Endava ... a majority of executives confirmed they are feeling optimistic about the current business climate, and as a result, are forecasting larger IT budgets, increased technology funding and rollout, and prioritized innovation in the coming year ...
November 28, 2023

Incident management processes are not keeping pace with the demands of modern operations teams, failing to meet the needs of SREs as well as platform and ops teams. Results from the State of DevOps Automation and AI Survey, commissioned by Transposit, point to an incident management paradox. Despite nearly 60% of ITOps and DevOps professionals reporting they have a defined incident management process that's fully documented in one place and over 70% saying they have a level of automation that meets their needs, teams are unable to quickly resolve incidents ...

November 27, 2023

Today, in the world of enterprise technology, the challenges posed by legacy Virtual Desktop Infrastructure (VDI) systems have long been a source of concern for IT departments. In many instances, this promising solution has become an organizational burden, hindering progress, depleting resources, and taking a psychological and operational toll on employees ...

November 22, 2023

Within retail organizations across the world, IT teams will be bracing themselves for a hectic holiday season ... While this is an exciting opportunity for retailers to boost sales, it also intensifies severe risk. Any application performance slipup will cause consumers to turn their back on brands, possibly forever. Online shoppers will be completely unforgiving to any retailer who doesn't deliver a seamless digital experience ...

November 21, 2023

Black Friday is a time when consumers can cash in on some of the biggest deals retailers offer all year long ... Nearly two-thirds of consumers utilize a retailer's web and mobile app for holiday shopping, raising the stakes for competitors to provide the best online experience to retain customer loyalty. Perforce's 2023 Black Friday survey sheds light on consumers' expectations this time of year and how developers can properly prepare their applications for increased online traffic ...

November 20, 2023

This holiday shopping season, the stakes for online retailers couldn't be higher ... Even an hour or two of downtime for a digital storefront during this critical period can cost millions in lost revenue and has the potential to damage brand credibility. Savvy retailers are increasingly investing in observability to help ensure a seamless, omnichannel customer experience. Just ahead of the holiday season, New Relic released its State of Observability for Retail report, which offers insight and analysis on the adoption and business value of observability for the global retail/consumer industry ...

November 16, 2023

As organizations struggle to find and retain the talent they need to manage complex cloud implementations, many are leaning toward hybrid cloud as a solution ... While it's true that using the cloud is not a "one size fits all" proposition, it is clear that both large and small companies prefer a hybrid cloud model ...

November 15, 2023

In the same way a city is a sum of its districts and neighborhoods, complex IT systems are made of many components that continually interact. Observability requires a comprehensive and connected view of all aspects of the system, including even some that don't directly relate to its technological innards ...

November 14, 2023

Multicasting in this context refers to the process of directing data streams to two or more destinations. This might look like sending the same telemetry data to both an on-premises storage system and a cloud-based observability platform concurrently. The two principal benefits of this strategy are cost savings and service redundancy ...

November 13, 2023

In today's rapidly evolving business environment, Chief Information Officers (CIOs) and Chief Technology Officers (CTOs) are grappling with the challenge of regaining control over their IT roadmap. The constant evolution and introduction of new technology releases, combined with the pressure to deliver innovation on shrinking budgets, has added layers of complexity for executives who must transform the perception of the role of the IT leader from cost managers and maintainers to strategic enablers of growth and profitability ...