Gen AI Is Improving But Still Experiencing Growing Pains
June 05, 2024

Rob Mason

Share this

Generative Artificial Intelligence (GenAI) is continuing to see massive adoption and expanding use cases, despite some ongoing concerns related to bias and performance. This is clear from the results of Applause's 2024 GenAI Survey, which examined how digital quality professionals use and experience GenAI technology. The survey collected input from more than 6,300 people, including consumers, software developers and QA testers. Here's what we found.

GenAI Is Still Seeing Growth and Improvement

A vast majority of respondents (75%) said that GenAI chatbots are getting better at managing toxic and inaccurate responses, which have long been a major concern for the technology. Additionally:

■ 91% of respondents are using GenAI for research, and 33% do so daily.

■ 81% of respondents have used GenAI chatbots for answering basic search queries in place of traditional search engines, and 32% of respondents do so daily.

■ Of respondents using GenAI for software development and testing, 51% use it for debugging code, 48% use it for test reporting, 46% use it for building test cases, and 42% use it for building applications.

The number of software developers leveraging GenAI, including those who use it on a daily basis for their work, has seen a big uptick from last year, where only 59% said their workplace even allowed for GenAI use. As GenAI has gone more mainstream, it's become more widely accepted and used in the workplace, while the performance of responses has improved.

GenAI Growing Pains

There is still plenty of room for improvement as concerns around GenAI persists. Half (50%) of respondents are still experiencing bias and 38% have seen inaccurate responses. Additionally:

■ Only 19% of users said the GenAI chatbot they used understood their prompt and gave a helpful response every time.

■ 89% of respondents are concerned about providing private information to GenAI chatbots, and 11% said they never would.

Even as performance improves and GenAI is used more widely and frequently, concerns and issues still remain over inaccurate responses, system bias and data privacy.

Additional Key Findings

As more GenAI applications are developed for users, ChatGPT is still leading the field in terms of popularity, with 91% of respondents using it. Meanwhile, 63% of respondents use Gemini, and 55% use Microsoft Copilot. Other chatbots listed ranked as follows:

■ 32% of respondents use Grok

■ 29% of respondents use Pi

■ 24% of respondents use Perplexity

■ 23% of respondents use Claude

■ 21% of respondents use Poe

Additionally, 38% of respondents shared that they use different chatbots for different tasks, and 27% have replaced one GenAI chatbot with another due to performance. Use cases are also expanding for GenAI as 61% of respondents said that multimedia is essential for a large portion of their GenAI usage.

GenAI Remains On the Rise

Despite ongoing concerns, users clearly see the potential in GenAI. Everyone from consumers to developers are using more GenAI apps for more tasks more often. To unlock even greater potential value for users, companies developing GenAI applications must take model training and testing seriously. In particular, they must include real users in testing to identify issues and subtleties in meaning that only humans can gauge.

One specific approach that can help fine-tune and improve GenAI responses is red teaming, a practice with origins in cybersecurity. A so-called red team of testers work to identify biased or inaccurate responses so they know where the model still needs improvement. The more diverse the red team, the better companies can mitigate biases toward or against different communities.

Rob Mason is CTO of Applause
Share this

The Latest

June 18, 2024

With the rise of digital transformation and the increasing reliance on applications for business operations, the need for application performance management (APM) has become more critical ... This blog explains what APM is all about, its significance and key features ...

June 17, 2024

Generative AI (GenAI) has captured significant attention by redefining content creation and automation processes. Despite this surge in GenAI's popularity, it's crucial to highlight the continuous, vital role of machine learning (ML) in underpinning crucial business functions. This era is not about GenAI replacing ML; rather, it's about these technologies collaborating to supercharge intelligent automation across industries ...

June 13, 2024

As organizations continue to navigate their digital transformation journeys, the need for efficient, secure, and scalable data movement strategies has never been more critical ... In an era when enterprise IT landscapes are continually evolving, the strategic movement of data has become a cornerstone of maintaining agility, competitive edge, and operational efficiency ...

June 12, 2024

In May, New Relic published the State of Observability for IT and Telecommunications Report to share insights, statistics, and analysis on the adoption and business value of observability for the IT and telecommunications industries. Here are five key takeaways from the report ...

June 11, 2024
Over the past decade, the pace of technological progress has reached unprecedented levels, where fads both quickly rise and shrink in popularity. From AI and composability to augmented reality and quantum computing, the toolkit of emerging technologies is continuing to expand, creating a complex set of opportunities and challenges for businesses to address. In order to keep pace with competitors, avoiding new models and ideas is not an option. It's critical for organizations to determine whether an idea has transformative properties or is just a flash in the pan — a challenge tackled in Endava's new 2024 Emerging Tech Unpacked Report ...
June 10, 2024

The rapidly evolving nature of the industry, particularly with the recent surge in generative AI, can catch firms off-guard, leaving them scrambling to adapt to new trends without the necessary funds ... This blog will discuss effective strategies for optimizing cloud expenses to free up funds for emerging AI technologies, ensuring companies can adapt and thrive without financial strain ...

June 06, 2024

Software developers are spending more than 57% of their time being dragged into "war rooms" to solve application performance issues, rather than investing their time developing new, cutting-edge software applications as part of their organization's innovation strategy, according to a new report from Cisco ...

June 05, 2024

Generative Artificial Intelligence (GenAI) is continuing to see massive adoption and expanding use cases, despite some ongoing concerns related to bias and performance. This is clear from the results of Applause's 2024 GenAI Survey, which examined how digital quality professionals use and experience GenAI technology ... Here's what we found ...

June 04, 2024

Many times customers want to know why their measured performance doesn't match the speed advertised (by the platform vendor, software vendor, network vendor, etc). Assuming the advertised speeds are (a) within the realm of physical possibility and obeys the laws of physics, and (b) are real achievable speeds and not "click-bait," there are at least ten reasons for being unable to achieve advertised speeds. In situations where customer expectations and measured performance don't align, use the following checklist to help determine the reason(s) why ...

June 03, 2024

With so many systems potentially impacting applications performance, it is critical to find ways to separate insights from data that is often white noise. When cross-functional teams have clear alignment on what KPIs matter to them and their users' experiences, they can implement tools and processes that best support them. In the end, there must be collective ownership ...