New Loggly Derived Fields Automatically Inject Intelligence and Structure into Any Log
June 16, 2015
Share this

Loggly announced Loggly Derived Fields, a new capability within Loggly Pro and Enterprise that allows users to specify custom parsing rules that generate derived fields as metadata during the ingestion process.

The derived fields provide intelligence and structure that is then used by Loggly Dynamic Field Explorer; to automatically catalog and summarize logs for one-click navigation and analysis. With derived fields and Field Explorer, development and DevOps teams can troubleshoot problems faster, more effectively monitor their log data for emerging issues, and do sanity checks after code releases without relying on cumbersome, manual queries.

A significant proportion of logs in organizations today are unstructured; the data is not separated into discrete data elements that allow for focused searching or aggregate metrics. Unstructured logs create considerable manual effort for DevOps professionals who traditionally have had to create one-off regular expressions and custom analyses. Many log management solutions require users to develop queries from scratch for each analysis, which is time consuming and inhibits sharing with less technical team members. Derived fields are different because once users create their rules, Loggly automatically applies those rules to all log data it receives, for all users of the account. Any user is then able to use Field Explorer to navigate by field names and values and to create sophisticated analyses without doing any additional work.

Derived fields expand the functionality of the flagship interface of Loggly, Loggly Dynamic Field Explorer. While most traditional log management solutions are designed with the search box as the primary interface, Field Explorer generates navigable lists and summaries to deliver a guided search experience. With Field Explorer, users get a bird’s-eye view of what’s happening, can quickly spot anomalies, understand the magnitude of issues, and search their logs with greater precision. Loggly Derived Fields extend the capabilities of Field Explorer to virtually any log type, structured or unstructured.

Derived fields are metadata, allowing for the original log data to stay intact so that logs can be analyzed in multiple ways. A user may create as many derived fields as needed from the same log events. Derived fields and Dynamic Field Explorer together help DevOps and IT professionals identify and resolve systems issues faster through more flexible and efficient log analytics.

“Unstructured logs are a reality for most cloud-centric companies, yet being able to efficiently mine those logs is critical for troubleshooting and delivering high application responsiveness to the business and end customers,” says Hector Angulo, Head of Product at Loggly. “Derived fields are another step in our unique ‘summarize first’ strategy. Customers can now gain insight even faster and spend more of their time solving or preventing IT issues rather than finding them.”

Loggly Derived Fields are currently in beta and will be generally available by the end of June 2015. Derived fields are a feature of the Loggly Pro and Enterprise subscription plans.

Share this

The Latest

April 27, 2017

According to the tenth annual State of the Network study from Viavi, nearly 90 percent of respondents say they are involved in troubleshooting security-related issues, with 80 percent reporting the time consumed by these issues has increased in the past year ...

April 25, 2017

While the idea of shifting toward digital business was speculative for most CEOs a few years ago, it has become a reality for many in 2017. 47 percent of CEOs are being challenged by the board of directors to make progress in digital business, and 56 percent said that their digital improvements have already improved profits ...

April 24, 2017

The Internet of Things (IoT) is increasingly present in our daily lives, at work, in the home and in the public sphere, making the world a more connected place. In fact, 2020 will see at least 20 billion connected devices across the globe. So, let's take a look at the most common iterations of the IoT at the moment, and what we can expect to see in the IoT landscape over the next 5 years ...

April 21, 2017

In the spirit of Earth Day, which is Saturday, April 22, we recently asked IT professionals for the tips and tricks they're using to help keep their data centers as green as possible. Here are a few ideas inspired by the responses we got ...

April 20, 2017

Almost One-Third (28 percent) of IT workers surveyed fear that cloud adoption is putting their job at risk, according to a survey conducted by ScienceLogic ...

April 19, 2017

A majority of senior IT leaders and decision-making managers of large companies surveyed around the world indicate their organizations have yet to fully embrace the aspects of IT Transformation needed to remain competitive, according to a new study conducted by Enterprise Strategy Group (ESG) ...

April 18, 2017

The move to cloud-based solutions like Office 365, Google Apps and others is one of the biggest fundamental changes IT professionals will undertake in the history of computing. The cost savings and productivity enhancements available to organizations are huge. But these savings and benefits can't be reaped without careful planning, network assessment, change management and continuous monitoring. Read on for things that you shouldn't do with your network in preparation for a move to one of these cloud providers ...

April 17, 2017

One of the most ubiquitous words in the development and DevOps vocabularies is "Agile." It is that shining, valued, and sometimes elusive goal that all enterprises strive for. But how do you get there? How does your organization become truly Agile? With these questions in mind, DEVOPSdigest asked experts across the industry — including analysts, consultants and vendors — for their opinions on the best way for a development or DevOps team to become more Agile ...

April 12, 2017

Is composable infrastructure the right choice for your IT environment? The following are 5 key questions that can help you begin to explore the capabilities of composable infrastructure and its applicability within your own IT environment ...

April 11, 2017

What is composable infrastructure, and is it the right choice for your IT environment? That's the question on many CIOs' minds today as they work to position their organizations as "digitally driven," delivering better, deeper, faster user experiences and a more agile response to change in whatever vertical market you do business in today ...