Shoreline.io announced Shoreline’s open source solutions library, a collection of Op Packs that make it easier to diagnose and repair the most common infrastructure incidents in production cloud environments.

Launching with over 35 Op Packs freely available to the community, the solutions library addresses issues like JVM memory leaks, filling disks, rogue processes, and stuck Kubernetes pods, among others.

Published and provisioned as open source Terraform modules, each Op Pack contains everything necessary to solve a specific issue, including pre-defined metrics, alarms, actions, bots, scripts, and tests. With Shoreline’s Op Pack library, the community identifies what to monitor, what alarms to set, and what scripts to run to complete the repair. All Op Packs are completely configurable and allow cloud operations teams to decide whether to use full automation or an interactive Notebook for human-in-the-loop repair. Co-developed with Shoreline customers, the Op Packs available at launch are based on real world on-call experience at large enterprises, rapidly growing unicorns, and the largest hyperscalar production environments.

"We're all working in the same cloud environments, yet every company has to figure out on their own how to automate even commonplace issues, like filling disks or JVM memory leaks,” said Anurag Gupta, co-founder and CEO of Shoreline. “Companies can no longer afford to write their own runbooks or custom code automations from scratch. With Shoreline, every time someone in our community fixes a problem, everyone else benefits.”

The following Op Pack solutions are immediately available, and free to Shoreline customers. The solutions library will continue to grow each month as new Op Packs are added by the Shoreline community. With each additional Op Pack in use by a customer, time is freed up for engineers to focus on innovation, rather than repetitive, mundane tasks that are better handled through automation. Op Packs available at launch include:

■ Streamline Kubernetes Operations

- Kubernetes node retirement - Gracefully terminate nodes when marked for retirement by the cloud provider.

- Kubernetes pod out of memory (OOM) - Generate diagnostic information and restart pods that ran out of memory.

- Kubernetes pods stuck in terminating - Identify, safely drain, and restart stuck pods.

- Kubernetes pods restarting too often - Detect pod restart loops and capture diagnostics to identify the root cause.

- IP exhaustion - Clear away failed jobs or pods that are consuming too many IP addresses.

- Stuck Argo workflows - Argo makes declaratively managing workflows easy, but it can leave behind many stale pods after workflow execution that should be deleted.

■ Reduce Toil (on both VMs or Kubernetes)

- Disk resize / disk clean - Disk full incidents can lead to wide-spread outages and data loss that can damage customer experiences and lose revenue.

- Networking issues - Network related issues are often hard to diagnose, and can lead to a very bad experience for customers.

- Intermittent JVM issues - Capture diagnostic information for intermittent issues that are hard to reproduce and debug.

- Server drift - Restore uniformity when configuration files, databases, and data sources on your VMs and containers differ.

- Config drift - Ensure observed state matches desired state on your system configuration, e.g. Kubernetes yaml, Cloud config, etc.

- Memory exhaustion - Running out of memory rapidly degrades customer experience and must be pre-empted.

- Disk failures in kern.log - Detect when a disk has errors or has entirely failed by inspecting the OS’s kern.log. Automatically capture these events and kick off fixes such as recycling the VM.

- Network failures in kern.log - Detect when a network interface has errors or has entirely failed by inspecting the OS’s kern.log. Automatically capture these events and initiate fixes such as recycling the VM.

- Endpoints unreachable - Determine when there are no endpoints behind your Kubernetes service or these endpoints have become unreachable.

- Elastic sharding replica management - Determine when your elastic search clusters have too few replicas per shard, and automatically kick off healing.

- Log processing at the edge - Analyze log files on the box to identify issues that cause production incidents, and eliminate costs of centralized logging.

- Kafka data Processing Lag - Restart slow/broken consumers when systems are falling behind in processing messages through a queue.

- Kafka topic management - When the length of your Kafka topic is too long, applications may begin to break.

- Processes consuming too many resources - Determine if the system is using too much memory or CPU at the process level.

- Restart CoreDNS service - CoreDNS, the default Kubernetes DNS service, can degrade in performance with too many calls causing massive latency.

■ Avoid Major Outages

- Certificate rotation - Sooner or later every company gets bitten by expired certificates and when they do, it can cause a catastrophic outage.

- DNS lag - Trigger rolling restarts of the DNS servers when they are responding slowly and causing widespread system issues.

Companies around the world rely on Shoreline’s incident automation platform to resolve common incidents in production, broaden the team that can safely repair incidents, and perform live site debugging of new incidents. Pairing this Op Pack solutions content with the Shoreline platform accelerates time to value and increases ROI for Shoreline customers.