banner image

LogSail - making waves with AI

The market for logging tools is a pretty crowded space and new entrants need a distinctive proposition to make themselves stand out. LogSail is a startup from Charlotte, North Carolina and their product of the same name seeks to differentiate itself with a focus on AI and automation. The company’s CEO and founder, Stephen Collins took some time out of his hectic schedule to chat to us about the company and its product.

Stephen is probably one of the younger leaders in the observability industry, having progressed from an internship at Oracle to starting up in his own company in just a few years. At Oracle he worked on building distributed systems for automated application testing and had a lightbulb moment when he realised that the same design principles he used in those systems could be applied to processes such as log forwarding - which tend to be manually configured one at a time. Stephen realised there was potential for a product which could take the toil out of log configuration - and so LogSail was born.

The Fourth Dimension

The LogSail platform offers a cloud-based service which automates the process of log configuration. Its intelligent agents forward logs to the LogsailLogSail API and users can view and query logs via the web-based console. For any company in this space, optimising throughput via techniques such as filtering and compression is a must. As well as adopting these patterns LogSail also implements its own proprietary techniques to push the performance envelope. This includes taking advantage of MongoDB’s pluggable architecture to build out their own custom storage scheme. This is optimised for providing fast insertion and parallel search capabilities.

For those interested in diving deeper into implementation detail, each log collected by LogSail can be referenced in the storage tree structure using a four dimensional coordinate system. This results in a “logarithmic narrowing of the search pattern” and provides a foundation for horizontal scale-out and performant analytics. One of the key selling points of the system is using automation to take the technical complexity out of log management. Indeed, you can be up and running in minutes by following a few simple steps. At the same time, where customers do have more complex needs, they can have direct access to engineers who can provide technical assistance with onboarding. Affording this level of customer support is one area where smaller companies such as LogSail can achieve a competitive advantage over some of the bigger players.

NMYF – No More YAML Files

There are two fundamental parts to the application architecture. The first is a control plane where users define logging policies and logging sources. The second is AI-powered agents which autonomously identify log sources on the defined hosts. The agents use Reactive Machine learning to detect changes in local environments and then automatically re-set log configuration without any need for engineers to edit configuration files. This is a centrepiece of the LogSail vision - you will never need to edit another YAML file. The LogSail agents themselves periodically check for new logging sources or any updates to policies and will reconfigure themselves accordingly.

Defining policies and assigning them to hosts is a simple process which is managed in the LogSail web UI. Users can build queries to search for log events across multiple hosts, containers, apps, and clouds, and pull logs from Docker, systemd, and flat files. LogSail also integrates with Zeek, automatically discovering Zeek logs and forwarding them to its archival service with zero configuration. The UI also offers the ability to pause logging at the touch of a button. This is a really powerful feature. Often, developers and sys admins either have to log on to a host and update config files or update application settings and redeploy in order to switch logs on or off. As well as avoiding the toil and risk involved in these interventions, the capability of turning off the tap on unwanted log streams can also be a valuable cost saving tool.

On the subject of cost, this is also an area where the company is highly competitive. They say that customers switching over from other providers can make savings of up 40%. LogSail is not just about logs – it can also capture a wide range of system and networking metrics such as SSH, HTTP, and DNS traffic. This data can then be viewed alongside your logs in a consolidated platform. The discovery and autoconfiguration capabilities of LogSails make it a particularly good candidate for IoT and edge scenarios where potentially vast numbers of hosts may be in operation and new devices are continually being brought onstream or taken offline.

Full Steam Ahead

The company also has exciting plans for their next phase of development. Using a round of seed funding as a springboard, they are planning to rollout a major upgrade in 2024. This will use AI-powered analytics to sift through large volumes of log data to detect patterns which can be used for anomaly prediction and detection. They are also planning to launch Sevi, a LLM trained on individual customer source code, log data, deployments, and detected anomalies. Sevi will be able to provide level 1 support for DevOps and security teams investigating anomalous activity.

Whilst stories of huge bills for log storage often hit the headlines, Stephen observes that for many of the company's clients the swift remediation of incidents the is number one concern. The financial and reputational costs of production outages can be huge and can far outweigh storage and ingress costs - especially for companies big enough to negotiate discounts.

The vision of the company is a bold one – their goal is to “build a completely automated observability offer that responds to customer infrastructure/application changes and can perform root cause analysis automatically”. Even though they are a comparative fledgeling in the marketplace, they do have some big-hitters in their camp. Their list of advisors includes Kord Campbell – one of the founders of the highly respected Loggly platform (now part of SolarWinds), and Michael Cucchi – a VP at Sumo Logic. With the power of AI behind them and experienced hands at the helm, LogSail may have cause to be feeling buoyant.

You can find out more and even get started for free at the LogSail web site.

Comments on this Article

You need register and be logged in to post a comment