Bauwee1, auteur op Elk Factory

LOG MANAGEMENT IN JUST 3 DAYS

Bauwee1 — Mon, 26 Feb 2024 14:27:23 +0000

LOG MANAGEMENT IN JUST 3 DAYS FOR a FLEMISH HOSPITAL GROUP

Hospitals face the challenge of not only acting swiftly but also ensuring the security and integrity of patient data. Managing and analyzing log data is crucial, and at a Flemish hospital group, we tackled this challenge using the Elastic Stack. The hospital group operates on various devices, each with its own set of challenges and requirements. The focus was on two essential categories: Windows-based systems and various network devices.

RAPID IMPLEMENTATION OF AN ELASTIC CLOUD

With Elastic Cloud, we set up a powerful cluster in just a few simple steps. The process starts by creating a cluster in Elastic Cloud, specifying desired configurations, the number of nodes, and available resources. Elastic Cloud takes care of the underlying infrastructure, allowing us to focus on the optimal use of Elasticsearch. The flexibility provided by Elastic Cloud is advantageous. We can easily scale up or down depending on our needs, whether adding extra nodes for more storage capacity or adjusting computing power for better performance. Elastic Cloud makes the process effortless and efficient.

FLEET SERVER

We quickly integrated the Fleet Server, a crucial part of our log management process, into Elastic Cloud. With Fleet Server, we can manage Elastic Agents at scale, with an intuitive interface that simplifies deployment, updates, and monitoring.

The implementation of Fleet Server in Elastic Cloud has given us the flexibility to centrally manage agents, regardless of their physical location. With just a few simple commands, we ensured that Elastic Agents operate synchronously and effectively, resulting in streamlined and optimized log data collection.

This Elastic Stack utilizes a thoughtful combination of Fleet Managed Elastic Agents and Logstash, creating an integrated and scalable approach to managing and analyzing log data within the hospital group. In a short period of only 3 days, we introduced an Elastic Stack implementation that not only embraces efficiency and speed but also seamlessly aligns with the unique needs of a hospital environment.

FLEET-MANAGED ELASTIC AGENTS

With Elastic Cloud Fleet, you don’t need to manually configure agents on each machine. Instead, Fleet provides a central interface for effortlessly deploying Elastic Agents across various machines, all with just a few simple commands.

The Fleet Server acts as the conductor, coordinating and managing Elastic Agents. It not only accelerates the deployment process but also makes it easy to apply configurations and monitor the status of agents. Fleet Server enables large-scale deployment, allowing agents to do their job without consuming valuable time and effort.

Elastic Agents act as digital watchdogs deployed to gather crucial information from different systems in the IT landscape of the Great Flemish Hospital Group.

The use of policies and Windows integration significantly reduces the need for extensive manual configurations. With this integration, default settings and configurations for Windows logs are automatically applied, allowing you to quickly and efficiently collect valuable log data without extensive manual adjustments. This streamlines the process and minimizes the time and effort required for manual configurations. Think of them as reliable monitors continuously monitoring logs and data for important insights. These agents are designed for efficiency, ensuring that all relevant information is securely captured.

LOGSTASH AS A DIGITAL TRANSFORMER

Logstash acts as the digital transformer of raw, unstructured data into an understandable format. Imagine it receiving raw data from various network devices and organizing it into a unified language. This purifies and structures the raw data, ready for further analysis. A local Logstash installation also reduces the load on the Elastic Cloud system by refining raw data before it moves to the next phase.

FLEET-MANAGED ELASTIC AGENTS AND LOGSTASH

Elastic Agents, Fleet Server, and Logstash work together as a well-oiled mechanism. Elastic Agents collect valuable information, Fleet Server centrally coordinates and manages these agents, and Logstash transforms raw data from network devices into understandable information. Together, they form a powerful solution that enables the Great Flemish Hospital Group to deal with log data quickly, securely, and effectively.

IN PRACTICE: LOGGING FOR WINDOWS SERVERS AND NETWORKING DEVICES

FOR WINDOWS-BASED SYSTEMS

To gain a comprehensive insight into the logs and performance data of Windows-based systems in this hospital group, we deployed Elastic Agents with Windows Integrations.

Deploying Elastic Agents with Windows Integrations was a crucial step in the log management process. With speed and efficiency, we implemented these agents to serve as valuable sources for capturing logs and performance data from Windows-based systems.

By deploying Elastic Agents with specific Windows Integrations, we ensured that no detail was lost. Whether it’s system logs, domain controller logs, or exchange server logs, these agents provide a holistic overview essential for thorough analysis and monitoring.

FOR NETWORK DEVICES

In managing log data from various network devices, Logstash plays a central role as the digital transformer.

To effectively direct logs from diverse network devices to Logstash, we set up syslog forwarders.

The logs from various network devices are first sent to Logstash for parsing and transformations, converting raw, unstructured data into a suitable format for analysis before forwarding it to Elastic Cloud. Logstash serves as the central point for log processing. Centralized management of log transformations in Logstash also reduces the load on Elastic Cloud, making it more efficiently utilized.

Additionally, the use of local Logstash reduces resource costs by minimizing the load on the Elastic Cloud Deployment. This allows Elastic Cloud to focus on indexing and searches, resulting in a cost-effective implementation for log management.

OUR STRATEGY

The heart of our strategy was to create an integrated environment for managing and analyzing log data. This solution enabled us to manage, transform, and analyze logs within a short timeframe.

The integrated Fleet Server formed the pivot in our log management process. With this powerful tool, we could efficiently manage Elastic Agents at scale. The straightforward implementation, updates, and monitoring via Fleet Server resulted in rapid and large-scale deployment of Elastic Agents, translating into significant time savings and reduced efforts.

Logstash acted as our digital transformer, where raw, unstructured data was converted into a structured format ready for in-depth analysis. The use of Logstash not only contributed to optimizing log transformation but also provided cost-efficiency by refining raw data before sending it to Elastic Cloud.

By uniting these components, we created an integrated, scalable solution for managing, transforming, and analyzing log data within a timeframe of just 3 days.

Het bericht LOG MANAGEMENT IN JUST 3 DAYS verscheen eerst op Elk Factory.

Getting the most out of Generative AI with Elastic.

Bauwee1 — Wed, 17 Jan 2024 08:15:39 +0000

Getting the most out of Generative AI with Elastic.

Generative AI, or Gen AI, is reshaping the technology landscape. Just like earlier breakthroughs in mainframe, cloud computing, and mobile did. While conversations often revolve around revenue growth and business objectives, the potential for Gen AI to benefit the public sector and private sector is profound.
Semantic search and Gen AI enable tailored experiences for customers and employees, generating economic value projected to exceed $240 billion.
Read on and explore how Gen AI, in conjunction with the Elastic Stack, can unlock transformative capabilities in the sectors of government, education, and technology.

Real-world Gen AI use cases

Gen AI innovations, particularly in semantic search, present a myriad of opportunities for the public and private sectors. From personalized responses for students, citizens, and customers to streamlining workflows for employees.

Read our other blogs about real-world use cases:
HelpdeskGPT & HrGPT

The challenge however lies in leveraging internal data to ensure relevance, accuracy, and security.

Your Data + Generative AI = Context-Rich Answers

The integration of Gen AI with private data is crucial for achieving mission value. While publicly available Gen AI applications are limited to internet data and prone to inaccuracies, the Elastic Stack platform ensures privacy-first Gen AI experiences. By prioritizing security, delivering hyper-relevant content, and reducing hallucinations, Elastic enables real-time, scalable, and secure Gen AI applications.

The Elastic Generative AI Value Proposition

To create business value with Gen AI, leveraging proprietary data is essential. Gen AI, using context windows, integrates with unstructured, structured, or semi-structured data to deliver highly relevant and contextual responses. ElasticSearch, trusted by over 50% of Fortune 500, ensures privacy-first Gen AI experiences.

Prioritizes Security and Confidentiality: ElasticSearch implements clearance-level access, removing private information.
Delivers Hyper-Relevant, Reliable Content: Ensures the most relevant content from proprietary data informs Gen AI responses.
Reduces GAI Hallucinations: Infrequent inaccuracies as ElasticSearch uses mission-specific information.
Lower Costs: By providing information most relevant to queries, ElasticSearch minimizes compute and storage resources.
Unified Platform for AI Apps: ElasticSearch offers an end-to-end platform for building and delivering AI search applications.
Real-time Guidance: The Elastic AI Assistant aids security teams in tasks like alert investigation, incident response, and query generation.
Maintains Security and Confidentiality: ElasticSearch recognizes and implements appropriate access, removing private information.

Conclusion

In the dynamic landscape of AI-driven search results, whether or not combined with a Gen AI’s question/answer solution, the synergy between ElasticSearch and Gen AI unlocks unprecedented capabilities.
Whether serving the public sector or reshaping experiences in the private sector, the collaboration between data and Generative AI is a powerful force driving innovation and efficiency. Elastic’s commitment to security, relevance, and cost-effectiveness positions it as a cornerstone in the transformative journey of Generative AI.

Het bericht Getting the most out of Generative AI with Elastic. verscheen eerst op Elk Factory.

Post ElasticON – Cybersecurity & AI

Bauwee1 — Thu, 07 Dec 2023 14:02:37 +0000

Post ElasticON: Cybersecurity & AI

Organizations face the daunting task of safeguarding their digital assets against a myriad of threats. Elastic Security emerges as a powerful solution, offering a modernized approach to security operations. At its core lies the Security Information and Event Management (SIEM) system, a key component empowering practitioners to detect and respond to threats proactively.

The Detection Challenge

Detecting threats is like finding a needle in a haystack, but what if the needle actively avoids detection? Distinguishing between normal and suspicious activities becomes crucial in this stage. Elastic Security tackles this challenge head-on, employing advanced techniques to discern anomalies and potential threats.

Empowering Analysts with Data Insights

Elastic plays a pivotal role in empowering analysts to navigate the complex landscape of security data. With the ability to ingest data from hundreds of sources, Elastic Security allows practitioners to search, analyze, correlate, and identify patterns across various infrastructure components and cloud environments. The integration of Artificial Intelligence (AI) and Machine Learning (ML) augments the process, providing insights that guide analysts in their decision-making.

The Secret Sauce: SIEM, Endpoint Security, and Cloud Security Combined

Elastic Security merges SIEM, security analytics, endpoint security, and cloud security into a unified solution. This integration forms the secret sauce, a potent weapon delivering comprehensive protection, investigation, and response capabilities to customers. Elastic Security Labs, with their proactive threat research, further enhances the platform by contributing valuable insights to the security community.

Entity Analytics: Understanding Holistic Behavior

Entity analytics complements event-based alerting by focusing on understanding the behavior and interactions of entities within an environment. This goes beyond traditional event-based analytics, providing a holistic view of entity behavior through machine learning and statistical analysis. This approach enables the proactive detection of insider threats, system compromises, and data exfiltration.

The Power of Automation

In a landscape where every moment counts, automation becomes a critical ally. Elastic Security incorporates automation to minimize the time between threat detection and investigation in cloud environments. With over 50 prebuilt machine learning jobs and use case packages, Elastic Security Labs significantly enhances the platform’s ability to automatically block threats.

Generative AI Revolutionizing Security Analytics

Elastic Security embraces the transformative power of generative AI, revolutionizing security analytics. This tool, powered by LLM (Large Language Models), allows users to bring in data from any source, facilitating proactive threat detection. Privacy concerns are addressed through data anonymization and role-based access control, ensuring secure usage of LLM.

AI and Analytics at Scale

Elastic’s extensive capabilities for AI and analytics extend to processing large datasets at scale. With machine learning-based anomaly detection, automatic correlations, log categorization, and interactive chat-based investigations, Elastic Security provides a comprehensive suite of tools. Over 100 Out-of-the-Box (OOTB) open ML models offer customization for diverse data and use cases.

Democratizing Access to Data and Analytics

The democratization of data and analytics is a hallmark of Elastic Security. The platform’s capabilities are not confined to data scientists alone; they are accessible to the entire organization. This inclusivity ensures that insights derived from AI and analytics are widely available, contributing to a more robust security posture.

In conclusion, Elastic Security stands as a transformative force in the realm of cybersecurity. By seamlessly integrating advanced technologies, proactive threat research, and a commitment to democratizing access to security insights, Elastic Security empowers organizations to stay one step ahead in the ongoing battle against cyber threats.

Het bericht Post ElasticON – Cybersecurity & AI verscheen eerst op Elk Factory.

Post ElasticON – Search & Generative AI

Bauwee1 — Thu, 07 Dec 2023 14:01:56 +0000

ELASTICSearch & Generative AI

The quest for searching and retrieving information has become an integral part of our daily lives. As technology advances, so do the expectations and requirements. This blog explores the transformative capabilities of Elasticsearch, a robust solution that not only meets traditional search needs but also paves the way for groundbreaking applications in the era of Artificial Intelligence (AI).

Traditional Search to Geo-Location Integration

The journey begins with understanding the foundational elements of search projects. Traditional requirements include facets search for result filtering and typeahead for suggestions during the search process. About a decade ago, the advent of the mobile era introduced new challenges, particularly in incorporating geo-location into search algorithms. Elastic, a robust solution, emerged to address these challenges, also offering capabilities such as synonym management, automatic language recognition, and in-depth statistics on search queries.

Elasticsearch in the Spotlight

Search powered by Elasticsearch has become ubiquitous. Even in applications that may not seem to have a search function, Elasticsearch is at work behind the scenes. Popular companies, including Netflix, Tinder, and Uber, rely on Elasticsearch as their search engine. The staggering numbers speak for themselves, with 4.28 trillion downloads and over 6 trillion daily search queries sent to the Elastic Cloud platform. Elasticsearch has become the ‘de facto’ standard for search engines, setting the benchmark in the industry.

Elasticsearch and AI

The capabilities of AI mark the beginning of a new era, and Elasticsearch stands out as a fundamental infrastructure for success in AI applications. Machine learning, including algorithms for image and speech recognition, has been in existence for some time. The emergence of Generative AI introduces new possibilities, such as image, music and text generation.

A closer look at the differences between traditional “keyword search” and “AI-powered search results and answers” reveal the transformative potential of the AI-powered elasticsearch. We highlight the improved results delivered by AI-powered searches, making it clear why industries are increasingly adopting this technology.

Note that the above results are obtained using out of the box elasticsearch functionalities. In the examples below the out the box elastic functionalities, including its security features, are combined with Generative AI to formulate the answers.

Elasticsearch and Gen AI at Cisco

The real-world success story of Elasticsearch and Generative AI at Cisco, known as “topic search,” underscores the transformative impact. With nearly 90% of service requests receiving immediate solutions, the integration of Elasticsearch and Gen AI enhances customer experience significantly.

AI-Powered Search in Diverse Domains

The advantages of AI-powered search extend to various domains, from telecom providers and energy companies streamlining helpdesk operations to e-commerce stores offering personalized recommendations based on user-uploaded images. For example, in the e-shop of a clothing store, a visitor could upload a photo of a celebrity, asking which similar clothing the respective store can offer. In the context of predictive maintenance, for instance, sensor data can be combined with customer reviews to identify where repairs are needed first.

Contemplate the challenges you face, and how Elasticsearch combined with AI could offer transformative solutions.

Challenges and Solutions

To identify the most relevant use cases for your organization, consider the data at your disposal. In a support or service desk context, making manuals not only fully text-searchable but also accessible with AI-powered search can be beneficial, enabling immediate responses to queries. Ticketing systems often contain valuable data about problems and solutions, facilitating quicker answers for users or customers. SLA documents, providing information on who to contact for specific issues, can be valuable in a support context. In an e-commerce setting, incorporating data such as previous interactions, purchases, and inventory levels into the search context can offer valuable insights.

The promises of AI come with their own set of challenges. Implementing AI often requires a complex tech stack, and training models is a time-consuming and costly process. Elastic addresses these challenges head-on, offering solutions that ensure data security, privacy, and effective implementation of AI models.

RAG Pattern and ESRE: Streamlining AI Integration

To combat hallucinations in AI, the Retrieval Augmentation Generation (RAG) pattern is introduced, an architecture designed to prevent LLMs (such as ChatGPT) from hallucinating. RAG achieves this by adding additional context to the data related to the question/answer pair.

For a business solution, the RAG architecture implies restricting generative AI to your business content derived from vectorized documents, images, audio, and video (as demonstrated with elastic data earlier). Implementing RAG requires a data store containing both data and context, a vector database, and a search engine. This gives rise to the perception that it might involve a complex tech stack.

This outlines the RAG architecture.

Elastic simplifies the process with ESRE, eliminating the need for a convoluted infrastructure. In contrast to other solutions, Elasticsearch stands out by providing out-of-the-box storage, a vector database, and a powerful search and relevance engine.
Importantly, within the GDPR framework, it’s worth noting that with the Elastic capabilities you can anonymize data before sending it to the GAI/LLM outside Elastic.

Elasticsearch Relevance Engine

The Elasticsearch Relevance Engine (ESRE) is highlighted as a result of years of research and development by Elastic. Developers gain access to a comprehensive set of tools for building AI-powered search applications, including both traditional and vector database-driven searches. The engine features RRF (reciprocal rank fusion) for hybrid ranking, offering the best of both worlds.

Ingredients for AI-Powered Search Experiences: Understanding Vectors

A closer look at the necessary ingredients for AI-powered search experiences involves understanding vectors. These multidimensional numerical representations of unstructured data (text, images, audio, and videos) form the backbone of effective AI-driven search.

ELSER

We introduce the concept of embeddings and vectors, highlighting Elastic’s out-of-the-box ELSER model. ELSER, now available for English and already trained by Elastic, provides excellent results (cfr the screenshots above). Other languages will follow soon and note that you can upload 3rd party models or your own models in elastic.

Elastic’s Versatility Beyond Vectors

Elastic’s open platform approach emphasizes its adaptability. Whether in the cloud, on-premises or in hybrid environments, Elastic goes beyond just being a vector database, offering numerous integrations to unlock various data sources and more.

Elastic not only has a vector database necessary for building Generative AI applications. Elastic can do much more! Elastic can be used, for instance, in the cloud, on-premises, and even in hybrid environments. It offers numerous integrations to easily unlock various data sources, and so on…

Navigating Search Architectures

We conclude by revisiting typical search architectures with Elastic Bm25 text search. It emphasizes the flexibility of Elasticsearch by allowing users to generate models both inside and outside Elastic, providing three distinct paths, all working seamlessly with the familiar Elasticsearch API.

Embarking on GEN AI with Elastic

For organizations looking to embark on their AI journey, we outline crucial steps.

Step 1, data consolidation. By bringing all your data together in one place, possibly with the help of the many out-of-the-box Elastic integrations, you can merge data from different data systems and services.

Step 2 is creating a secure data layer. By incorporating role-based access and security at field/level and document levels, only the right people have the right access to information. Elastic also includes comprehensive monitoring and audit capabilities to understand how people are using the data platform. This allows monitoring of platform performance and potential misuse.

Step 3 is the transition from textual searching to semantic and hybrid searching.

Step 4, now we are ready to apply generative AI to these domain-specific data. By integrating this data with a large language model, it becomes possible to interact with this data in a new way.

Elasticsearch, Innovation in Search & AI

Elasticsearch stands out as a pillar of innovation in the realm of search and AI. Its seamless integration with AI technologies opens new frontiers, promising a future where information retrieval is not just efficient but also intelligent and transformative.

Het bericht Post ElasticON – Search & Generative AI verscheen eerst op Elk Factory.

Post ElasticON – Observability & AI

Bauwee1 — Thu, 07 Dec 2023 14:01:22 +0000

Post ElasticON: Observability and AI

The guarantee of stellar performance has become a complex challenge. The journey from monolithic applications on a handful of servers to the era of cloud-native technologies has undeniably increased efficiency but concurrently introduced a labyrinth of complexities. The surge in observability tools, initially a response to the need for visibility into new technologies, has spiraled into a sprawl that poses new challenges.

The Evolution of Complexity

In the early days, when applications were monolithic and ran on a limited number of physical servers, manual processes and specialized tools sufficed for monitoring. However, the advent of cloud-native technologies ushered in greater efficiency alongside unprecedented complexity. A myriad of tools emerged to address the need for visibility, resulting in the sprawl of observability tools. We now find ourselves at another inflection point, grappling with an avalanche of data generated by a plethora of tools and technologies, fueled by the relentless growth of digitization.

Consider the monumental increase in applications, services, users, and customers over the past five years. This exponential growth has led to a data deluge, stored in siloed data stores across a distributed multi-cloud IT environment—a complexity never witnessed before. To tackle this challenge, teams now require AI-powered observability, not merely for unifying disparate data sources but for delivering actionable insights driven by the latest AI technologies, all at a petabyte scale.

Elastic Observability: Unifying Telemetry for Enhanced Efficiency

Enter Elastic Observability, a solution designed to unify all telemetry, whether business or operational. By ingesting high cardinality and high dimensionality data into a single scalable datastore powered by AI and analytics, Elastic breaks down silos, enabling context and correlation for faster root cause analysis. This full-stack observability solution provides interactive and context-aware insights, accelerating problem resolution and driving operational efficiency.

Harnessing the Power of AI: Elastic’s Extensive Capabilities

To consume and process large observability datasets at scale, Elastic offers extensive capabilities in AI and analytics. These include machine learning-based anomaly detection, automatic correlations for surfacing root causes of application errors and latency, and log categorization to streamline vast amounts of unstructured log data. Interactive chat-based investigations, driven by the Elasticsearch Relevance Engine (ESRE), empower users to make sense of complex data. With over 100 out-of-the-box open ML models, Elastic democratizes access to data and analytics across the entire organization.

Elastic Observability and Machine Learning: Navigating Complexity

The integration of Elastic Observability with machine learning introduces innovative approaches to managing the challenges posed by complex digital environments. The zero-configuration, built-in machine learning facilitates AI-driven anomaly detection and root cause analysis across all observability data. This significantly reduces Mean Time to Detection (MTTD) and Mean Time to Resolution (MTTR), working seamlessly across thousands of dimensions.

Out-of-the-box anomaly detection for Application Performance Monitoring (APM) services and infrastructure monitoring, combined with ML jobs analyzing telemetry, allows teams to create alerts for high-scoring anomalies swiftly. AI-powered log categorization and anomaly detection further enhance the ability to make sense of billions of logs, identifying patterns and detecting changes in frequency.

Elastic AI Assistant for Observability

To expedite incident management and root cause analysis, Elastic introduces an AI assistant. Powered by the Elasticsearch Relevance Engine and OpenAI, this assistant breaks down knowledge silos, allowing teams to interactively explore problems and execute remedies with generative AI. It generates context-aware, business-specific output based on proprietary data, such as knowledge bases and runbooks.

Shay’s Insights: The Power of Search and ESRE

Shay emphasizes the practicality and usefulness of Large Language Models (LLMs) in the context of Elastic Observability. The need for RAG (Relevance, Applicability, Grounding) is crucial, and Shay underscores why search technology, particularly Elastic, is the ideal solution. Elastic’s capabilities extend beyond being just a vector search engine—it excels in relevance, making it indispensable in the face of the data deluge.

Investments in Elastic’s Elasticsearch Relevance Engine (ESRE) further enhance its capabilities, combining vector search, broad search capabilities, and recent strides in context and relevance. Elastic emerges as not just the most downloaded or used search engine but also a formidable vector database.

Challenges and Opportunities

Parsing through an overwhelming amount of information and data is a challenge that organizations face, and it’s only getting more complex. Data is the lifeblood of businesses, offering the potential for smart decision-making and improved operations. Elastic Observability, with its AI-powered capabilities, open and extensible platform, and unified contextual visibility, presents a tremendous opportunity for organizations.

In conclusion, Elastic Observability is not just a tool; it’s a comprehensive solution to the complex challenges posed by the modern digital landscape. With its advanced AI capabilities, it empowers teams to navigate the intricacies of data, providing actionable insights and driving operational efficiency in an era where data abundance is both a challenge and an opportunity.

Het bericht Post ElasticON – Observability & AI verscheen eerst op Elk Factory.

Reducing the SecOps workload with an AI powered Cybersecurity Solution.

Bauwee1 — Thu, 30 Nov 2023 14:10:42 +0000

Reducing the SecOps workload with an AI-powered Cybersecurity Solution.

Setting up Elastic Security

Elk Factory has secured Formica using Elastic Security in the Elastic Cloud. With the Elastic Cloud, you can easily and intuitively set up an environment, configuring security as follows:

1. CREATE AGENT POLICIES

An agent policy was created in Elastic Fleet for each operating system (Windows – MacOS) that needed monitoring.

2. ADD INTEGRATIONS TO POLICIES

Various integrations were added to these policies to log specific components such as Windows event logs, network packets, etc.

Both policies for both Windows and MacOS had the “System,” “Network Packet Capture,” and “Endpoint Security” integrations added. An additional integration named “Windows” was added to the Windows policy to capture Windows event logs.

3. CONFIGURE INTEGRATIONS

All integrations were then configured to capture the necessary data. The endpoint protection integration was also configured as an antivirus, taking action to detect and block threats.

4. ROLL OUT AGENTS ON ENDPOINTS

After setting up the agent policies, Elastic Agents were deployed on Formica employees’ laptops. Once installed, the Elastic Agents sent data to Elastic Cloud, securing the laptops.

5. ENABLE RULES BASED ON MITRE ATT&CK

At the time of writing this article, Elastic had 1051 pre-made detection rules to support your SIEM. To determine which rules were relevant, the MITRE ATT&CK framework was used. Specific threat groups targeting the technology sector were identified, and rules for the attack techniques these groups use were activated. This way, alerts are generated when these attack methods occur on the laptops.

6. DASHBOARDS

Elastic Security includes numerous out-of-the-box dashboards that provide insights into your environment. These dashboards display alerts, the number of incoming logs, and an overview of the protected laptops.

Elastic Security in action

Recently, there was a critical alert about an endpoint showing a suspicious launch agent named ‘ksinstall.’

1. First actions

Upon noticing this alert, details were examined in a timeline. Filtering on the “ksinstall” process revealed extensive information about all actions taken by ksinstall on the host.

Initially, it was unclear whether this was malicious or not.

2. Using the AI assistant

To further and quickly investigate the situation, the decision was made to use the security AI assistant introduced by Elastic in June 2023. The AI assistant operates using the well-known GPT-4 from OpenAI. To use the AI assistant, the following setup is required:

1. OpenAI developer account

Start by obtaining an OpenAI developer account to generate an API key for setting up the connector in Elastic Cloud.

2. Credit card top-up

Perform a one-time top-up via a credit card to add credits or tokens to your account, granting access to the GPT-4 model. You can only generate your API key after this top-up.

3. Elastic Cloud Configuration

Once you have the API key, open the AI assistant in Elastic Cloud, and it will guide you in setting up the connector.

4. Elastic AI assistance

Vraag 1 – Alert Summary

For the first question, we used one of the pre-made Elastic prompts to provide a summary of the alert and what it could mean. This gives a quick and clear overview of what is happening.

Vraag 2 – Workflow Alert Investigation

Next, we asked for a workflow to investigate the alert.

This workflow includes KQL queries that can be directly copied into Elastic “discover” or added to the timeline.

We placed these in “discover” and then displayed the fields ‘process.name’ and ‘process.args’ to quickly see what this process did to the system.

From this, you can see that ksinstall attempted to install the file Keystone.tbz. This file is in a Google Chrome folder, and ksinstall was a child process of Google Chrome.

Vraag 3 – Process arguments clarification

To clarify this, we asked the AI assistant what Keystone.tbz is and whether it is normal for Google Chrome to do this.

You quickly get an answer that this is part of the Chrome update process.

Vraag 4 – Malicious examples

Furthermore, we asked for examples of things that are suspicious and require investigation.

While continuing to search in the discover, none of these four points came up. So, we asked the assistant a final question to conclude.

Vraag 5 – Wrapping up

This was the answer that came out, and thus, we can conclude that this alert is a false positive.

3. RESULTS

With the help of the AI assistant, the alert was quickly addressed. The total cost of the conversation with the AI assistant was $0.32.

CONCLUSION

The use of Elastic Security with the AI Assistant significantly increases the productivity and efficiency of security engineers and analysts for a minimal investment.

Elk Factory – Elastic Premier Partner

Elk Factory is the Elastic partner to implement the Elastic platform. We aim for a win-win! We look at how this platform can make your company benefit the best, in return we can enjoy another satisfied customer!

Get to know us and contact us without obligation.

[contact-form-7]

Het bericht Reducing the SecOps workload with an AI powered Cybersecurity Solution. verscheen eerst op Elk Factory.

Elastic AI Assistant for Observability

Bauwee1 — Wed, 11 Oct 2023 11:50:20 +0000

Elastic AI Assistant for Observability

Elastic has extended its Elastic AI Assistant, a generative AI powered by the Elasticsearch Relevance Engine (ESRE).
The AI Assistant, currently in technical preview for Observability, aims to redefine how Site Reliability Engineers (SREs) identify and resolve problems, eliminating manual data hunting across various silos.

By providing context-aware information, Elastic’s AI Assistant enhances the understanding of application errors, log messages, alert analysis, and code efficiency, offering support for SREs.

Key Features:

Interactive Chat Interface: Elastic AI Assistant facilitates cohesive communication for SREs, offering an interactive chat interface where users can chat and visualize relevant telemetry data in one place. This interface also integrates proprietary data and runbooks, providing additional context.
Access to Private Information: Users can share private data, such as runbooks, incident histories, and case data, with the AI Assistant. An inference processor, driven by the Elastic Learned Sparse Encoder, grants the Assistant access to the most pertinent data for answering questions and completing tasks.
Knowledge Expansion: The AI Assistant can continuously expand its knowledge base through user interactions. SREs can teach it about specific problems, enabling the Assistant to offer support for similar scenarios in the future. This includes composing outage reports, updating runbooks, and enhancing automated root cause analysis, ultimately expediting issue resolution.

How It Works:

Imagine you’re an SRE who receives an alert related to an exceeded log entry threshold. While Elastic Observability provides some insights, you need further analysis of the log spike. This is where the AI Assistant steps in. It sends a pre-built prompt to your configured Large Language Model (LLM), which not only provides a description and context of the issue but also offers recommendations on how to proceed. Additionally, you can initiate a chat with the AI Assistant to delve deeper into your investigation.

The AI Assistant’s chat interface supports natural language queries and enables you to:

Obtain conclusions and context, and receive recommendations from your private data (powered by Elastic Learned Sparse Encoder) and the connected LLM.
Analyze responses and output from the AI Assistant.
Summarize information throughout the conversation.
Generate Lens visualizations within the chat.
Execute Kibana® and Elasticsearch® APIs through the chat interface.
Perform root cause analysis using specific APM functions.

With Elastic’s AI Assistant, SREs can gain deeper insights into issues, understand their impact on the business, and leverage private data that LLMs aren’t trained on. This tool aims to streamline observability analysis, reduce manual data retrieval, and enhance AIOps capabilities.

In summary, Elastic’s AI Assistant for Observability is set to transform the way SREs operate, providing them with context-aware insights and efficient problem-solving capabilities. This extension of the Elastic AI Assistant is poised to play a pivotal role in the observability landscape, offering a powerful tool for SREs to enhance their workflow.

Het bericht Elastic AI Assistant for Observability verscheen eerst op Elk Factory.

ElasticON 2023

Bauwee1 — Thu, 05 Oct 2023 07:48:29 +0000

ElasticOn 2023 Amsterdam

ElasticON 2023 will take place on 21 November in Amsterdam. As a premier Elastic partner, Elk Factory will of course be present and happy to welcome you to our booth.

As the only exclusive brand for Elastic services, we have had a fantastic year since the launch – exactly 1 year ago!

We are your go-to for solutions related to Observability, Cybersecurity, and Enterprise Search, offering consultancy, managed services, side-by-side training, and Elastic Subscriptions.

Feel free to come and meet us!

You can find more information and free registration here

Post ElasticON

Maybe the move to Amsterdam or the specific date is not suitable for you. For these reasons, we are organizing together with Elastic the ‘Post ElasticON’ event in Kontich, Belgium on 13 December 2023!

What you can expect is a retrospective and summary of ElasticON 2023 Amsterdam, tech talks, ‘food & drinks’, and a comedy show* with Bert Gabriels.

You can find more information and free registration here

See you there!

Do you want to stay up-to-date with Elk Factory? Follow us on Linkedin!

Het bericht ElasticON 2023 verscheen eerst op Elk Factory.

Revolutionizing Cyber Security with Intelligent Automation

Bauwee1 — Tue, 03 Oct 2023 08:24:07 +0000

Revolutionizing Cyber Security with Intelligent Automation

With the introduction of the Elastic AI Assistant, the game is changing. This advanced tool integrates Artificial Intelligence, empowering cybersecurity professionals, and enabling them to tackle complex challenges with unprecedented efficiency and precision.

The Power of Elastic AI Assistant

Imagine having a virtual teammate, ready to assist you with just a click or a keyboard shortcut. Elastic AI Assistant, nestled within Elastic Security, provides an intuitive interface for your professionals to effortlessly access its capabilities.
One of its unique strengths lies in its ability to offer prebuilt, recommended prompts, ensuring that the AI-generated responses are tailored precisely for the user, be it a tier 1 or 2 security analyst.

The Magic Lies in Prompts and Context

At the heart of Elastic AI Assistant’s efficacy are its prebuilt prompts and context-driven approach. These prompts, carefully curated for various scenarios, facilitate tasks that are crucial for cybersecurity teams:

Alert Summarization: Instantly transforms an alert document into a detailed explanation of the triggering factors and provides recommended steps for effective triage and remediation. This dynamic runbook creation simplifies the response process during cyber attacks.
Workflow Suggestions: Offers step-by-step guides within Elastic, assisting users in tasks like adding an alert exception or creating custom dashboards. This feature streamlines operations, enhancing overall productivity.
Query Conversion: Simplifies migration from legacy SIEMs by converting queries from other products into Elastic queries. This process not only saves time but also significantly reduces migration costs, making the transition smoother and more cost-effective.
Agent Integration Advice: Provides guidance on the best methods for collecting information within Elastic, ensuring that users make informed decisions while gathering essential data.

Customization: Tailoring AI to Fit Your Workflow

Elastic AI Assistant goes a step further by allowing users to create their own prompts, tailoring the AI’s capabilities to align seamlessly with their unique workflows. This level of customization transforms Elastic AI Assistant from a tool into a trusted team member, adapting to the specific needs of each cybersecurity professional.

Seamless Collaboration and Integration

Elastic AI Assistant doesn’t just stop at providing answers; it fosters continuous conversation. As users interact with the model, it retains the context of the conversation. Once satisfied with the results, users can effortlessly integrate the insights into timeline investigations or cases, enhancing collaboration and knowledge sharing within the cybersecurity team.

Elevating Cybersecurity Efforts to New Heights

For cybersecurity, time is of the essence, and Elastic AI Assistant ensures that valuable time is spent on strategic decision-making rather than mundane tasks.
By harnessing the power of generative AI, Elastic AI Assistant stands as a beacon of innovation, transforming the way cybersecurity professionals operate. With its intuitive prompts, context-driven responses, and seamless customization, it’s not just an assistant; it’s a strategic partner, reinforcing the arsenal of cybersecurity experts in their ongoing battle against digital threats. Elastic AI Assistant represents a giant leap towards a safer, more secure digital future.

Cybersecurity Starter Pack

Protect your enterprise with the advanced cybersecurity solution of Elastic. Elk Factory, as premier Elastic partner offers a starter pack to bring your security to the next level. More info here.

Het bericht Revolutionizing Cyber Security with Intelligent Automation verscheen eerst op Elk Factory.

What is a Platform Business

Bauwee1 — Wed, 06 Sep 2023 10:01:13 +0000

What is a Platform Business?

A platform business is a unique economic model that serves as an intermediary, facilitating interactions and transactions between different groups of users. Instead of producing goods or services themselves, platform businesses create a space where producers and consumers can connect and exchange value. These platforms leverage technology, often in the form of digital or online marketplaces, to foster these interactions.

Key characteristics of platform businesses include their ability to bring together multiple user groups, often referred to as “sides” (e.g., buyers and sellers), and create a network effect where the platform becomes more valuable as more participants join. They are data-driven, using information about user behavior to enhance their services and provide tailored experiences.

Prominent examples include companies like Airbnb, which connects travelers with hosts, and Uber, which links riders with drivers. These platforms have disrupted traditional industries by leveraging technology to create efficient and scalable marketplaces, reshaping how people access and consume goods, services, and information.

We can distinguish a few different types of platforms:

Single-Sided Platform

A single-sided platform focuses on bringing together a single group of users or participants. These platforms enable interactions and transactions among members of the same user base. A classic example of a one-sided platform is a social media network like Facebook or Twitter, where users connect, share content and interact within the same community.

Elasticsearch can significantly enhance the performance of one-sided platforms by providing robust search capabilities and efficient data retrieval. For instance, in a social media network, Elasticsearch can power the search functionality, allowing users to find relevant posts, users, and hashtags quickly. Its real-time indexing and search capabilities ensure that users receive up-to-date and accurate search results, leading to a more engaging user experience.

Two-Sided Platform

Two-sided platforms, often also referred to as multi-sided platforms, cater to two distinct user groups that are interconnected by the platform’s services. These platforms create value by facilitating interactions between the two groups. A classic example of a two-sided platform is Uber, which connects drivers and riders, enabling ride-sharing transactions.

For two-sided platforms, Elasticsearch can play a crucial role in optimizing user experiences and operational efficiency. In the case of Uber, Elasticsearch is employed to enhance the rider and driver matching process. It can consider various factors such as location, availability, and user preferences to quickly identify suitable matches, resulting in reduced wait times and improved customer satisfaction. Furthermore Elastic will also manage the access levels for the different user levels and types. Empowering privacy and data security.

Multi-Sided Platform

The multi-sided platform is a broader term and can go beyond connecting only two user groups; they involve three or more distinct user segments, each contributing to the platform’s overall value proposition. These platforms often create a network effect, where the value of the platform increases exponentially with the number of participating segments. An example of a multi-sided platform is Airbnb, which connects hosts, travelers, and local service providers.

In multi-sided platforms like Airbnb, Elasticsearch’s capabilities can be harnessed to create more personalized and relevant recommendations for users across various segments. By analyzing user preferences, historical data, and location-based information, Elasticsearch can provide tailored suggestions for accommodations, experiences, and services. This level of personalization enhances user engagement and encourages cross-segment interactions.

user level/roles data access

Summary

Digital platforms will continue to shape industries and redefine business models, understanding the differences between one-sided, two-sided, and multi-sided platforms is essential for effective platform design and management.
Elasticsearch, with its advanced search and analytics capabilities, can significantly enhance the performance of these platforms by improving data retrieval, user matching, and personalized recommendations.
By harnessing the power of Elasticsearch, platform operators can create seamless and value-driven experiences for their users, leading to increased engagement, satisfaction, and ultimately, business success.

Het bericht What is a Platform Business verscheen eerst op Elk Factory.