you can use refer link, incase you not using tryhackme and came to this blog randomly. get-5$-credit-tryhackme

Linux Logs Investigations

Logs are like maps that help explorers. In this room, we’ll learn to read these maps, called logs, which help people who take care of computers. We’ll learn about different kinds of logs, like how the system talks in /var/log/kern.log, what users do, and how to keep an eye on everything with tools like auditd, syslog, and journal. By the end, we’ll know how to find important clues and protect our computer systems.

Objectives

Objectives are really important. Sometimes, I skip reading them and just try to brute force whatever comes to mind, which is a bad way of doing things. Reading objectives helps us understand where to focus and what to get from the lesson.

• Learn about different types of logs on Linux systems.
• Perform forensic analysis to find malicious processes, services, and scripts.
• Hunt and stop malicious activities in a hands-on incident response scenario.

Connecting to machine & 1st Question.

We have started the VM inside the Tryhackme, while you can use your personal machine and Tryhackme VPN to access the room.

Q1.) I'm ready to learn about Linux logs!
Ans : NO, Answer needed. 

Types of logging.

In Linux administration, understanding logs is like reading your server’s language. Logs trace system activities, errors, and events, crucial for troubleshooting, auditing, and security. Mastering how to locate, parse, and analyze logs is essential.

• Kernel logs: Show system inner workings, like hardware events, driver operations, and system errors.
• User logs: Capture user interactions, including login attempts, command executions, and app activities.

You must be on the TryHackMe page now. I suggest you read about all the logging levels, as this will help you understand and help you report similar incidents in the future. Spending that 1 minute and 27 seconds will be worth it.

Kernel logging

little Brief : The kernel, the OS core, manages hardware, processes, and resources. Kernel logs, or messages, reveal the system’s internal workings. They track everything from hardware starts to driver crashes, helping diagnose and troubleshoot issues.

During Linux boot, the console shows system startup info. The kernel ring buffer logs messages to avoid data loss. This memory-only buffer has a fixed size, so older entries are overwritten by new ones.

View the ring buffer by running ‘dmesg’ as root or a privileged user.

kernel logs are stored in /var/log/kern.log , managed by the rsyslog or syslog service.

Q.2) Which type of logs provide messages related to hardware events and system errors?
Ans : Kernel

Q.3) What is the memory space used to store system messages?
Ans : Kernel ring buffer

Q.4) What is the default log level used to inform about non-imminent errors? 
Ans : WARNING

As usual , all answeres were within the few lines we read before and moving to next we get little harder questions now and need to explore through systems now , well which is fun too anyway.

Linux logs file overview

The /var/log directory stores essential system logs. Key files include:

• kern.log and dmesg: Track kernel messages and diagnose hardware failures.
• Use sudo dmesg -T | grep ’exploit’ to find rootkit traces.
• auth.log: Logs all authentication events. Check for SSH attacks with grep ‘Accepted password’ /var/log/auth.log and elevated commands with grep ‘sudo’ /var/log/auth.log.
• syslog: Captures various system messages, from cron jobs to kernel activities. Search for ‘CRON’ or ‘kernel’ entries to identify issues.
• btmp and wtmp: Log failed and successful login attempts, helping to detect brute-force attacks and unauthorized access.

Q.5) Which log file can be used to record failed login attempts only?
Ans : btmp

Overview of Syslog

User space logging captures log messages from applications, services, and user activities. Syslog is the standard system for centralized log management on Linux. Here’s a brief guide:

Syslog Overview

• syslogd: The original daemon with basic capabilities.
• syslog-ng: Adds features like content-based filtering and TCP transport.
• rsyslog: The latest daemon with high performance, TLS encryption, database support, and modularity.

Syslog Components

• Daemon: Handles logging (rsyslogd or syslogd).
• Configuration File: /etc/rsyslog.conf defines logging rules.
• Log Files: Stored in /var/log.

Syslog Configuration

The default configuration file /etc/rsyslog.d/50-default.conf controls logging behavior. Architecture Components

• Originator (Client): Sends syslog messages.

• Relay: Forwards and potentially modifies messages.

• Collector (Server): Stores, visualizes, and retrieves logs.

• Syslog Message Format (RFC 5424)

• PRI: Message priority (facility + severity).

• Header: Includes timestamp, hostname, app name, process ID, and message ID.

• MSG: The event information.

Severity Levels

• Details on severity levels can be found on the TryHackMe room page.

Q.6) What severity level keyword is used to indicate immediate action is needed in a syslog message?
Ans : alert

Q.7) What facility code is used for cron jobs?
Ans : 9

Using Journal and Journalctl for Logging

Journal and journalctl offer robust logging for modern Linux systems, capturing messages from the kernel, system services, and user applications. Here’s how to use them for monitoring, security, and anomaly investigation.

Understanding the Journal

The journal is a binary logging system used by systemd-based distributions. It provides structured, indexed logs for efficient searching and analysis, unlike traditional text-based logs.

By default, journal logs are volatile and erased on restart. To make logs persistent, modify /etc/systemd/journald.conf by setting the Storage parameter to persistent, then restart the systemd journal daemon.

Q.8) To configure the persistence of journal logs, which parameter has to be modified within the journald configuration file?
Ans : storage

Audit Rules

Auditd is a powerful tool for improving Linux system security. As the user-space component of the Linux Auditing System, the Audit Daemon (auditd) collects and logs data about file access, user logins, process execution, and more. It offers a customizable framework for detailed monitoring and analysis of security incidents. Let’s explore some of the tools and telemetry auditd provides.

• Defining Audit Rules

Audit rules specify which activities to log. These rules can be set in /etc/audit/audit.rules for persistence or added temporarily using the auditctl utility. For more on auditctl, refer to its man page. Here, we’ll cover key functionalities relevant to security.

Q.9) Which utility is used to search for auditd logs?
Ans: ausearch

Examining auth logs

• Location and Structure

Auth logs are found in /var/log/auth.log on most Linux distributions. They record authentication events like SSH logins and sudo attempts, including a timestamp, service, user, and event description.

• Analyzing Auth Logs

Use command-line tools like grep, awk, and tail to analyze auth logs. This helps detect unauthorized access and other security incidents. View the logs with cat, less, or tail. For recent entries, use:

 sudo tail /var/log/auth.log

Q.10) What command can be used to search logs related to a session opened for a user?
Ans : sudo grep -i "session opened" /var/log/auth.log

Analysing Application Logs

Understanding Apache2 Logs

Apache2, a popular web server, generates two main types of logs:

1. Access logs: Record client requests, including IP addresses, URLs, request methods, and response statuses.

2. Error logs: Capture server-generated error messages, like configuration errors and crashes.

Logs are typically found in /var/log/apache2/. Key files include access.log and error.log.

Viewing Apache2 Logs

To monitor logs in real-time, use the tail command:

tail -f /var/log/apache2/access.log

This displays the latest entries in the access log.

Q.11) Which folder contains Apache2 logs?
Ans : /var/log/apache2

Pratical (Linux Log Capstone)

Incident Investigation at Deer Inc.

Anna, the Incident Response lead at Deer Inc., discovered their staging server (Linux, Apache web server) was accessible from the Internet. After isolating it, she investigated potential compromises:

1. Initial Access: Logs revealed suspicious POST attempts and a reverse shell indicating unauthorized access.

2. File Modifications: Audit logs (auditd) indicated unauthorized changes in /var/www/html, potentially involving privilege escalation.

3. Persistence: A compromised script linked to a service (journalctl) showed persistence was established, and a new user was created.

4. User Creation: Auth logs (zgrep) confirmed the creation of a new user with elevated privileges (sudo group).

5. Resolution: Anna verified her findings, prepared a detailed incident report, and initiated server re-imaging to secure Deer Inc.’s assets.

Q.12) What is the IP address from which the application was exploited?
Ans : 10.10.190.69

Q.13) What file contains the reverse shell?
Ans : cmd.php

Q.14) At which port was the reverse shell running?
Ans : 5000

Q.15) What is the file name that was being executed with sudo privileges?
Ans : tests.sh

Q.16) What is the name of the user created using the service?
Ans : attacker

Q.17) Was the new account ever logged in to? y/n
Ans : n

Q.18) Yay! I have completed the room!
Hit submit and proceed.

Enjoyed reading? kewl, any feedbacks? mail me at [email protected]