Operating Systems

Protection and Security

Every day, ~350,000 new malware appear. Ransomware paralyzes hospitals, steals billions. Vulnerabilities in processors (Spectre, Meltdown) allow reading others' memory. The operating system is the last line of defense between user data and the chaos of the internet. Understanding protection mechanisms is not an academic exercise, but a critical skill in the modern world.

  • **WannaCry (2017):** ransomware affected 200,000+ computers, including the NHS (UK healthcare). Surgeries were canceled, patients redirected to other hospitals. Damage: $4 billion. Cause: unpatched Windows security update.
  • **Spectre/Meltdown (2018):** hardware vulnerabilities in CPUs allowed any program to read kernel memory. Billions of devices affected. Patches reduced performance by 5-30%. Showed: even hardware is not safe.
  • **SolarWinds (2020):** backdoor in software update gave access to 18,000 organizations, including the US government, Microsoft, Intel. Attackers (APT29) were in networks for 9+ months undetected. Supply chain attack - new reality.

Цели урока

  • Know protection rings (ring 0 kernel, ring 3 user) and the syscall as a controlled transition
  • Distinguish DAC (Unix permissions), MAC (SELinux, AppArmor), and Linux capabilities
  • Separate authentication vs authorization; recognize password, MITM, replay, privilege escalation
  • Understand Spectre/Meltdown (2018): side-channel via CPU speculative execution
  • Apply the principle of least privilege, defense in depth, and seccomp-bpf

Protection Domains and Privilege Levels

**Protection Domains** - a fundamental isolation mechanism in operating systems. The idea is simple: different system components operate with different privilege levels. The OS kernel can do everything, a regular program - almost nothing.

The processor hardware blocks dangerous operations in user mode. An attempt to execute a privileged instruction from Ring 3 causes a **General Protection Fault** - the program will be killed by the OS.

What happens if a program tries to bypass protection?

Consider a browser that decides to directly read the memory of another process (e.g., a banking application). In user mode, this is impossible: the processor checks each memory access via the **Page Table**. Attempting to access another's page → **Segmentation Fault**. The program crashes, but the system remains secure.

**Modern processors have even more nuanced mechanisms:** - **Intel VT-x / AMD-V:** Ring -1 for hypervisors (virtualization) - **Intel SMM (System Management Mode):** Ring -2 for firmware (BIOS/UEFI) - **Intel ME (Management Engine):** Ring -3, operates independently of the OS, has network access even when the computer is "off" This creates threats: malware in Ring -2 is invisible to antivirus in Ring 0!

**Mode switching** - an expensive operation. Each system call requires: 1. Saving user mode registers 2. Switching CPU to kernel mode 3. Checking access rights 4. Performing the requested operation 5. Returning to user mode

Why does sudo require a password?

The `sudo` command runs a program with root privileges (Ring 0 equivalent in userspace). Without a password, any program could call `sudo rm -rf /` and destroy the system. The password is authentication that a **person** truly wants to elevate privileges, not a malicious script.

What happens if a program in user mode (Ring 3) tries to directly execute an IN/OUT instruction to read from disk?

Access Control

**Access Control** - a mechanism that decides: can a subject (user, process) perform an operation on an object (file, resource)? Two main models: **DAC (Discretionary)** and **MAC (Mandatory)**.

**DAC (Discretionary Access Control)** - the owner of the object decides who has access. Used in UNIX/Linux through **permissions** (rwx) and **ACLs**.

**Problem with DAC:** a user can accidentally (or intentionally) give access to anyone. A program running under a user's name inherits ALL their rights - if malware is downloaded, it can read all of the user's files.

Attack via setuid: how to gain root

**setuid bit** - a mechanism for temporary privilege escalation. A file with setuid runs as the owner, not the executing user. Vulnerable example: ```bash # /usr/bin/backup - program with setuid root -rwsr-xr-x 1 root root /usr/bin/backup # Inside backup.c: system("cp /home/user/file /backup/"); # Attack: $ export PATH="/tmp:$PATH" $ echo '#!/bin/sh' > /tmp/cp $ echo 'exec /bin/sh' >> /tmp/cp $ chmod +x /tmp/cp $ /usr/bin/backup # → /tmp/cp is called instead of /bin/cp # → Gain root shell! 💀 ``` **Protection:** never use `system()` in setuid programs, use full paths.

**MAC (Mandatory Access Control)** - a centralized security policy that the user CANNOT change. Examples: SELinux (Security-Enhanced Linux), AppArmor.

**Principle of Least Privilege:** A program should have only the rights necessary for its operation. No more. **Linux Capabilities** - an alternative to "all or nothing" (root vs user). Instead of full root, a process can be given only specific capabilities: - `CAP_NET_BIND_SERVICE` - bind to ports <1024 - `CAP_NET_RAW` - create raw sockets (ping) - `CAP_SYS_TIME` - change system time For example, a web server can listen on port 80 without full root access.

Privilege Escalation: real attack Dirty COW

**CVE-2016-5195 (Dirty COW)** - a vulnerability in the Linux kernel (2007-2016) that allowed a regular user to overwrite ANY file, including `/etc/passwd`. **Attack mechanism:** 1. Open a read-only file (e.g., `/etc/passwd`) via mmap 2. Run two threads: one reads, the other writes to the same address 3. Race condition in page copying (Copy-On-Write) → write hits the original! **Result:** add a user with uid=0 to `/etc/passwd` → root access. A patch was urgently released in all distributions. It shows: even kernel-level protection can have critical bugs.

What is the key difference between MAC (Mandatory Access Control) and DAC (Discretionary Access Control)?

Authentication and Authorization

**Authentication** - identity verification: who is the subject? **Authorization** - rights verification: what is the subject allowed to do? Often confused, but these are different stages.

**Three factors of authentication:** 1. **Knowledge factor** (password, PIN) 2. **Possession factor** (USB key, phone with OTP) 3. **Inherence factor** (fingerprint, Face ID) **Multi-Factor Authentication (MFA)** - a combination of at least two factors. Much harder to hack: if the password is stolen, a physical key is still needed.

How passwords are stored in Linux (/etc/shadow)

Passwords are NEVER stored in plain text. A **cryptographic hash** with **salt** (a random string) is used.

**Public Key Authentication (SSH)** - an alternative to passwords. Uses asymmetric cryptography: private key on the client, public key on the server.

**Biometric authentication - convenient but risky:** **Pros:** - Cannot be forgotten (fingerprint is always present) - Hard to fake (modern sensors detect fakes) **Cons:** - Cannot be changed! If a fingerprint is compromised (database leak) - it's forever - A court can compel a finger scan, but not password disclosure (5th Amendment in the USA) - Works while asleep (Touch ID can be fooled by placing a sleeping person's finger)

Brute Force Attack and Protection Against It

**Brute Force** - trying all possible passwords until a match is found. For a 4-digit PIN (0000-9999), a maximum of 10,000 attempts is needed.

**Password Managers solve the problem of weak passwords.** People use one password everywhere: if one database leaks, all accounts are compromised. PM generates unique complex passwords, stores them encrypted, requires one master password.

Why do modern OS store passwords with salt, not just hash them with SHA-512?

Security Threats and Protection

Operating systems are constantly under attack: malware, exploits, social engineering. The following sections examine real threats and protection mechanisms.

**Main classes of threats:** 1. **Malware:** viruses, worms, trojans, ransomware 2. **Exploits:** buffer overflow, race conditions, privilege escalation 3. **Social Engineering:** phishing, pretexting, user deception 4. **Side-channel attacks:** Spectre, Meltdown, timing attacks 5. **Physical access:** cold boot attack, DMA attacks via Thunderbolt

Buffer Overflow: a classic attack

**Buffer overflow** - writing data beyond the allocated memory boundary. Allows overwriting the function's return address and executing arbitrary code.

**Protection against buffer overflow:** 1. **Stack Canaries** - a random value between the buffer and return address. Checked before return: has it changed? If yes → abort(). 2. **ASLR (Address Space Layout Randomization)** - randomizes stack, heap, library addresses. The attacker doesn't know where to place shellcode. 3. **DEP/NX (Data Execution Prevention)** - memory pages are either writable or executable, but not both. Code cannot be executed from the stack.

Ransomware: encrypting data for ransom

**Ransomware** - malware that encrypts user files. The decryption key is only with the attackers. They demand ransom (usually in Bitcoin). **WannaCry mechanism (2017):** 1. Exploit EternalBlue (SMB vulnerability in Windows) 2. Network infection (worm spreads itself) 3. Generate AES-128 key for each file 4. Encrypt documents, photos, databases 5. RSA-2048 public key embedded in malware 6. AES keys are encrypted with RSA and sent to C&C server 7. Screen: "Files are encrypted. $300 in Bitcoin for decryption" **Damage:** $4 billion, 200,000+ computers in 150 countries. **Protection:** regular backups (offline!), security updates, network segmentation.

**Spectre and Meltdown (2018)** - hardware vulnerabilities in Intel/AMD/ARM CPUs. Use **speculative execution** of the processor to read protected memory.

**Protection against Spectre/Meltdown:** - **Processor microcode** (updates from Intel/AMD) - **KPTI (Kernel Page Table Isolation)** - separate page tables for kernel/user - **Retpoline** - replace indirect jumps with safe sequences - **Disable hyperthreading** (radical measure, performance loss)

**Principles of Secure Development:** 1. **Defense in Depth** - multi-layered protection (not one wall, but several) 2. **Least Privilege** - minimum rights (a process should not be root without necessity) 3. **Fail-Safe Defaults** - deny by default, explicitly allow what is needed 4. **Economy of Mechanism** - simplicity of code (fewer lines, fewer bugs) 5. **Complete Mediation** - check rights for every operation (do not cache decisions) 6. **Open Design** - security should not depend on the secrecy of the algorithm (Kerckhoffs's principle)

How sudo protects against exploits

`sudo` - a critical program that grants root access. A vulnerability in it = complete system compromise. Protection mechanisms: 1. **Minimal code in setuid part** (fewer lines = fewer bugs) 2. **Environment cleanup** (reset $PATH, $LD_PRELOAD - otherwise a malicious library can be injected) 3. **TTY check** (command run from terminal, not script) 4. **Timeouts** (15-minute password cache, then request again) 5. **Logging** (all sudo calls → syslog, can trace attack) 6. **Regular code audits** (Qualys regularly found CVEs in sudo → fixes within hours)

Antivirus reliably protects against all threats - just install and forget

Antivirus is one layer of protection (defense in depth), but not a panacea. Zero-day exploits, targeted attacks go unnoticed

Antivirus uses signatures (known malware patterns) and heuristics. New malware (zero-day) has no signature → not detected until database update. Targeted attacks (APT) use unique code for each victim. Relying solely on antivirus is dangerous. Multi-layered protection is needed: regular OS updates, network firewall, user education (not clicking on .exe in email), backups (protection against ransomware), network segmentation (limiting worm spread). Security is a process, not a product.

How does ASLR (Address Space Layout Randomization) hinder buffer overflow exploits?

Key Ideas

  • **Protection Rings separate privileges.** User mode (Ring 3) is isolated from kernel mode (Ring 0). The processor hardware blocks dangerous operations. System calls are the only legal path from user to kernel. This protects the kernel from programs.
  • **Access Control decides: who can do what.** DAC (rwx permissions) - the owner controls access. MAC (SELinux) - centralized policy, even root cannot bypass. Capabilities - an alternative to "all or nothing", gives the process only the necessary rights.
  • **Authentication ≠ Authorization.** The first verifies identity (password, SSH key, biometrics), the second - access rights (can UID read the file). MFA (multi-factor) is critical: one password is unreliable, a combination of factors is much harder to hack.
  • **Defense in Depth - multi-layered protection.** Stack canaries detect buffer overflow. ASLR randomizes addresses. DEP/NX forbids execution from data. Each layer complicates the attack. There is no "silver bullet" - only a combination of measures works.

Related Topics

OS security is related to many other areas - from cryptography to network protocols:

  • Virtual Memory — Page tables and MMU - the basis of process isolation. Each process sees its own address space, cannot read others
  • Processes and Threads — fork() creates isolated processes. Inheritance of rights (uid, gid) during exec(). setuid bit for temporary privilege escalation
  • File Systems — Permissions (rwx), ownership, ACLs - mechanisms for file access control. Journaling for protection against corruption
  • Cryptography — Hashes (SHA-512) for passwords, asymmetric encryption (RSA) for SSH, AES for disk encryption (LUKS, FileVault)

Вопросы для размышления

  • With full root access to a Linux system but SELinux enabled, can /etc/shadow be read without changing the policy? Why is MAC more effective than DAC against malware?
  • A new Spectre-like vulnerability is discovered in ARM processors. Billions of smartphones are vulnerable. The patch reduces performance by 20%. What is the trade-off: security vs performance?
  • Why is biometrics (Face ID, Touch ID) convenient but should not be the only authentication factor? In what scenarios is a password more reliable than a fingerprint?

Связанные уроки

  • os-08-virtual-memory — Memory isolation is the foundational OS security mechanism
  • os-02-processes — UID/GID and capabilities are tied to processes
  • os-12-virtualization — Virtualization strengthens isolation through the hypervisor boundary
  • net-44-zero-trust
Protection and Security

0

1

Sign In