From the free course Systems Programming in C and Unix Environments on Lykke
Introduction to Systems Programming and C
Key concepts: Systems Programming vs. Operating Systems · GCC Compilation Process · Object Lifetimes (Static, Stack, Heap) · C Type System and Casting · Undefined Behavior
An introduction to the role of systems programming, the GCC compilation process, and the fundamental type system of C.
Introduction to Systems Programming and C
Systems programming is the discipline of building software that provides services to other software. It is the "scaffolding" of the computing world, sitting directly above the hardware and below the application layer. While application programmers focus on user needs—UI, business logic, and data processing—systems programmers focus on resource management: memory allocation, process scheduling, file system integrity, and network protocols.
In this domain, the C programming language remains the industry standard. Developed at Bell Labs in the early 1970s alongside the Unix operating system, C provides a unique balance of high-level abstraction and low-level control. It allows a developer to manipulate individual bits and memory addresses while providing the structural constructs (functions, loops, types) necessary for complex software engineering.
Systems Programming vs. Operating Systems
It is a common misconception that systems programming is synonymous with operating system (OS) development. While an OS is the most prominent example of a system-level program, the field is much broader.
Definition: Systems programming involves developing software that interfaces with the hardware and provides a runtime environment for application software.
| Feature | Application Programming | Systems Programming |
|---|---|---|
| Primary Goal | Solving user problems (e.g., banking, social media). | Managing hardware and providing services. |
| Resource Access | Abstracted via APIs and virtual machines. | Direct access to memory, registers, and I/O. |
| Languages | Java, Python, JavaScript, Swift. | C, C++, Rust, Assembly. |
| Failure Impact | Application crash (isolated). | System instability or kernel panic. |
| Runtime | Managed (Garbage Collection, JIT). | Unmanaged (Manual memory, static linking). |
Systems programming includes the development of Language Runtimes (like the Java Virtual Machine or the Python interpreter), Compilers (like GCC or Clang), Database Engines, Web Servers, and Embedded Firmware.
The GCC Compilation Process
Unlike interpreted languages where code is executed line-by-line by a host program, C is a compiled language. The transformation from human-readable .c files to a machine-executable binary is a multi-stage pipeline managed by the GNU Compiler Collection (GCC).
1. Preprocessing (cpp)
The preprocessor handles directives starting with #. It performs text substitution, includes header files, and handles conditional compilation.
- Input:
source.c - Output:
source.i(Expanded source code)
2. Compilation (cc1)
The compiler translates the expanded source code into assembly language for a specific processor architecture (e.g., x86_64 or ARM). This is where syntax checking and optimization occur.
- Input:
source.i - Output:
source.s(Assembly code)
3. Assembly (as)
The assembler converts assembly instructions into object code—machine-level instructions (binary) that the CPU understands, but which are not yet executable because they lack addresses for external functions.
- Input:
source.s - Output:
source.o(Object file)
4. Linking (ld)
The linker combines multiple object files and libraries (like libc) into a single executable. It resolves symbols (function names and variables) to their final memory addresses.
- Input:
source.o,library.a - Output:
a.out(Executable)
Essential Compiler Flags
In a professional systems environment, we use flags to enforce strictness and aid debugging:
| Flag | Purpose |
|---|---|
-Wall |
Enables "all" common warning messages. |
-Werror |
Treats all warnings as errors, halting compilation. |
-g |
Includes debug symbols for use with gdb. |
-O2 |
Enables standard optimizations for performance. |
-fsanitize=address |
Instruments code to detect memory leaks and buffer overflows at runtime. |
Object Lifetimes: Static, Stack, and Heap
In C, an object is a region of data storage in the execution environment. Every object has a lifetime (or storage duration), which determines when the memory is allocated and when it is deallocated. Understanding these is the difference between a stable system and one riddled with "Heisenbugs."
Static Lifetime
Static objects are allocated when the program starts and persist until the program terminates. This includes global variables and variables declared with the static keyword.
- Location: Data segment (
.datafor initialized,.bssfor uninitialized). - Risk: Global state makes code harder to test and thread-unsafe.
Stack Lifetime (Automatic)
Stack objects are local variables within a function. They are "pushed" onto the stack when the function is called and "popped" (deallocated) when the function returns.
- Location: The Stack.
- Mechanics: Managed via the Stack Pointer (SP) and Frame Pointer (FP).
- Risk: Returning a pointer to a stack variable is a classic error, as that memory is reclaimed immediately upon function exit.
Heap Lifetime (Dynamic)
Heap objects are manually managed by the programmer using malloc(), calloc(), realloc(), and free().
- Location: The Heap.
- Mechanics: The
malloclibrary manages a pool of memory, requesting large chunks from the OS viasbrkormmap. - Risk: Memory leaks (forgetting to
free) and dangling pointers (using memory afterfree).
| Lifetime | Allocation | Deallocation | Scope |
|---|---|---|---|
| Static | Program Start | Program End | Global or File-local |
| Stack | Function Entry | Function Exit | Block-local |
| Heap | Manual (malloc) |
Manual (free) |
Pointer-based |
C Type System and Pointer Arithmetic
The C type system is static (types are checked at compile time) but weak (the language allows bypassing the type system via casting). At the hardware level, everything is just bytes; C types provide a lens through which we interpret those bytes.
Pointers and Dereferencing
A pointer is simply a variable that stores a memory address.
&(Address-of): Returns the memory address of a variable.*(Dereference): Accesses the value stored at the address held by the pointer.
int x = 42;
int *p = &x; // p holds the address of x
printf("%d", *p); // prints 42
Pointer Arithmetic
Pointer arithmetic is scaled by the size of the underlying type. If p is an int* (assuming 4-byte ints), p + 1 increments the address by 4 bytes, not 1.
The Pointer-Array Identity: In most contexts, the name of an array decays into a pointer to its first element.
arr[i]is mathematically equivalent to*(arr + i).
Type Casting
C allows you to treat a region of memory as a different type using a cast.
- Safe Casting: Upcasting (e.g.,
inttolong). - Dangerous Casting: Casting between unrelated pointer types (e.g.,
int*tostruct user*). This is often necessary in systems programming for "generic" functions usingvoid*.
void *raw_memory = malloc(100);
int *int_array = (int *)raw_memory; // Treating raw bytes as integers
The C Preprocessor: Power and Peril
The preprocessor is a text-processing engine that runs before the actual compiler. While powerful, it lacks knowledge of C's scope or type rules.
Macros and Constants
#define is used for constants and "function-like" macros.
#define PI 3.14159
#define SQUARE(x) ((x) * (x))
Pitfall: Precedence. Without parentheses, SQUARE(1 + 1) expands to 1 + 1 * 1 + 1, which equals 3, not 4. Always wrap macro arguments and the total expression in parentheses.
Conditional Compilation
This allows the same source code to be compiled differently for different platforms (e.g., Linux vs. Windows).
#ifdef DEBUG
printf("Debug: x = %d\n", x);
#endif
Header Guards
To prevent a header file from being included multiple times (which causes redefinition errors), we use guards:
#ifndef MY_HEADER_H
#define MY_HEADER_H
// Declarations here
#endif
POSIX File I/O and Streams
In Unix-like systems, "everything is a file." This abstraction allows us to use the same system calls to read from a disk, a keyboard, or a network socket.
File Descriptors
A File Descriptor (FD) is a non-negative integer that acts as an index into the process's open file table.
0:STDIN(Standard Input)1:STDOUT(Standard Output)2:STDERR(Standard Error)
Core System Calls
Unlike printf or scanf (which are buffered library functions), the following are direct entries into the kernel:
open(): Requests access to a file. Returns an FD.read(): Copies bytes from a file/stream into a buffer.write(): Copies bytes from a buffer to a file/stream.close(): Releases the FD.
int fd = open("data.txt", O_RDONLY);
char buf[1024];
ssize_t bytes_read = read(fd, buf, sizeof(buf));
write(STDOUT_FILENO, buf, bytes_read);
close(fd);
Process Management: Fork, Exec, and Wait
A process is an instance of a program in execution. It consists of the executable code, its own private virtual memory space, and metadata (PID, owner).
The fork() System Call
fork() creates a new process (the child) by making an exact copy of the calling process (the parent).
- In the Parent:
fork()returns the PID of the child. - In the Child:
fork()returns 0.
The exec() Family
exec() replaces the current process's memory image with a new program. The PID does not change, but the code, stack, and heap are wiped and replaced.
The wait() System Call
A parent process uses wait() to block until a child process finishes. This allows the parent to collect the child's exit status.
Zombies and Orphans:
- A Zombie is a process that has finished but its parent hasn't called
wait()yet. It occupies a slot in the process table.- An Orphan is a process whose parent has died. It is "adopted" by the
initprocess (PID 1).
Undefined Behavior (UB)
Undefined Behavior is a unique and terrifying aspect of C. The C standard defines certain actions for which the language places no requirements. When a program encounters UB, the compiler is allowed to do anything—including deleting your code, crashing the program, or (most dangerously) appearing to work correctly while leaving a security hole.
Common Sources of UB:
- Buffer Overflows: Writing past the end of an array.
- Use-after-free: Accessing memory after it has been
free()'d. - Dereferencing NULL: Attempting to read/write to address 0.
- Signed Integer Overflow: Adding 1 to
INT_MAX.
Why does UB exist?
UB allows the compiler to assume that certain "bad" things never happen, which enables aggressive optimizations. For example, if the compiler assumes an integer will never overflow, it can simplify loops and math logic that would otherwise require expensive checks.
Defense: Sanitizers
Modern systems programming relies on Sanitizers. These are tools (built into GCC/Clang) that add runtime checks to catch UB.
AddressSanitizer (ASan): Detects memory errors.UndefinedBehaviorSanitizer (UBSan): Detects integer overflows, null pointer dereferences, etc.
Source Materials
Study Systems Programming in C and Unix Environments on Lykke
This course page is free to open: signed out, you can read its first two sections, and signing in, on any plan, opens the rest. The free Lykke Chrome extension syncs your own Canvas courses, so ChatGPT, Claude and Muse can answer from your classes and deadlines. It only reads Canvas.
Add the free extensionSign in to open every sectionAll sections of Systems Programming in C and Unix Environments · Browse more course pages