Memory Allocation APIs: malloc, free, brk, and mmap

Introduction

Programs request memory while they run, but every small allocation does not usually go directly to the operating system. Most programs first talk to a user-space allocator through functions such as malloc.

The allocator manages memory blocks inside the process and asks the OS for more memory only when needed. This separation explains why malloc, free, brk, and mmap have different roles.

Application, Allocator and OS

Memory allocation is best understood as a layered flow. The application asks for memory, the allocator manages available heap blocks, and the operating system provides larger memory regions when the allocator needs more space.

A call like malloc(100) does not necessarily mean the kernel is asked for exactly 100 bytes. The allocator may already have a suitable free block and can satisfy the request without a system call.

malloc and free

malloc(size) requests a block of at least size bytes and returns a pointer to that block if allocation succeeds. The allocator may internally reserve extra space for alignment, size classes, or metadata, but the program should use only the memory it requested.

free(ptr) returns a previously allocated block to the allocator. After that, the allocator can reuse the block for future allocations, split it, merge it with nearby free blocks, or keep it in an internal free list.

Together, malloc and free form the application-facing pair.

Typical malloc Flow

When a program calls malloc(100), the allocator first checks the memory it already manages. If a suitable free block exists, it returns that block to the program.

If no suitable block exists, the allocator asks the operating system for more memory. The OS may provide a larger region through mechanisms such as brk, sbrk, or mmap.

The basic flow is:

• Program requests memory: It calls malloc(100).

• Allocator checks free blocks: It looks for reusable memory.

• Suitable block exists: The allocator returns a block without asking the OS.

• No suitable block exists: The allocator requests more memory from the OS.

• OS provides a larger region: The allocator adds it to its managed space.

• Allocator returns a pointer: The program receives usable memory.

This is why many small allocations can be handled without one system call per allocation.

brk and sbrk

brk and sbrk are older Unix-style mechanisms related to the program break. The program break marks the end of the traditional heap area.

Moving the program break can grow or shrink the traditional heap. Allocators may use this mechanism to obtain more contiguous heap space from the operating system.

Key points:

• Traditional heap growth: brk or sbrk changes the end of the heap.

• Contiguous style: It is associated with extending the traditional heap region.

• Usually indirect: Applications normally call malloc, not brk or sbrk directly.

• Allocator-dependent: Whether and when this path is used depends on the allocator.

In modern systems, applications should generally treat brk and sbrk as low-level mechanisms rather than everyday allocation APIs.

mmap

mmap creates a separate mapping in the process address space. It can map anonymous memory, files, devices, or shared objects.

For memory allocation, mmap gives the allocator another way to obtain memory from the OS. It is often used for large allocations because the mapped region can be managed and unmapped independently using munmap.

mmap is also used outside ordinary heap allocation. It supports memory-mapped files, shared memory, and file-backed mappings.

brk vs mmap

brk and mmap both can provide memory to a process, but they do it differently.

The allocator decides which mechanism to use based on request size, current free blocks, implementation policy, and system behavior.

Allocator Metadata

An allocator needs metadata to remember the blocks it manages. This metadata helps it know which blocks are free, which are allocated, and how large each block is.

Metadata may track:

• Block size: How large the block is.

• Free or used state: Whether the block is available.

• Links between free blocks: Useful for free lists.

• Alignment information: Helps place blocks correctly.

• Neighbor information: Helps split or merge blocks.

Metadata allows the allocator to reuse memory, divide larger blocks, and merge adjacent free blocks into larger reusable regions.

Common Allocation Bugs

Memory allocation bugs often come from using memory outside its valid lifetime or valid boundary.

Common bugs include:

• Memory leak: Memory is allocated but never freed.

• Use-after-free: Code uses a block after it has been returned to the allocator.

• Buffer overflow: Code writes beyond the allocated block.

• Metadata corruption: An overflow damages allocator metadata, causing failures later.

Metadata corruption is especially difficult because the visible crash may happen long after the original invalid write.

Summary

Memory allocation works through layers. The application calls malloc, the user-space allocator manages heap blocks, and the allocator asks the OS for more memory through mechanisms such as brk, sbrk, or mmap when needed.

malloc obtains memory, free returns it to the allocator, brk and sbrk relate to traditional heap growth, and mmap creates separate mapped regions. The central rule is that malloc is allocator-facing, while brk and mmap are lower-level OS-facing mechanisms.