← all writeups
fuzzingcve

How a stale metadata field became a heap buffer overflow

2026.03.18 · 10 min read · 0xA13

Some colleagues of mine do security research and some of them present the findings at conferences. Watching that happen from the sidelines was motivation enough to finally try fuzzing myself. Media libraries felt like a good starting point.

One of the libraries I picked was libde265.

libde265 is an open-source H.265/HEVC decoder maintained by Dirk Farin. The library serves as the default HEVC decoding backend for libheif, a widely used HEIF/HEIC image processing library. libde265 has also been integrated into multimedia frameworks such as GStreamer and has historically provided HEVC decoding support for projects including VLC and FFmpeg. Because HEIC is the default photo format used by modern iPhones, libde265 is frequently involved in the decoding of iPhone-generated images on Linux systems and in cross-platform applications that rely on libheif for HEIC support. That makes it a fairly interesting target.

While fuzzing libde265 1.0.16 with libFuzzer and AddressSanitizer I found a heap out-of-bounds write, which was later assigned CVE-2026-33165.

The crash itself came back quickly. Understanding why it happened took considerably longer.

What even is a CTB?

Before getting into the crash, one HEVC concept needs to be explained: the Coding Tree Block, usually written as CTB.

A video decoder does not reason about individual pixels all the time. Instead, HEVC divides a picture into larger square regions. Those are CTBs. The decoder tracks per-CTB metadata: which slice a given block belongs to, its quantisation parameters, deblocking information, rather than per-pixel metadata.

The size of each CTB is not fixed. It is described by a parameter inside the bitstream called Log2CtbSizeY:

Log2CtbSizeY = 5  ->  2^5 = 32px CTBs
Log2CtbSizeY = 6  ->  2^6 = 64px CTBs

This matters because Log2CtbSizeY is what the decoder uses to convert between pixel coordinates and CTB indices. If one part of the decoder thinks CTBs are 32px wide and another part thinks they are 64px wide, the same pixel coordinate maps to two different CTB indices, and one of them may be out of range.

What is an SPS?

The SPS, or Sequence Parameter Set, is a chunk of configuration inside an HEVC bitstream. It tells the decoder how to interpret the data that follows.

For this bug, only three SPS fields matter:

PicWidthInCtbsY   - how many CTBs wide the picture is
PicHeightInCtbsY  - how many CTBs tall it is
Log2CtbSizeY      - how large each CTB is

Two SPS entries can describe images with completely different resolutions while sharing identical PicWidthInCtbsY and PicHeightInCtbsY values, as long as the CTB size scales proportionally. For example:

SPS Log2CtbSizeY PicWidthInCtbsY PicHeightInCtbsY
SPS 0 5 (32px CTBs) 13 8
SPS 1 6 (64px CTBs) 13 8

The CTB size changes, the CTB grid stays 13x8. That is exactly the condition in the reproducer. Getting back to that in a moment.

The metadata array

libde265 stores per-CTB metadata in de265_image::ctb_info, backed by a flat array allocated in alloc_image. The structure that wraps it looks roughly like:

template<class DATA>
struct image_data_array {
  DATA*  data;
  int    data_size;
  int    width_in_units;
  int    height_in_units;
  int    log2unitSize;      //cached CTB size, set at allocation time
};

For a 13x8 CTB grid, the array holds 104 entries. Valid indices are 0-103. The log2unitSize field is stored alongside the array so that later code can convert pixel coordinates back into CTB indices without needing to reach into the active SPS.

That cached log2unitSize is the bug.

The reallocation condition

When the active SPS changes, alloc_image checks whether ctb_info needs to be rebuilt:

//image.cc:458-459
if (ctb_info.width_in_units  != sps->PicWidthInCtbsY ||
    ctb_info.height_in_units != sps->PicHeightInCtbsY)
{
    //reallocate
}

Notice what is not checked: Log2CtbSizeY.

If the CTB grid dimensions stay the same but the CTB size changes, the check sees equal dimensions, decides there is nothing to rebuild, and moves on. The ctb_info array is reused as-is, including its cached log2unitSize, which now reflects the old SPS rather than the new one.

The reproducer contains two SPS entries that share PicWidthInCtbsY=13 and PicHeightInCtbsY=8 but differ in Log2CtbSizeY: 5 in SPS 0, 6 in SPS 1. The reallocation check sees 13==13, 8==8, and skips. The array keeps log2unitSize=5 while the rest of the decoder now operates under Log2CtbSizeY=6.

A note on the >> operator

The root cause is a stale value being fed into an otherwise correct calculation, so it helps to understand what that calculation actually does.

The >> operator is called a right shift. Numbers in a computer are stored in binary, and shifting right moves all the bits a certain number of places to the right, dropping whatever falls off the end. Right-shifting by N is exactly the same as dividing by 2^N and throwing away the remainder.

So:

256 >> 5  =  256 / 32  =  8
256 >> 6  =  256 / 64  =  4

The decoder uses this to convert a pixel coordinate into a CTB row number. Pixel 256, with 32px CTBs, is in row 8. Pixel 256, with 64px CTBs, is in row 4. The number stored in log2unitSize is the shift amount, so it controls which answer you get.

The bug is that log2unitSize still says 5 when the active SPS has already moved to 6. So the decoder asks “which row is pixel 256 in?” and gets 8 instead of 4. Row 8 does not exist in an 8-row array. Row 4 would have been perfectly fine.

The crash

Once the second SPS is active, the decoder starts decoding a CTB at row 4. Pixel coordinates are computed using the current SPS:

//slice.cc:2857-2866
int yCtbPixels = yCtb << sps.Log2CtbSizeY;   //4 << 6 = 256
img->set_SliceHeaderIndex(xCtbPixels, yCtbPixels, shdr->slice_index);

That’s fine. Row 4 times 64 pixels per CTB equals pixel 256. But then set_SliceHeaderIndex passes those coordinates to ctb_info.get(), which converts them back to CTB indices using the cached log2unitSize:

//image.h:741 -> image.h:124-131
DataUnit& get(int x, int y) {
    int unitY = y >> log2unitSize;      //256 >> 5 = 8  (stale log2unitSize)
    //assert(unitY < height_in_units) removed in release build (-DNDEBUG)
    return data[ unitX + unitY*width_in_units ];    //OOB WRITE
}

256 >> 5 = 8. The array has 8 rows, valid range 0-7, and row 8 is one past the end.

There is a per-iteration guard in decode_substream that checks CTB coordinates before this point:

//slice.cc:4732-4733
if (ctbx >= sps.PicWidthInCtbsY ||
    ctby >= sps.PicHeightInCtbsY) {   //0 >= 13 -> false, 4 >= 8 -> false

The guard uses coordinates from the active SPS and sees a completely valid CTB at (0, 4) so it passes. The bug is not that the guard is wrong, it is that the guard and the metadata write do not share the same coordinate system.

ASan confirms it:

==46762==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x51e000000a42 at pc 0x5bdb7a052986 bp 0x7ffdf7d40390 sp 0x7ffdf7d40388
WRITE of size 2 at 0x51e000000a42 thread T0
    #0 0x5bdb7a052985 in de265_image::set_SliceHeaderIndex(int, int, int) /home/ana/fuzzing-lab/libde265/libde265-1.0.16/libde265/../libde265/image.h:741:40
    #1 0x5bdb7a052985 in read_coding_tree_unit(thread_context*) /home/ana/fuzzing-lab/libde265/libde265-1.0.16/libde265/slice.cc:2866:8
    #2 0x5bdb7a05d8c0 in decode_substream(thread_context*, bool, bool) /home/ana/fuzzing-lab/libde265/libde265-1.0.16/libde265/slice.cc:4755:5
    #3 0x5bdb7a060413 in read_slice_segment_data(thread_context*) /home/ana/fuzzing-lab/libde265/libde265-1.0.16/libde265/slice.cc:5068:14
    #4 0x5bdb7a00d576 in decoder_context::decode_slice_unit_sequential(image_unit*, slice_unit*) /home/ana/fuzzing-lab/libde265/libde265-1.0.16/libde265/decctx.cc:857:7
    #5 0x5bdb7a00bb39 in decoder_context::decode_slice_unit_parallel(image_unit*, slice_unit*) /home/ana/fuzzing-lab/libde265/libde265-1.0.16/libde265/decctx.cc:959:11
    #6 0x5bdb7a00aa66 in decoder_context::decode_some(bool*) /home/ana/fuzzing-lab/libde265/libde265-1.0.16/libde265/decctx.cc:744:13
    #7 0x5bdb7a0100a6 in decoder_context::decode(int*) /home/ana/fuzzing-lab/libde265/libde265-1.0.16/libde265/decctx.cc:1343:11
    #8 0x5bdb79fffb92 in main /home/ana/fuzzing-lab/libde265/issue/poc.c:40:15

0x51e000000a42 is located 2 bytes after 2496-byte region [0x51e000000080,0x51e000000a40)
allocated by thread T0 here:
    #0 0x5bdb79fc13c3 in malloc
    #1 0x5bdb7a02d40a in MetaDataArray<CTB_info>::alloc(int, int, int) /home/ana/fuzzing-lab/libde265/libde265-1.0.16/libde265/./image.h:94:25
    #2 0x5bdb7a02d40a in de265_image::alloc_image(...) /home/ana/fuzzing-lab/libde265/libde265-1.0.16/libde265/image.cc:463:39

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/ana/fuzzing-lab/libde265/libde265-1.0.16/libde265/../libde265/image.h:741:40 in de265_image::set_SliceHeaderIndex(int, int, int)
Shadow bytes around the buggy address:
  0x51e000000980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x51e000000a00: 00 00 00 00 00 00 00 00[fa]fa fa fa fa fa fa fa
  0x51e000000a80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
==46762==ABORTING

The => row is the shadow row containing the crash address 0x51e000000a42. Each shadow byte covers 8 application bytes; [fa] marks the shadow byte whose range includes 0xa42. The allocation ends at 0xa40, so 0xa40-0xa47 is the first redzone cell, the write at 0xa42 is 2 bytes into it.

The write is 2 bytes past a 2496-byte region. 2496 = 104 * 24, matching the 104-entry array exactly.

Debug builds confirm it independently

What’s worth noting is that the assert guards at image.h:128-129 fire independently in a debug build without any sanitizers involved:

assert(unitX >= 0 && unitX < width_in_units);   //image.h:128
assert(unitY >= 0 && unitY < height_in_units);  //image.h:129

Two separate crash inputs trigger each assert independently, one for the unitX out-of-range path, one for unitY. That matters because it means the OOB condition is reachable on both axes, and it is the library’s own bounds check firing. In a release build these asserts are compiled out with -DNDEBUG and the write proceeds silently.

The summary of what actually happened

  1. The program allocated an array of 104 entries to track metadata for a 13x8 grid of CTBs, and stored the CTB size (32px, log2 = 5) alongside it.
  2. The video configuration switched to a new SPS that kept the 13x8 grid but changed the CTB size to 64px (log2 = 6).
  3. The program checked whether the grid dimensions changed. They had not, so it skipped rebuilding the array. But it forgot to update the stored CTB size.
  4. Later, the program needed to record which slice a CTB at pixel row 256 belongs to. It divided 256 by the stored (stale) CTB size of 32, getting row 8.
  5. Row 8 is past the end of an 8-row array. The write landed past the end of the heap allocation.
  6. The deblocking filter read the corrupted value back in the same decode pass.
  7. ASan caught this in testing. In a production build, it would have been silent corruption.

Disclosure

  • CVE: CVE-2026-33165
  • Affected version: libde265 1.0.16
  • CWE: CWE-787 (Out-of-bounds Write)
  • Reported to: Dirk Farin (farindk) via GitHub Security Advisory
  • Advisory: published, CVE assigned via GitHub CNA
  • References: https://www.sentinelone.com/vulnerability-database/cve-2026-33165/, https://github.com/strukturag/libde265/security/advisories/GHSA-653q-9f73-8hvg