Input image exceeds pixel limit: resizing a 400 MP JPEG with sharp
The sharp library stops a 400-megapixel JPEG with "Input image exceeds pixel limit", and lifting the limit alone can cost more memory than a small server has. Here is what we measured, what libvips really does with the file, and the one kind of JPEG that stays out. Part of the image compression API.
By Sahadat HusainUpdated 9 min readEngineering
Pixel-shift cameras and 200-megapixel phones make files that a small server cannot simply open. The same engine runs behind the web tool and the image compression API, so this is the path every such file takes, with the numbers from our own runs.
Why sharp says “Input image exceeds pixel limit”
sharp checks the pixel count in the file’s header before it decodes anything. The default ceiling, limitInputPixels, is 268,402,689 pixels (16,383 × 16,383), so a 400-megapixel JPEG is refused with that error (sharp’s constructor). The fix is a higher number you can afford, never false, which removes the only check before the decode.
// The limit is a memory budget: set it to what the server can afford.
const image = sharp(input, { limitInputPixels: 400_000_000 });
// Resize in the same pipeline, so libvips streams the decode through it.
const out = await image
.resize({ width: 10_000, height: 10_000, fit: 'inside' })
.jpeg({ quality: 78, mozjpeg: true })
.toBuffer();How much memory a 400-megapixel JPEG needs
An 8-bit RGB raster of 20,000 × 20,000 is 1.2 GB before an encoder touches it. Run at full size through our engine with no ceiling, the file took 19.2 seconds and 2.4 GB. Our worker runs on a shared box with 7.8 GB and is held to 3.67 GB; the overflow instances on Cloud Run have 2.15 GB. And the browser decodes the full image for the before-and-after slider.
The two options
- Option A: a bigger machine. A heavy Cloud Run service and a per-account cap on heavy jobs, for a result nobody asked for: a 400-megapixel output file.
- Option B: halve it on the way in. Resize the JPEG to the largest power-of-two fraction under 100 megapixels and do everything after that as if the photo had arrived at that size. The person with the pixel-shift camera wants a shareable file, not a 400-megapixel one.
We took B. This is what the libraries do with it, what it cost, and where it stops.
What libvips shrink-on-load does, and when sharp uses it
A JPEG stores each 8 × 8 block as frequency coefficients. libjpeg can draw a block at half, a quarter or an eighth of its size by running a reduced inverse transform, so the scaling happens inside the decode. Each pixel it draws is the average of the 2 × 2, 4 × 4 or 8 × 8 pixels a full decode would give. libvips calls this shrink-on-load.
We built the path believing that asking for exactly half would trigger it. It does not. sharp picks a shrink of 2 for a reduction of 2 or more, then halves it again when the reduction rounds down to that same factor, to avoid libjpeg’s rounding errors. At exactly half, the JPEG is decoded at full size. libvips’ own log, with VIPS_INFO=1, shows it on the engine’s final pass: a reduce by 0.5, from the full size, behind a line cache.
VIPS-INFO: residual reducev by 0.5
VIPS-INFO: reducev sequential line cache
VIPS-INFO: residual reduceh by 0.5What holds the memory down is streaming. libvips pulls rows from the decoder in strips, the shrink and reduce turn them into output rows, and a line cache keeps only the strips in flight. The 1.2 GB raster never exists at once. Before it, two small passes for the content analysis, a 512-pixel proxy and a 64-pixel sample, are reductions far past 3, so sharp does shrink those inside the decode.
When sharp shrinks a JPEG on load
| Reduction of each side | Shrink inside the decode |
|---|---|
| Under 3 times | None: the full size is decoded |
| 3 up to 5 times | 2 |
| 5 up to 9 times | 4 |
| 9 times or more | 8 |
Full decode vs streamed halving: what we measured
Time
- Full size, no ceiling19.2 s
- Decoded whole, then halved5.2 s
- Halved, decode streamed6.3 s
Peak memory
- Full size, no ceiling2.4 GB
- Decoded whole, then halved2.1 GB
- Halved, decode streamed1.1 GB
| Path | Time | Peak memory | Output |
|---|---|---|---|
| Halved, decode streamed (the engine) | 6.3 s | 1.1 GB | 10,000 × 10,000 |
| Decoded whole, then halved | 5.2 s | 2.1 GB | 10,000 × 10,000 |
| Full size, no ceiling | 19.2 s | 2.4 GB | 20,000 × 20,000 |
| A third, shrunk inside the decode | 2.5 s | 0.54 GB | 6,666 × 6,666 |
Streaming costs about a second against holding the whole raster, because the engine reads the file three times, and it halves the peak. A third of each side would be faster still, with sharp’s shrink running inside the decode, but it throws away more than half the pixels a Pro Max user paid for. The job’s timeout budget follows the source’s pixels, not the result’s, because every byte of the file is still read.
Where the saving stops
Two limits: the formats we measured, and one kind of JPEG.
Why the engine takes only JPEG above 100 megapixels
We built and measured this path on baseline JPEG, the format pixel-shift cameras and phones write. Other formats behave differently: libvips can scale a WebP while loading it, and a HEIC carries a small preview, but neither is what a person with a 400-megapixel photo holds. So the engine refuses PNG, WebP, AVIF and HEIC above 100 megapixels from the header, in words, and the page says the same before the upload.
Progressive vs baseline JPEG
A baseline JPEG stores each block’s coefficients together, so the decoder can hand libvips one strip of rows after another. A progressive JPEG spreads every block across several scans of the file, so all of them must be buffered before a single row can be drawn, and nothing streams. The libvips developer checklist measured one image at 4 GB and 4.3 seconds to thumbnail as a progressive JPEG, against 72 MB and 0.26 seconds as a baseline one.
The header carries the flag, so the engine reads it and refuses a progressive JPEG above 100 megapixels before any decode, and the page reads the same frame marker in the browser and says so before the upload. The fix is a baseline export, which every editor offers.
What changed for a person using the tool
- On Pro Max, a JPEG up to 400 megapixels runs; the result is at most 100 megapixels, with a resize clamped to that raster.
- Free and Pro keep the 100-megapixel ceiling on every format, and a JPEG under it takes the ordinary path to compress JPG.
- Any format other than JPEG above 100 megapixels is turned away before the upload, with the reason in the dialog. If your camera app can export JPEG, export JPEG.
Read the full guide: Image optimization for the web: formats, sizes and lazy loading.
Questions
- How do I fix “Input image exceeds pixel limit” in sharp?
- Pass limitInputPixels at the ceiling your server can afford, for example 400,000,000, instead of the default 268,402,689 (16,383 × 16,383). Then resize down in the same pipeline, so libvips streams the decode through the resize rather than holding the whole raster in memory.
- Is limitInputPixels: false safe?
- No. False or 0 removes the only check sharp makes before decoding, so a header that claims a huge image is decoded as claimed. Our full-size run of a 400-megapixel JPEG peaked at 2.4 GB, and a hostile file can claim far more. Set a number your server can afford.
- What is sharp's memory usage on a 400-megapixel JPEG?
- Halved in the same pipeline, with the decode streamed, our run peaked at 1.1 GB. Decoded whole into memory first, the same halving peaked at 2.1 GB, and processing the file at full size took 2.4 GB. The raw RGB raster alone is 1.2 GB, which is why streaming matters.
- When does sharp shrink a JPEG on load?
- From a reduction of 3 times or more on each side, with fastShrinkOnLoad on, its default: a shrink of 2 from 3 times, 4 from 5 times, 8 from 9 times. Below 3 times the decode runs at full size, because sharp steps around libjpeg’s rounding errors at a factor of exactly 2.
- Does shrink-on-load lower the quality?
- A little. libjpeg draws each pixel as the average of the 2 × 2, 4 × 4 or 8 × 8 pixels a full decode would give, a box filter, and sharp’s resize docs warn of a slight moiré pattern. With fastShrinkOnLoad set to false, sharp’s source code leaves at least a factor of 2 for its own resampler.
- Does streaming help a progressive JPEG?
- No. A progressive JPEG spreads every block over several scans, so the whole file must be buffered before a row can be drawn and nothing streams. The libvips developer checklist measured 4 GB against 72 MB for the same image as baseline. Convert it to a baseline JPEG first.
- How do I tell whether a JPEG is progressive?
- Read the header: sharp’s metadata() returns isProgressive without decoding the image. In the file itself the frame marker says it, SOF2 being progressive and SOF0 baseline. Blinkimg reads the same marker in the browser before an upload, and the engine reads it again before any decode.
- How do I convert a progressive JPEG to baseline?
- Use jpegtran from libjpeg-turbo: jpegtran -copy all input.jpg > baseline.jpg. Without the -progressive switch it writes a plain baseline file, and because it rearranges the stored coefficients without decoding them, its manual says there is no image degradation at all.
