mytool.cool

Green fringing: where it comes from and how to kill it

6 min read

On this page

Cut a sprite out of a green screen, drop the result on a white page, and look at the edge. There is a pale green halo. Now put the same PNG on a green page: the halo disappears.

That is the tell. The green is not in your sprite. It is in the pixels at the boundary, and those pixels are a mixture.

Every edge pixel is a weighted average

When a renderer draws a sprite edge at a sub-pixel position, it blends:

C_pixel = α · C_fg + (1 − α) · C_bg

A pixel that is 70% sprite and 30% green screen has a colour that is 30% green. That is not a defect — it is how anti-aliasing works, and it is why the edge looks smooth instead of stair-stepped.

The problem starts when a background remover has to decide what to do with that pixel. There are only three options, and two of them are wrong:

  • Keep it as-is. The edge stays smooth, and carries 30% green into your asset.
  • Delete it. The edge is clean, and 1px thinner than the original. Do this everywhere and outlines start to disappear.
  • Unmix it. Recover α and C_fg separately, then write a pixel that is 70% sprite colour at 70% opacity. Smooth and clean.

The third option is the only correct one. It is also the only one that requires knowing the background colour — which, on a flat colour screen, you do.

Unmixing is a division, and divisions amplify errors

Given the background colour C_bg and an estimate of alpha, the foreground colour is:

C_fg = (C_pixel − (1 − α) · C_bg) / α

Small alpha means a large denominator, and any error in alpha is multiplied. This is where most fringe artefacts are actually born — not at the edge, but inside this division.

A measured example from our own pipeline, on a dark red outline that had been assigned α = 0.74 instead of ~1.0:

C_g = (52 − 0.26 × 249) / 0.74 = −17 → 0

A negative channel. Clamped to zero, the dark red stroke came out bright red. The same mechanism turns near-black colours black and produces grey fringes wherever the division overshoots. Every one of those artefacts traced back to a single alpha value being 0.26 too low.

So the ordering matters: alpha first, then colour. A pipeline that estimates colour before alpha is guessing twice.

Three places a fringe escapes

Fixing this in one place is not enough. In our postmortem, green halos came from three separate mechanisms, and each needed its own gate.

1. The edge band gets deleted, then resurrected at a floor alpha

A threshold removes the anti-aliased band — correct, but it leaves a hole in the alpha field. A later stage that fills holes then writes those same pixels back, with a minimum alpha floor of 0.74 to avoid semi-transparent gaps.

Now a pixel whose true alpha was 0.15 comes back at 0.74, and its colour is solved from a white-veil model that keeps the green:

raw (97, 229, 68) → out (116, 224, 84, α = 189)
composited on white ≈ (163, 238, 133)

A pale green halo, produced by a step whose entire purpose was to avoid holes.

The fix was a gate rather than a lower floor: a killed pixel may only be resurrected if its own estimated alpha is at least 0.30. The cane's edge band measured 0.06–0.24 — essentially background, correctly deleted. The sphere's genuine fade-out band measured 0.5–0.9, and survives.

2. A fill with no material check

The same hole-filling step grew along "unreachable" regions with no colour or material test attached. On one image, a candy cane's unreachable area was connected to a glass sphere's — so the fill started at the sphere and swallowed 806,254 pixels, about 19% of the frame, including the entire cane and its anti-aliased band. Everything it wrote kept a green tint.

Fixing it took three constraints, all cheap:

  • a wall: the fill stops at pixels already classified as foreground or hard edge;
  • the alpha gate above;
  • a distance limit: a resurrection candidate must be within 12 pixels of a genuine glass write region. The sphere's own band qualifies; a cane's edge band running hundreds of pixels around a different object does not.

3. The classification boundary is a 1–2px step

The nastiest one is not a colour bug at all. Two adjacent regions get different treatments — glass at α ≈ 0.74–0.95 with its material green, shadow at α = 1 − L/L_bg ≈ 0.4 in neutral grey. The border between them is a flood-fill boundary with pixel-level noise, so it wanders.

On white, that reads as alternating green and grey "stitches" along an arc: a hard step between 0.9-green and 0.4-grey that moves forward and back pixel by pixel.

No amount of despill fixes this, because each side is internally correct. What fixes it is smoothing along the boundary — tangential consistency — so the step becomes a short ramp instead of a cliff, and the noise only shifts where the ramp sits rather than whether it exists.

What the fix looks like in numbers

After the gates, on the candy cane that originally showed the halo:

  • the anti-aliased band at x = 671–673 is fully transparent: (0,0,0,0);
  • x = 674 gets a 1px soften at α = 198 with RGB unchanged — a real edge, not a green one;
  • the dark red outline stays dark red ((129, 32, 8), alpha 253), within 2 per channel of the source;
  • the sphere's fade-out band is preserved as a continuous arc with no fragments.

The general rule those numbers encode: an edge pixel should be judged by its own alpha, not by the region it happens to touch.

How to tell whether a tool has this problem

Three checks, in order of how quickly they expose it:

  1. Composite on white and on black. A correctly unmixed edge looks the same on both. A green-tinted band changes colour between them, because the background is still baked in.
  2. Sample the alpha ramp. Walk a horizontal line across the silhouette. You want four or more distinct alpha values. Two values means thresholding; a long flat run followed by a jump means a floor alpha was applied.
  3. Look for stitches. Zoom into a curved edge. Alternating green and grey segments along the arc is a classification-boundary artefact, and it will show up on any background.

The short version

Green fringing is not one bug. It is what happens whenever a mixed pixel is treated as if it belonged entirely to one side — kept (green halo), deleted (thinner outline), or resurrected without checking its own alpha (halo again).

The way out is boring and exact: know the background colour, estimate alpha before colour, unmix with a division that respects its own error, and gate every later stage on the pixel's own alpha rather than on the region around it.

If you want to check a sprite of your own, the green screen remover runs locally in the browser — nothing is uploaded, and the edge band is where you should look first.

Try it: Green screen removerSprites out of a green screen, fringe included.

Related reading