Skip to the main content
mkfifo.com

magick, and placeholder images

planned discovered changed fixed

The builtin image became magick, with the words of ImageMagick 7 and a test against ImageMagick itself. A FIFO can have 32 ingress envelopes now, and /fpo gives placeholder images with no new code.

magick, with the words of ImageMagick

The builtin image had the options of a URL of Cloudflare Images as its word. It was the only builtin that was not a tool of a shell, and /bin/image next to the FIFO /images was strange. Now the builtin is magick: a part of the command magick of ImageMagick 7, with the same words.

$(magick - -auto-orient -strip -thumbnail '192x>' -quality 100 webp:- < photo)

A person who knows ImageMagick knows these words. And a test can give the same words to ImageMagick and compare the two images.

The same routes

/images/width=192,format=webp/NAME still works. The envelopes of /images change the options width, height and format into the words of magick. They come in this sequence, with a comma between them, and each shape has a route. A route does not make an image larger (192x>), a PNG and a WebP are lossless, and a JPEG goes on white with the quality 85. The other options of Cloudflare are gone.

I wrote one route for each of the 11 shapes, and the seed failed: a FIFO has at most 16 ingress envelopes, and mkfifo had 8. A route can have an optional group, so a route with a width also takes a height:

To: images/width=:width([1-9]\d{0,3}){,height=:height([1-9]\d{0,3})}?/:src([^\/]+(?<!\.svg))

With no height, -thumbnail "${width}x${height}>" is 192x>. I could not do the same for the width: ImageMagick reads x>, with no width and no height, as a size of 1 by 1 pixel. Thus there are 8 routes, and mkfifo has 16 envelopes now.

A test against the real tool

Ubuntu 24.04 has no package of ImageMagick 7. scripts/magick.sh builds ImageMagick 7.1.2-32 from its source, and gives the same random images (PNG, JPEG, WebP and GIF) and the same random words to the two. The two images must have the same format and the same size, and a pixel of a PNG or a WebP can differ by at most 3 levels of 255. The CI runs it on each push, and keeps the build in its cache.

To get there, the builtin does the work as ImageMagick does it:

  • The resize has the code of MagickCore/resize.c: the same filters (Lanczos, Mitchell for an image with alpha, and Box and LanczosSharp for -thumbnail), the same weights, and the same pass first. Between the two passes, ImageMagick HDRI keeps floats with no clamp: a pixel near a sharp edge can be more than white for a moment. With a clamp there, the test found pixels that differed by more than 3 levels.
  • After a resize, the pixels have 16 bits, with no clamp, and the colors stay multiplied by their alpha. A color with an alpha near 0 can be very large in ImageMagick, but the product stays small.
  • A JPEG of ImageMagick has the sampling factors of its input, else 4:4:4 for a quality of 90 or more, and 4:2:0 below. Its quality comes from -quality, else ImageMagick estimates the quality of the input from its tables. The encoder is now the crate jpeg-encoder, which can write these sampling factors.
  • At the right and the bottom edges of a JPEG with half the color, the first decoder took the padding of the blocks, and libjpeg repeats the last color. The crate jpeg-decoder gives the colors of libjpeg.

Some differences stay. A JPEG of ImageMagick uses the fast DCT of libjpeg up to the quality 90, and the builtin the accurate DCT: the mean of the pixels differs by up to 4 levels. On a small image with much noise, two encoders of JPEG round in different ways, and the difference can be more. There the test asks that the JPEG of the builtin is as near to the image as the JPEG of ImageMagick.

An error, not a different image

Some words make an image that the builtin cannot make. Then magick gives an error, and the error tells the words that give the same image:

  • An image with an EXIF orientation needs -auto-orient or -strip, and an image with a color profile needs -strip: the builtin writes no profiles.
  • -crop needs +repage after it. After -auto-orient of the orientation 5 or 7, ImageMagick keeps the old size of the page, and -crop, -extent and -flatten read it: +repage must come before them.
  • A JPEG has no alpha: transparent pixels need -background COLOR -flatten.
  • A WebP must be lossless: webp:- needs -quality 100 or -define webp:lossless=true. The encoder of the builtin writes only lossless WebP. With -define webp:lossless=true and a quality below 100, ImageMagick writes a "near lossless" WebP, which the builtin cannot write either.

I found each of these with the test: a case that differed, then the source of ImageMagick. -crop of an area out of the image gives an image of one transparent pixel, not an error. -resize 56x^ reads the missing height as 0, not as the height of the image.

The cost

The first version needed 12,200,000 steps for an image of 1200 by 630 pixels to a WebP 600 pixels wide: more than the 10,000,000 that a Request has. The resize read each source pixel again for each weight, and changed it to a float each time. Now it changes each row one time. It keeps only a few rows: the rows of a target row, or, when the image gets much smaller, the sums of the target rows. The small functions of a pixel are inline, and the output goes into a buffer of its size. A work of many output pixels takes a third of the time it took. I measured each part again: a step of the resize is about 0.08 µs, as a step of resvg, in the interpreter of WAMR on my computer. The same image needs 9,200,000 steps now.

A photo of 1920 by 1008 pixels to a width of 800 still needs 27,900,000 steps, so it gets no image. The Worker runs the WebAssembly in V8, which is much faster than the interpreter, but the steps of all the builtins come from the same interpreter, so the limit stays the same for each of them.

More envelopes in a FIFO

A FIFO can have 32 ingress envelopes now, not 16.

The limit of 16 was a first value from the start of the project, and the routes of /images found it. The FIFO mkfifo of the site holds fonts, the icons, robots.txt and images, with 8 envelopes. I wanted one route for each shape of the options of /images, 11 routes, and the seed failed. So the routes with a width took an optional height, and mkfifo had 16 envelopes, with no free place.

Why a limit stays

Each person who knows the name of a FIFO can add an envelope to it. Each Request to a FIFO goes through all of its ingress envelopes, the process of the FIFO keeps all of them in memory, and the editor shows all of them. One Durable Object does the work of all the FIFOs. A FIFO with thousands of envelopes would make each FIFO slow, not only its own. 32 gives room for a FIFO with many routes, and keeps the work of one Request small.

The limit of 64 egress messages for one Request stays. Each ingress envelope can have 4 egress envelopes, so 32 of them can have 128. But a Request that agrees with more than 16 ingress envelopes is not usual, and the messages after the 64th fail with an error.

One route for each shape

/images has one route for each shape of width, height and format again: 11 routes, and mkfifo has 19 envelopes. A route is easier to read with no optional group:

To: images/width=:width([1-9]\d{0,3}),height=:height([1-9]\d{0,3})/:src([^\/]+(?<!\.svg))

The URLs did not change: /images/width=192,format=webp/NAME gives the same image as before.

Placeholder images

/fpo/300x250 gives a placeholder image: a gray box of 300 by 250 pixels, with 300×250 on it. FPO is "for position only": an image that holds the place in a design until the real image is there.

The words are those of fpoimg.com, as the words of /images are those of Cloudflare: the size in the path, one number for a square, and the fields of the query.

/fpo/300x250?text=Hero+image&bg_color=4361ee&text_color=white

text goes above the label, bg_color and text_color take hexadecimal digits or a name of CSS, and dims=false removes the label.

No new code

/fpo is one ingress envelope and one answer envelope in the seed of the FIFO mkfifo, and it uses only the parts that are there already. A route takes the size, with the height as an optional group: fpo/:width([1-9]\d{0,3}){x:height([1-9]\d{0,3})}?. A GET of a held FIFO is a form, so the fields of the query are field lines, and their patterns refuse a color that is not a color. The answer envelope is the SVG itself, with Content-Type: image/svg+xml.

A template cannot calculate, and the size of the text must change with the size of the image and the length of the text. The builtin jq does the arithmetic: the font size is the smaller of a part of the height and the width divided by the number of characters, and two lines stay at the center together.

The colors needed a default with no if. CSS ignores a declaration that is not valid, so the rectangle has fill="#dddddd" style="fill: $bg_color; fill: #$bg_color". With steelblue, the first declaration is valid. With 4361ee, only the second one is valid. With no color, the two are not valid, and the gray of the attribute stays.

Safe text in an SVG

An SVG can have a script, and the text comes from the URL. Two rules of the answers make it safe. A value in a body of XML gets the escapes of HTML, so <script> becomes text. And each answer has Content-Security-Policy: sandbox with default-src 'none', so a script does not run. The tests send <script> in text, and check that the SVG is well-formed XML.

Only SVG

A placeholder as a PNG needs the label as pixels. resvg draws an SVG in the VM, but it has no fonts, and refuses <text>. Thus /fpo gives only SVG, and the browser draws the text with its own font.

Late timers in the tests

Two tests failed one time each this week: a timer of 250 ms came after 1 s, and a trace message after 100 ms. The cause was the NIFs of magick and resvg. The calls into one library of WebAssembly run one at a time in beam.com, and a call that waits for a different call waits on its scheduler. The tests of magick render large images at the same time, so they kept all the normal schedulers. With 8 renders, a timer of 50 ms came 4.8 s late on my computer.

Now the functions of the two NIFs run on the dirty CPU schedulers, and the same timer comes at most 1 ms late. In the Worker there is one thread, so this changes nothing there: the steps of a Request still limit the work of each call.