AVIF is now being hosted here.

First there was AV1. Now there’s AVIF

A while back during my video encoding sessions, I toyed with the concept of serving AV1 videos on my site. The results were not great out of respect that we could not find a great way of detecting the client’s capabilities of video decoding without writing some shitty JavaScript system to snoop around and break privacy rights all the way. However, there’s a branch of the Av1 system known as “AVIF” which is designed to use for still images. It boasts about getting better compression ratio’s to that of even WebP images that we discussed a while back.

Why did we implement AVIF here? Read on if you want to know more.

Cel Phone Towers.

Getting from point A to B.

One of the biggest reasons for all of these new and weird codecs appearing out of the woodwork is that when you graphically design something such as a website. That you want to be able to make it look good on high res monitors. But you also want to make it look as fast as a GeoCities website from the ’90s. Everyone is getting faster and faster so the concept of compression becomes irrelevant for major networks that want to hit you wish 1.8Mb of text and 22.7Mb of Javascript like CNN. That’s effectively an application every time you load their webpage! Now, you have a lot of big-boy companies out there that are serving millions of images per second. If you could deliver near-perfect quality content to your audience and save 30-50-70 percent of your bandwidth in doing so. Then it’s a win for everyone.

In our article about maybe not thinking with Google, you have people who believe that time is a temporal paradox that you can have 12 times the amount of people leaving your site than entering if your site exceeds 5 seconds.

That example is extreme; However, as a web designer, you want to at LEAST deliver the content of your site as fast as possible as there could be people on crappy cellphone towers or in countries where the internet is oftentimes not fantastic. Like Colorado.

Tor Networks blog title.

Also noted that about %10 of our readership comes from the Tor/Onion network environment. Because of the nature of Tor where of obfuscates your IP by bouncing your connection through a series of proxies connection time can vary from barely tolerable DSL connection speed to 56k modem mode depending on the locale and also depending on what supernode you are going through on their network.

Tor Results of S-Config.comPressing “12” will pull up the “network” tab for you to see your traffic.

Because the Tor/Onion client is built upon firefox builds it’s currently taking advantage of the WebP conversion that is happening within this site. The moment Tor adopts the later builds of Firefox those will be able to pull in AVIF files which will result in faster load times for our readers on the Onion/Darknet.

Deployment.

Now, we’re not going to use a PHP WordPress plugin in order to accomplish such feats. We -used- to use EWWW image optimizer back in the days where we were on shared hosting. but the settings it uses for WebP conversion were rather poor. To top it all off it will probably take years if it all for them to implement AV1F image formats.

Instead, we’re going to use some cron.script magic and Nginx fuckery.

Nginx action.

You’ve probably noticed in the network logs of our Tor/Onion browser that we are requesting a PNG or JPG but the “Type” category is a WebP. This is because the Nginx web-server is intercepting the request from my blog and converting the file path on the flow.

Let’s dive into Nginx to get this whole process going here.

First we need to tell Nginx what the *.avif file really is. So we need to open our mime.types within the root of nginx and add the following lines:

image/png png              apng;
image/webp                   webp;
image/avif                      avif;
image/avif-sequence   avifs;

and save.

Future versions of Nginx may already have these format-types set. But as of 1.19.x we certainly know avif is not apart of Nginx.

Moving onto the real action.

This is probably the point where you want to go to your /nginx/sites-available folder and find the site profile that you wish to modify. This is all done after your locations are defined within your profile.

# for PNG and JPG - This will test if a browser can run AVIF and fall back to WEBP.

set $webp_suffix "";
if ($http_accept ~* "webp") {
set $webp_suffix ".webp";
}
if ($http_accept ~* "avif") {
set $webp_suffix ".avif";
}

location ~ \.(jpg|png)$ {
expires 1y;
add_header Cache-Control "public, no-transform";
add_header Vary "Accept-Encoding";
try_files $uri$webp_suffix $uri $uri/ =404;
}

# for GIF - This will test for WEBP and then fallback to APNG files.

set $gifv_suffix "";
if ($http_accept ~* "apng") {
set $gifv_suffix ".apng";
}
if ($http_accept ~* "webp") {
set $gifv_suffix ".webp";
}

location ~ \.(gif)$ {
expires 1y;
add_header Cache-Control "public, no-transform";
add_header Vary "Accept-Encoding";
try_files $uri$gifv_suffix $uri $uri/ =404;

This is essentially a modified version of the Nginx script to force WebP if it’s available. It firsts asks the client if they can accept a certain format such as WEBP first, then AVIF . This way, If a Tor/Onion user enters my site they will at least get the advantage of WebP. If a Google Chrome user enters my site then it will bypass WebP imaging and go right for AVIF.

Finally, the script TRIES the file to see if an appropriate WEBP or AVIF is out there. for example, if our blog has a link to “./logo.png” it will search for “./logo.png.webp” or “./logo.png.avif” whichever the browser admits to accepting. If the browser does not respond to Nginx or the user is operating on a Potato for a PC then Nginx will naturally serve them the default .jpg or .png files.

Addressing the GIFs in the room.

It’s super sad in this era that we are still trying to clean up after a limited image format that can only provide 256 colors from the 1980s. But the script has an exception clause for that as well for the 0.002 percent of my site that relies on .GIF’s due to certain WordPress plugins. In this scenario, we Test for APNG and WebP. As at the time of our writing this AVIF motion is not supported by browsers or by the exporter utilities that we are using.

Apache time.

We haven’t really tested this as we’ve moved away from apache long ago. But as this too is based on the webp format you could just force everyone to avif all the same through the .htaccess file of your shared provider:

AddType image/avif .avif

<IfModule mod_rewrite.c>
  RewriteEngine On

  RewriteCond %{HTTP_ACCEPT} image/avif
  RewriteCond %{REQUEST_FILENAME} -f
  RewriteCond %{REQUEST_FILENAME}.avif -f
  RewriteRule ^/?(.+?)\.(jpe?g|png)$ /$1.$2.avif [NC,T=image/avif,E=EXISTING:1,E=ADDVARY:1,L]

  <IfModule mod_headers.c>
     <FilesMatch "(?i)\.(jpe?g|png)$">
     Header append "Vary" "Accept"
  </FilesMatch>
 </IfModule>
</IfModule>

CRON time!

Alright, so we addressed if the files are there or not. if we had a small website we could probably use a service like “squoosh” for example. No need to install files onto the server. Just convert the files you need to upload with the proper extensions and be done with it. However, as the image format wars wage onward and we like trying new things we needed a methodology mass conversion.

We’re about to show our CRON job. We are not fantastic at scripting so to those Linux users that are slapping their foreheads with the way we wrote it. Let us know. We would very much like to improve as well.

 

#!/bin/bash

Username=Your-Super-Awesome-Web-User

cd /place/where/your/www/image/data/exists/
shopt -s globstar
for i in **/*.jpg;
do
if [ ! -f ${i}.flag ]; then
/usr/bin/nice -n 20 sudo -u $Username /usr/bin/jpegoptim --strip-all ${i}
/usr/bin/nice -n 20 sudo -u $Username echo "Optimization Complete Flag." >> ${i}.flag
fi

if [ ! -f ${i}.avif ]; then
/usr/bin/nice -n 20 sudo -u $Username /usr/bin/avifenc --min 0 --max 25 --minalpha 0 --maxalpha 25 --speed 0 ${i} ${i}.avif
fi

if [ ! -f ${i}.webp ]; then
/usr/bin/nice -n 20 sudo -u $Username /usr/bin/cwebp ${i} -q 90 -m 6 -pass 10 -o ${i}.webp
fi
done

if [ ! -f ${i}.avif ]; then
/usr/bin/nice -n 20 sudo -u $Username /usr/bin/avifenc --min 0 --max 25 --minalpha 0 --maxalpha 25 --speed 0 ${i} ${i}.avif
fi

if [ ! -f ${i}.webp ]; then
/usr/bin/nice -n 20 sudo -u $Username /usr/bin/cwebp ${i} -q 90 -m 6 -pass 10 -o ${i}.webp
fi
done


for i in **/*.png;
do
if [ ! -f ${i}.flag ]; then
/usr/bin/nice -n 20 sudo -u $Username /usr/bin/optipng -o7 -zm1-9 -strip all ${i}
/usr/bin/nice -n 20 sudo -u $Username echo "Optimization Complete Flag." >> ${i}.flag
fi

if [ ! -f ${i}.avif ]; then
/usr/bin/nice -n 20 sudo -u $Username /usr/bin/avifenc --min 0 --max 25 --minalpha 0 --maxalpha 25 --speed 0 ${i} ${i}.avif
fi

if [ ! -f ${i}.webp ]; then
/usr/bin/nice -n 20 sudo -u $Username /usr/bin/cwebp ${i} -q 90 -m 6 -pass 10 -alpha_q 90 -o ${i}.webp
fi
done


for i in **/*.gif;

if [ ! -f ${i}.flag ]; then
/usr/bin/nice -n 20 sudo -u $Username echo "Optimization Complete Flag." >> ${i}.flag
fi

if [ ! -f ${i}.apng ]; then
/usr/bin/nice -n 20 sudo -u $Username /usr/bin/gif2apng -z0 ${i} ${i}.apng
fi

if [ ! -f ${i}.webp ]; then
/usr/bin/nice -n 20 sudo -u $Username /usr/bin/gif2webp -q 100 -m 6 ${i} -o ${i}.webp



for i in **/*.webp;
do if [ ! -f ${i:0:-5} ]; then
echo "removing ${i} as ${i:0:-5} is no longer existing."
rm ${i}
fi
done

for i in **/*.avif;
do if [ ! -f ${i:0:-5} ]; then
echo "removing ${i} as ${i:0:-5} is no longer existing."
rm ${i}
fi
done

for i in **/*.flag;
do if [ ! -f ${i:0:-5} ]; then
echo "removing ${i} as ${i:0:-5} is no longer existing."
rm ${i}
fi
done

for i in **/*.apng;
do if [ ! -f ${i:0:-5} ]; then
echo "removing ${i} as ${i:0:-5} is no longer existing."
rm ${i}
fi
done



fi
done

Now, we’re using colorists avifenc on my site as we can control the threads and memory usage which we simply could not do with Kagami’s Go-Avif . Each time we are converting an image on this site we echo a .flag file. This is important because we initially run this script outside of Cron to it takes the time to process all of the png and jpeg files with ridiculous settings for compression. On my site, it took a solid week before it was able to place a flag file on all of them. Then, we insert the script into our cron.daily so that it skips past all of the “.flag”ed files and only processes new files that come into this blog. Which is not as painful as doing the entire folder!

Finally, it does a reverse lookup on the original file. If we delete something in WordPress then the script will remove the files it has made as well keeping my directories clean.

The command we keep on passing to AVIF is the following.

avifenc --min 0 --max 25 --minalpha 0 --maxalpha 25 --speed 0 ${i} ${i}.avif

We setup a minimum to maximum compression to 25 which is the default. If you’re a photographer and you want to get the most compression out of AVIF without any perceptual quality loss I would recommend setting the –max and –maxalpha to 20 instead of 25 as there’s a bit of gaussian blurring that happens in finely detailed images.  Of course, sitting the compression lower will result in bigger files. So it’s a bit of a balancing act as to what you are trying to accomplish.

–speed 0 even though just like our PNG command is overkill. But it’s a background task on my VPS. I don’t care.

If you have no fucks to give about your still images like YouTube you’d probably be more inclined to up the compression to 30-32.

Now as for my commands passed on PNG files. I discussed all of those options in a previous blog.

Security considerations with WordPress.

You’ll probably be as ambitious as we were by instead of simply pointing this to the /wp-content/images folder that you’ll just point it to the root folder of WordPress. This way you’ll not only optimize images but any images that your plugins will be using. This is OKAY but you have to mind that if you are running any security plugins like WordFence of Sucuri Security you will more than likely piss those programs off. WordPress security software looks for any changes to the core of WordPress. and by adding *.webp, *.avif, *.flag files all over the place. Guess what? change detected.

So before you run a script like this (and if you are running a CMS similar to WordPress) make sure that the appropriate extensions are white-listed so you don’t come back to thousands of false-positives because of what you have done.

Shared hosting with AVIF/WEBP.

Chances are, your web-host administrator will probably not let you install a whole bunch of third-party binaries like gif2png, cwebp, avifenc, and jpegoptim. If you have the option of SSH so you can transfer files to your account you could try to rsync your files back and forth using an offsite process like a Raspberry Pi to handle all of the image conversions for you. Sure, it would eat away at your bandwidth costs a little but it keeps your hard processing offline and just you serving a website online.

We’re using a Virtual-Private-Server (VPS) so we don’t need to do the back and forth r-sync services for something like that.

Final thoughts.

Serving AVIF through Chrome Browser.Serving AVIF files through the Chrome Browser.

 

Serving all of these different formats media-types does eat up the hard drive space. In the server world, it is no longer a question of how much space your website takes up but how fast your webserver delivers the media. Media Storage is cheap. Bandwidth is not.

Until next time.

Server protect you.

+++END OF LINE

Leave a Comment to the Void