“Extinction is the rule. Survival is the exception.” – Carl Sagan

DRAFT UPDATED 6/8/2020 This page is Work In Progress- I am posting the scripting I use for compositing, cropping, and animating GOES received imagery using ImageMagick and FFMPEG.

GOES provides a lot of imagery!

The ABI can multitask. The default scan mode concurrently takes a full disk image every 10 minutes, an image of the contiguous U.S. every 5 minutes, and two smaller, more detailed images of areas where storm activity is present, every 60 seconds (or one every 30
seconds). Alternatively, ABI can run in full-disk mode, continuously imaging the full disk every 5 minutes.

ABI Scan Mode Demonstration

Now, with the ABI imagery capabilities listed, why is that on HRIT, we do not get those cool 10-minute full disk scans, but only get them every thirty minutes? The answer is bandwidth. There is a limited amount of data that can be sent, so we, who receive the HRIT and not GRB get a bit downgraded timing and imagery.

So we get all these cool images, but how to stitch them together into some semblance of animation? The first, and easiest way is through GIF animation. Take a bunch of timed pictures, assemble them together frame by frame and use a gif animator to make a “moving’ image. There are many online gif animation sites that are quite good like https://gifmaker.org/ or https://gifmaker.me/ But, because the images we get are 15 to 30 minutes apart. they can appear jerky or stuttered. (Imagine watching Star Wars at 1 frame every 30 minutes instead of one frame every 24 seconds!).

So here is an example of a full disk compiled into a gif animation, nice, but still kind of jerky. After all, each “snapshot” of the planet is taken every 30 minutes.

Gif animation of Earth

So how do we try and smooth this out? This is where magic comes in, in the form of ImageMagick and FFMPEG software. Below is an example of a gif animation turned into a smooth-flowing mp4 movie using a combination of Imagemagick and FFMPEG Scripting. The first script compiles all the frames, and then layers channel 15 over the full-color earth. The second script, using FFMEG, then takes the compiled gif animation and using a video filter called minterpolate, which performs motion interpolation. It’s a tool that can: Increase a video’s frame rate by rendering new frames based on motion (pixel-warping + blending). Make smooth motion videos (simple blending between frames) between the frames So basically it is filling in the 30-minute gap in full-disk imagery.

So, what’s the secret sauce that creates this magic?

I Use ImageMagick and FFMPEG on my Windows 10 Machines. What this does is allows me to write Batch Files (example.BAT) that I can run by double-clicking the chosen script in my “MagickScripts” Folder.

Since I do a lot of different types of animation, I divided my folder structure into many types of animations, such as F:/Fulldiskfile, F:/Conusfile, F:/CMIPFILE , etc. Under each one of these, I have subfolders where the program stores the temporary files used in creating the animations. As an example, one full disk animation with one or two overlays of ABI Band imagery might contain 4,500 separate images depending on the length of the capture!

Full Disk Animation


So here is one example of a ‘simple’ Full Disk animation, that does the following:

  1. Reduces the file size while maintaining the best balance of color and sharpness.
  2. Reduces the image size from an as-received image of 5424×5424 (that’s 75 inches wide and tall!) to 2000×2000 pixels
  3. It gives me the option to manually delete night time images (I did this for the animations that DO NOT have any layering of other bands).
  4. Adds my watermark (“branding”, you would have to create your own transparent png) to my image.
  5. Creates a GIF of all the reformatted images
  6. Then creates an Mp4 with interpolation of the frames.
  7. Then it deletes the temp files.
REM  FULL DISK ANIMATION

REM STEP #1 AND #2 - REDUCE FILESIZE
for /r "F:\Satellite Imagery\goes16\fd\NOMAP\" %%a in (*.jpg) do "C:\Program Files\ImageMagick-7.0.9-Q16\magick.exe" mogrify -path "F:\FULLDISKFILE\FULLDISK\AUTORESIZE" -filter Triangle -define filter:support=2 -thumbnail 2000 -unsharp 0.25x0.25+8+0.065 -dither None -posterize 136 -quality 82 -define jpeg:fancy-upsampling=off -define png:compression-filter=5 -define png:compression-level=9 -define png:compression-strategy=1 -define png:exclude-chunk=all -interlace none -colorspace sRGB -strip "%%~a"

REM STEP #3 OPTION TO DELETE NIGHT TIME IMAGERY
ECHO Manually delete night imagery in /FULLDISKFILE/FULLDISK/AUTORESIZE/?
PAUSE

REM STEP #4 ADD WATERMARK TO REDUCED FULL DISK IMAGE
magick mogrify -path "F:/FULLDISKFILE/FULLDISK/FDWATERMARKED/" -format jpg -gravity northwest -draw "image over 30,30 0,0 'F:/Satellite Imagery/GOESFULLDISKWHITE500x100.png'" "F:/FULLDISKFILE/FULLDISK/AUTORESIZE/*.jpg"

REM STEP #5 CREATE GIF
magick convert -loop 0 -delay 20 "F:\FULLDISKFILE\FULLDISK\FDWATERMARKED\GOES16_*.jpg" "F:/FULLDISKFILE/FULLDISK/FULLDISKFINAL/NOMAPout.gif"

REM STEP #6 CREATE MP4
ffmpeg -i "F:/FULLDISKFILE/FULLDISK/FULLDISKFINAL/NOMAPout.gif" -filter "minterpolate='mi_mode=blend'" -c:v libx264 -pix_fmt yuv420p "F:/FULLDISKFILE/FULLDISK/FULLDISKFINAL/NOMAPGOES.mp4"

REM STEP #7 DELETE TEMPORARY FILES
del "F:\FULLDISKFILE\FULLDISK\AUTORESIZE*." /s /f /q 
del "F:\FULLDISKFILE\FULLDISK\FDWATERMARKED*." /s /f /q
echo Your file is finished

NOTE: My file structure will obviously be different than your own! And also note that because this bat file is calling out ImageMagick and FFMPEG to run, you will see some commands in quotes “” and also note the difference in file structuring with the “/” vs. the “\” signs. They make a huge difference and need to be correct! Also take note that in steps 1 and 2 above I am using a Recursive For Loop to loop through the directories where necessary to modify every image inside every dated folder.

Imagine the for loop is being put “on pause” while the function (in this case reducing file size and reducing pixel size) call is running .

The new recursive call starts its for loop and again, pauses while calling the functions again, and so on, until every sib directory and file has been modified.

Once you have the basics down, lines of this code can be used in multiple scripts, I have 19 and counting! One for CONUS, one for Compositing, one for fulldisk, one for Meso, etc.

This bit of code I use in many of my scripts, its the one that does the file size reduction and pixel size reduction. My File size reducer and optimizer script (Note the OUTPUT_PATH, OUTPUT_WIDTH, and INPUT_PATH) :

REM FILE SIZE REDUCTION
mogrify -path OUTPUT_PATH -filter Triangle -define filter:support=2 -thumbnail OUTPUT_WIDTH -unsharp 0.25x0.25+8+0.065 -dither None -posterize 136 -quality 82 -define jpeg:fancy-upsampling=off -define png:compression-filter=5 -define png:compression-level=9 -define png:compression-strategy=1 -define png:exclude-chunk=all -interlace none -colorspace sRGB -strip INPUT_PATH

Here are the two other lines of code that take the images and first create the GIF, then the MP4 which is “smoothed out” through the minterpolate function of the script:

REM  CREATE GIF
magick convert -loop 0 -delay 20 "F:/INPUT_FILE_DIRECTORY_AND_FILENAME*.jpg" "F:/OUTPUT_FILE_DIRECTORY_AND_FILENAME/FILENAME_OF_GIF.gif"

REM  CREATE MP4
ffmpeg -i "F:/INPUT_FILE_DIRECTORY_AND_/FILENAME_OF_GIF.gif" -filter "minterpolate='mi_mode=blend'" -c:v libx264 -pix_fmt yuv420p "F:/OUTPUT_FILE_DIRECTORY_AND_FILENAME/FILENAME_OF_MP4.mp4/"

Compositing Full-Color Imagery with Band 15 Enhanced Imagery at 120FPS

Here is the latest result in my attempt to smooth out and interpolate the final video animations I create from GOES 16 Imagery. GOES 16 and 17 transmit HRIT imagery full disk every half hour or 2 frames every 3600 seconds. (2FPH (Frames Per Hour))
Compare that to a standard video that runs at 24 frames every second (24 FPS or 86400FPH).

The easiest method to stream the individual frames is to decrease the time in a GIF animation to about a 20th of a second between frames, rendering a shorter time span for the compiled gif and making the resulting gif “smoother but still jumpy”, after all, there is a half-hour gap in realtime between frames.

So, how to improve the transition between the frames?

Depending on the image source (M1, M2, Full Disks, what Band, etc.) I use a combination of three tools
Irfanview – to Date stamp individual frames if necessary
Image Magick – to crop, resize, rename, , watermark, compress, set color space and create the gif animation, compose, blend and composite
FFmpeg – To create the final mp 4 video through the use of the motion interpolation modes.

It is through the interpolation mode in FFmpeg, where most of the ‘Magick” happens. I tried using “Butterflow” but the results were not as smooth and it took hours to run on my computer. So I stuck with FFmpeg.
The FFmpeg script I wrote first takes the compiled gif animation and:

  1. Duplicates the previous and next frame for interpolating new frames.
  2. Adds Bidirectional motion estimation. Motion vectors are estimated for each source frame in both forward and backward directions.
  3. Adding Motion compensation mode. Using overlapped block motion compensation.
    1, 2, and 3 together create “new” filler frames based on the information in the original frames thereby filling the 30-minute gaps between frames with estimated imagery based on every previous and nest frame. (Whew!)
  4. The algorithm I chose which seems to work best for this type of motion estimation is the Diamond search algorithm.
  5. The video is also increased from the default of 60 FPS to 120 FPS and then rendered into a format that most video players can handle.

All of the above steps are performed in a single line of the script. And it is a balance between the time it takes to create the animation and the resulting file size.

Also, note that this scripting that I do is a work in progress, I intend to add sub-routines to create temporary directories, Irfanview subroutine scripting instead of manually running a batch process. and more.

120 FPS Interpolated and Composited animation, using script below.
Batch File for Creating a video of FC with my custom CLUT and Channel 15 Enhanced and a choice for creating a CONUS animation as well:
REM USRADIOGUY Build Imagery for Compositing FC and 15Enhanced 8/1/2020
REM MOVE AND RESIZE AND RENAME

for /r "F:\Satellite Imagery\goes16\fd\CUSTOMLUT\" %%a in (*.jpg) do "C:\Program Files\ImageMagick-7.0.9-Q16\magick.exe"  mogrify -path ../COMPOSITEFILE/compositeFC -filter Triangle -define filter:support=2 -thumbnail 1000 -unsharp 0.25x0.25+8+0.065 -dither None -posterize 136 -quality 82 -define jpeg:fancy-upsampling=off -define png:compression-filter=5 -define png:compression-level=9 -define png:compression-strategy=1 -define png:exclude-chunk=all -interlace none -colorspace sRGB -strip  -gravity south -stroke none -pointsize 24 -fill white -font Ailerons -annotate +0+0 GOES_16_FC_and_Channel_15 "%%~a"


for /r "F:\Satellite Imagery\goes16\fd\ch15_enhanced\" %%a in (*.jpg) do "C:\Program Files\ImageMagick-7.0.9-Q16\magick.exe"  mogrify -path ../COMPOSITEFILE/composite15 -filter Triangle -define filter:support=2 -thumbnail 1000 -unsharp 0.25x0.25+8+0.065 -dither None -posterize 136 -quality 82 -define jpeg:fancy-upsampling=off -define png:compression-filter=5 -define png:compression-level=9 -define png:compression-strategy=1 -define png:exclude-chunk=all -interlace none -colorspace sRGB -strip  -gravity south -stroke none -pointsize 24 -fill white -font Ailerons -annotate +0+0 GOES_16_FC_and_Channel_15 "%%~a"

@echo off
echo  CHECK FILE COUNT TO BE SURE IT"S EQUAL BEFORE CONTINUING IN FOLDER F:\COMPOSITEFILE\compositeFC\ AND F:\COMPOSITEFILE\composite15\ AND ADJUST IF NEEDED 


Pause

echo START COMPOSITING False Color into Channel_15 Enhanced

setlocal enabledelayedexpansion

set dirA=F:\COMPOSITEFILE\compositeFC\
set dirB=F:\COMPOSITEFILE\composite15\
set dirOut=F:\COMPOSITEFILE\compositedimages\

dir /b /on %dirA%*.jpg >listA.lis

set cntA=0
for /F "tokens=*" %%F in (listA.lis) do (
  set listA[!cntA!]=%%F
  set /A cntA+=1
)

echo There are %cntA% file in list A.


dir /b /on %dirB%*.jpg >listB.lis

set cntB=0
for /F "tokens=*" %%F in (listB.lis) do (
  set listB[!cntB!]=%%F
  set /A cntB+=1
)

echo There are %cntB% file in list B.

if not %cntA%==%cntB% (
  echo The lists have different numbers of files.
  exit /B 1
)

set /A lastN=%cntA%-1

for /L %%N in (0,1,%lastN%) do (
  echo %%N: composite %dirB%!listB[%%N]! and %dirA%!listA[%%N]!

  rem magick.exe composite -blend 50 %dirB%!listB[%%N]! %dirA%!listA[%%N]! %dirOut%!listA[%%N]!
  magick %dirA%!listA[%%N]! %dirB%!listB[%%N]! -define compose:args=55 -compose blend -composite %dirOut%!listA[%%N]!
  rem magick %dirA%!listA[%%N]! { %dirB%!listB[%%N]! -transparent black } -define compose:args=50 -compose blend -composite %dirOut%!listA[%%N]!
)

REM ADD WATERMARK TO THE REDUCED FULL DISK IMAGE
magick mogrify -path "F:/COMPOSITEFILE/compositefinal/" -format jpg -gravity northwest -draw "image over 15,15 0,0 'F:/Satellite Imagery/GOESFULLDISKWHITE250x50.png'" "F:/COMPOSITEFILE/compositedimages/*.jpg"

REM CREATE GIF
magick convert -loop 0 -delay 25 "F:\COMPOSITEFILE\compositefinal\GOES16_*" "F:/COMPOSITEFILE/compositefinal/compositeout.gif"

REM CREATE MP4
ffmpeg -i "F:/COMPOSITEFILE/compositefinal/compositeout.gif" -filter "minterpolate='mi_mode=blend:me_mode=bidir:fps=120:mc_mode=obmc:me=ds:vsbmc=1'" -c:v libx264 -pix_fmt yuv420p "F:/COMPOSITEFILE/compositefinal/compositeout120fps.mp4"

@ECHO OFF
:start
SET choice=
SET /p choice=Do you want to make a CONUS animation? [N]: 
IF NOT '%choice%'=='' SET choice=%choice:~0,1%
IF '%choice%'=='Y' GOTO yes
IF '%choice%'=='y' GOTO yes
IF '%choice%'=='N' GOTO no
IF '%choice%'=='n' GOTO no
IF '%choice%'=='' GOTO no
ECHO "%choice%" is not valid
ECHO.
GOTO start

:no
ECHO I guess no one gets to see the cool stuff!
del "F:\CONUSFILE\CONUSCROPPED\*.*" /s /f /q
del "F:\COMPOSITEFILE\composite15\*.*" /s /f /q
del "F:\COMPOSITEFILE\compositeFC\*.*" /s /f /q
del "F:\COMPOSITEFILE\compositedimages\*.*" /s /f /q
del "F:\COMPOSITEFILE\compositefinal\*.jpg" /s /f /q
PAUSE
EXIT

:yes
ECHO Creating CONUS now!
REM After using CompositingFCandch15.bat

REM CROP ALL IMAGES IN F:\Satellite Imagery\goes16\fd\NOMAP\
REM FOR 1000X1000 only!!! 
for /r "F:\COMPOSITEFILE\compositefinal\" %%a in (*.jpg) do "C:\Program Files\ImageMagick-7.0.9-Q16\magick.exe"  mogrify -path ../CONUSFILE/CONUSCROPPED -format jpg -crop 618x376+21+5 "%%~a"

REM FOR 2000X2000 only!!!
REM for /r "F:\COMPOSITEFILE\compositefinal\" %%a in (*.jpg) do "C:\Program Files\ImageMagick-7.0.9-Q16\magick.exe"  mogrify -path ../CONUSFILE/CONUSCROPPED -format jpg -crop 1133x625+235+55 "%%~a"


REM CREATE GIF
magick convert -loop 0 -delay 15 "F:\CONUSFILE\CONUSCROPPED\GOES16_*" "F:/CONUSFILE/CONUS/CONUSfinal/COMPOSITECONUSout.gif"

REM CREATE MP4 WITH INTERPOLATIN
ffmpeg -i "F:/CONUSFILE/CONUS/CONUSfinal/COMPOSITECONUSout.gif" -filter "minterpolate='mi_mode=blend:me_mode=bidir:fps=120:mc_mode=obmc:me=ds:vsbmc=1'" -c:v libx264 -pix_fmt yuv420p "F:/CONUSFILE/CONUS/CONUSFinal/COMPOSITECONUS120FPSout.mp4


REM DELETE ALL TEMP FILES
del "F:\CONUSFILE\CONUSCROPPED\*.*" /s /f /q
del "F:\COMPOSITEFILE\composite15\*.*" /s /f /q
del "F:\COMPOSITEFILE\compositeFC\*.*" /s /f /q
del "F:\COMPOSITEFILE\compositedimages\*.*" /s /f /q
del "F:\COMPOSITEFILE\compositefinal\*.jpg" /s /f /q
PAUSE
EXIT

Using a false color static image to ‘colorize’ a grey scale IR Image.

Sanchez brings your dull IR satellite images to life.

Matt Painter has just released Sanchez which can be downloaded from his GitHub repository. Having heard about it from him on Reddit I knew I had to give it a try for my dull GOES 17 IR Imagery (NOTE: it will work on Himawari, GK2A GOES 16, and GOES 17).

Utilizing a full color high resolution, static underlay image, and combining it with a single greyscale IR satellite image, uch as I get relayed from GOES 17, along with adding an optional mask and overlay and some coding magic, Sanchez will create beautiful images to be proud of.

Sanchez has been updated with:

  • Properly projected underlays
  • New high-resolution underlays for GK-2A and Himawari-8
  • Brand new underlays for GOES-16 and GOES-17
  • Batch conversion mode with glob support
  • Overlay support
  • Raspberry Pi support
  • Bug fixes

Here is the Batch File I wrote to use Sanchez on my windows 10 Machine.

"C:/Users/PATHTOSANCHEZ/Sanchez.exe" -s "F:\Satellite Imagery\goes17\fd\ch13" -m "C:\Users\PATHTOSANCHEZ\Resources\Mask.jpg" -u "C:\Users\PATHTOSANCHEZ\Resources\GOES-17\Underlay.jpg" -o "F:\GOES17COLOR" -t #0096FA

The resulting Image!