Layering Geo-Spatial Fire Data into Satellite Imagery

Global Composite 12-27-2021 With Global MODIS Fire/Hotspot data projected onto received and compiled map from 6 geo sats.

I created a script that downloads current fire and hotspot geo-spatial data from FIRMS (Fire Information and Resource Management System) every 12 hours as a shapefile format. Goestools uses a geojson format to place Geo-spatial shapes, like country borders, etc. onto the imagery. I needed a way to convert the .shp files to .json files automatically.

Through a lot of trial and error, I settled upon using a piece of software called ogr2ogr. The initial conversion was quite simple, but alas not the right format.

After a LOT of trials and many errors I called in two PhD’s from UC Davis and Cornell Universities Geo-spatial research sections. After explaining my predicament they were able to come up with a relatively painless way for me to convert the original FIRMS file to what the software required by adding buffering to the points of the fire hotspots, thereby creating a small polygon that could be displayed accurately on an HRIT GOES (and Himawari and GK-2A) image or mesoscale imagery.

Here are a few examples of the outputs:

GOES 17 showing the fires in July 2021 including the Bootleg Fire

Fires in Australia from Himawari 8 imagery (relayed)

I have written the process up on my github page at https://github.com/creinemann/goestools-with-Fire-Data-Layer or you may follow the instructions below.

I have done it in a way that does not require you to edit your original goestools files, rather add some handlers to the system.

Hope you enjoy it!

Update 7-28-2021 Edouard Lafargue (@elafargue on github) Added some LINUX scripting to allow the fire layering process to be added and processed via the pi as well.

Scripts

These scripts allow goestools to use the built-in map handlers to apply fire data from

FIRMS Fire Information for Resource Management System US / Canada: https://firms.modaps.eosdis.nasa.gov/

I have a windows task scheduler created to run the batch file in this repository every 12 hours then convert that data to a format that can be processed by goestools. It then uploads the latest fire data json file to the pi I use to run goestools.

Within goestools I have a custom process that reads the imagery, applies geo-political boundaries, then applies the fire hotspot data. Since I do not intend to run the process all the time, I use the command as needed or seasonally.

Goestools by default utilizes a function that allows the software to apply map data from Natural Earth. Originals from https://www.naturalearthdata.com/.

However, the files needed to apply geo-spatial data from FIRMS is of a different format. The geo spatial data available for public download is generated in a points-based GIS geometry. Whereas goestools processes with a polygon, or multipolygon geometry.

Requirements

I run this script on a Windows 10 PC this could easily be adapted to run on an ARM board or similar computer

System dependencies:

GOESTOOLS https://github.com/pietern/goestools
POWERSHELL https://docs.microsoft.com/en-us/powershell/?view=powershell-7.1
OGR2OGR https://gdal.org/programs/ogr2ogr.html#ogr2ogr
WINSCP https://winscp.net/eng/index.php

Creating the neccesary files

Create a batch file using a text editor such as Notepad or better yet Notepad++ with the following script, changing the directories to match your own:

Update with MODIS FIRMS file access 09/07/2021

FIRMS has been overwhelmed with data access requests, and they have opened a mirror site to download data from, you can paste this address in the batch file below if you have issues downloading the data: https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Global_24h.zip

What this script does.

The script changes the directory to the working folder, which is F:\FIREDATA in this case.
The script removes an existing fire.json file. This is necessary because the ogr2ogr command-line tool does not allow overwriting existing files.
The script downloads the 24-hour fire data from the NASA FIRMS website. The data is downloaded as a zip file, which is saved to the F:\FIREDATA\global.zip file.
The script unzips the global.zip file to the VIIRSFIREDATA folder. This creates a folder structure that contains the following files:
- J1_VIIRS_C2_Global_24h.shp
- J1_VIIRS_C2_Global_24h.dbf
- J1_VIIRS_C2_Global_24h.prj
- J1_VIIRS_C2_Global_24h.shx
The script converts the MODIS_C6_1_Global_24h.shp shapefile to fire.json using the ogr2ogr command-line tool. The ogr2ogr tool is a command-line tool that can be used to convert between different spatial data formats. In this case, the ogr2ogr tool is being used to convert the MODIS_C6_1_Global_24h.shp shapefile to the GeoJSON format. The GeoJSON format is a lightweight open standard for representing geographic data.
The script uploads the fire.json file to the Goestools server using the WinSCP command-line tool. The WinSCP command-line tool is a command-line tool that can be used to transfer files between a local computer and a remote server. In this case, the WinSCP tool is being used to upload the fire.json file to the Goestools server.
The script displays a message indicating whether the upload was successful or not.

::Batch file to download, unzip, convert geospatial data to polygons, and upload to goestools
::  DEPENDECIES: 
:: 	POWERSHELL
::	OGR2OGR
::	WINSCP
::	GOESTOOLS

:: Change Directory to working folder

	cd C:\USERDIRECTORY
	
:: REMOVE EXISTING fire.json FILE (OGR2OGR DOES NOT ALLOW OVERWRITING FILE.)

	del "F:\FIREDATA\fire.json" /s /f /q
	
:: DOWNLOAD THE 24 HOUR FIRE DATA
:: NOTE THIS IS GLOBAL DATA, A USA (Conterminous) and Hawaii FIRE DATA FILE CAN BE SUBSTITUTED FOR THE ONE IN THE NEXT LINE
:: THAT LINK IS https://firms.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_USA_contiguous_and_Hawaii_24h.zip
	
	powershell -Command Invoke-WebRequest https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Global_24h.zip -OutFile C:\USERDIRECTORY\global.zip
	
:: UNZIP THE ARCHIVE

	powershell Expand-Archive C:\USERDIRECTORY\global.zip -DestinationPath C:\USERDIRECTORY -force
	
:: CONVERT THE SHAPEFILE MODIS_C6_1_Global_24h.shp TO fire.json

	ogr2ogr -f "GeoJSON" -dialect SQLite -sql "select ST_Buffer(geometry,0.01) from MODIS_C6_1_Global_24h" fire.json MODIS_C6_1_Global_24h.shp


@echo off

:: UPLOAD THE MOST RECENT fire.json TO GOESTOOLS

:: NOTE: Winscp can auto generate this script by runnung winscp GUI looging into your pi, then
:: select 'Session' 'Generate URL/code"

"C:\Program Files (x86)\WinSCP\WinSCP.com" ^
  /log="C:\USERDIRECTORY\FireDatausradioguy.log" /ini=nul ^
  /command ^
    "open sftp://piusername:piuserpassword@pi_IP adress/ -hostkey=""your pi host key""" ^
    "put -latest ""C:\USERDIRECTORY\fire.json" " ^

    "exit"

set WINSCP_RESULT=%ERRORLEVEL%
if %WINSCP_RESULT% equ 0 (
  echo Success
) else (
  echo Error
)

exit /b %WINSCP_RESULT%

Save the file as a batch script, such as shp2json.bat

I run the batch automatically by calling it from within windows using the Windows Task scheduler. https://docs.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page

The script can also be run as needed by running the batch file.

Creating the New Goestools Process

Instead of editing goesr-goesproc.conf file within goestools I chose to write a short handler that, again, can be run as needed (during fire season).

Create a .conf file using a text editor such as Notepad or better yet Notepad++. This process creates imagery using bands 02 and 13

The output files are saved in the usual fashion, but under a ‘fire’ directory under M1, M2 and FD directories. You can change this to work with different bands if needed. This will generate Full-Color Imagery, with the fire hotspots appearing as Orange dots or clusters Insert the following into the file:

# GOES-16 ABI FIRE HOTSPOT LAYERING
# FIRE DATA FROM https://firms.modaps.eosdis.nasa.gov/usfs/
[[handler]]
type = "image"
origin = "goes16"
regions = [ "fd", "m1", "m2" ]
channels = [ "ch02", "ch13" ]
directory = "./goes16/{region:short|lower}/fire/{time:%Y-%m-%d}"
filename = "GOES16_{region:short}_FCFIREMAP_{time:%Y%m%dT%H%M%SZ}"
format = "jpg"
json = false

  [handler.remap.ch02]
  path = "/usr/share/goestools/wxstar/wxstar_goes16_ch02_curve.png"

  [handler.lut]
  path = "usr/share/goestools/wxstar/wxstar_goes16_lut.png"

  [[handler.map]]
  path = "/usr/share/goestools/ne/ne_50m_admin_0_countries_lakes.json"

  [[handler.map]]
  path = "/usr/share/goestools/ne/ne_50m_admin_1_states_provinces_lakes.json"

  [[handler.map]]
  path = "/home/pi/fire.json"
  color = "#FF6700"

NOTE: You will need to change the ‘origin’, ‘directory’, and ‘filename’ to 17 to work with GOES 17.

Save the file as goesfireproc.conf

Copy the file to /home/pi/

Make sure you sudo reboot the pi to enable the new handler,

Then use the command

goesproc -c goesfireproc.conf -m packet --subscribe tcp://127.0.0.1:5004 --out /home/pi/goes

On the pi to start the process.

Future Ideas

Use the plotting function to maybe plot hurricanes, cyclones, etc using geospatial data.

Acknowledgments

Thanks to Pieter Noordhuis (https://github.com/pietern/goestools) for building an open-source utility -goestools- for receiving, and decoding signals from GOES satellites.

Thanks to Keith Jenkins from Cornell University GIS & Geospatial Applications Librarian for assistance with the ogr2ogr conversion.

Thanks to Dr. Michele Tobias Geospatial Data Specialist, UC Davis Library for assistance on testing with QGIS software.

Regional Settings

If you do not want full global data you can substitute the following links for regional coverage (as shown on map).

You need to update the script portion shown below with the new links (shown with BOLD) and shapefile names.

::Batch file to download, unzip, convert geospatial data to polygons, and upload to goestools

	
	powershell -Command Invoke-WebRequest https://firms.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Global_24h.zip -OutFile C:\USERDIRECTORY\global.zip
	
:: UNZIP THE ARCHIVE

	powershell Expand-Archive C:\USERDIRECTORY\global.zip -DestinationPath C:\USERDIRECTORY -force
	
:: CONVERT THE SHAPEFILE MODIS_C6_1_Global_24h.shp TO fire.json

	ogr2ogr -f "GeoJSON" -dialect SQLite -sql "select ST_Buffer(geometry,0.01) from MODIS_C6_1_Global_24h" fire.json MODIS_C6_1_Global_24h.shp

World	24h	https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Global_24h.zip
Canada	24h	https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Canada_24h.zip
Alaska	24h	https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Alaska_24h.zip
USA (Conterminous) and Hawaii	24h	https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_USA_contiguous_and_Hawaii_24h.zip
Central America	24h	https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Central_America_24h.zip
South America	24h	https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_South_America_24h.zip
Europe	24h	https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Europe_24h.zip
North and Central Africa	24h	https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Northern_and_Central_Africa_24h.zip
Southern Africa	24h	https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Southern_Africa_24h.zip
Russia and Asia	24h	https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Russia_Asia_24h.zip
South Asia	24h	https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_South_Asia_24h.zip
South East Asia	24h	https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_SouthEast_Asia_24h.zip
Australia and New Zealand	24h	https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Australia_NewZealand_24h.zip