Layering Geo-Spatial Fire Data into Satellite Imagery

Layering Geo-Spatial Fire Data into Satellite Imagery

Global Composite 12-27-2021 With Global MODIS Fire/Hotspot data projected onto received and compiled map from 6 geo sats.

The VIIRS thermal activity layer can be used to visualize and assess wildfires worldwide. However, it should be noted that this dataset contains many “false positives” (e.g., oil/natural gas wells or volcanoes) since the satellite will detect any large thermal signal.

Even though these data display as point features, each point in fact represents a pixel that is >= 375 m high and wide. A point feature means somewhere in this pixel at least one “hot” spot was detected which may be a fire.

VIIRS is a scanning radiometer device aboard the Suomi NPP,NOAA-20 and NOAA-21 satellites that collects imagery and radiometric measurements of the land, atmosphere, cryosphere, and oceans in several visible and infrared bands. The VIIRS Thermal Hotspots and Fire Activity layer is a livefeed from a subset of the overall VIIRS imagery, in particular from NASA’s Near Real Time NRT active fire detection product. The downloads are automatically downloaded from LANCE, NASA’s near real time data and imagery site, every 15 minutes.

To visualize real-time fire and hotspot data on satellite imagery, I developed a script to automatically download current data from the FIRMS system every 12 hours in shapefile format. This data then needed to be converted to GeoJSON, a format compatible with the Goestools software used for overlaying geographic features on imagery.

Initially, I attempted to directly convert the shapefiles to GeoJSON using ogr2ogr, but the resulting format was incompatible with Goestools’ requirements. After numerous unsuccessful attempts, I sought expertise from geospatial researchers at UC Davis and Cornell. Their guidance was instrumental in resolving the issue.

By implementing a buffering technique on the fire hotspot points within the original FIRMS data, we were able to create small polygons. This modified data could then be successfully converted into a GeoJSON format that was accurately displayed on HRIT GOES, Himawari, GK-2A, and mesoscale imagery through Goestools.

Here are a few examples of the outputs:

GOES 17 showing the fires in July 2021 including the Bootleg Fire
Fires in S America
Fires in Quebec and Ontario

Fires in Australia from Himawari 8 imagery (relayed)

I have written the process up on my github page at https://github.com/creinemann/goestools-with-Fire-Data-Layer or you may follow the instructions below.

I have done it in a way that does not require you to edit your original goestools files, rather add some handlers to the system.

I hope you enjoy it!

Update 7-28-2021 Edouard Lafargue (@elafargue on github) Added some LINUX scripting to allow the fire layering process to be added and processed via the pi as well.

Scripts

These scripts allow goestools to use the built-in map handlers to apply fire data from

FIRMS Fire Information for Resource Management System US / Canada: https://firms.modaps.eosdis.nasa.gov/

I have a windows task scheduler created to run the batch file in this repository every 12 hours then convert that data to a format that can be processed by goestools. It then uploads the latest fire data json file to the pi I use to run goestools.

Within goestools I have a custom process that reads the imagery, applies geo-political boundaries, then applies the fire hotspot data. Since I do not intend to run the process all the time, I use the command as needed or seasonally.

Goestools by default utilizes a function that allows the software to apply map data from Natural Earth. Originals from https://www.naturalearthdata.com/.

However, the files needed to apply geo-spatial data from FIRMS is of a different format. The geo spatial data available for public download is generated in a points-based GIS geometry. Whereas goestools processes with a polygon, or multipolygon geometry.

Requirements

I run this script on a Windows 10 PC this could easily be adapted to run on an ARM board or similar computer

System dependencies:

Creating the neccesary files

Create a batch file using a text editor such as Notepad or better yet Notepad++ with the following script, changing the directories to match your own:

Update with MODIS FIRMS file access 09/07/2021

FIRMS has been overwhelmed with data access requests, and they have opened a mirror site to download data from, you can paste this address in the batch file below if you have issues downloading the data: https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Global_24h.zip

What this script does.

  1. The script changes the directory to the working folder, which is F:\FIREDATA in this case.
  2. The script removes an existing fire.json file. This is necessary because the ogr2ogr command-line tool does not allow overwriting existing files.
  3. The script downloads the 24-hour fire data from the NASA FIRMS website. The data is downloaded as a zip file, which is saved to the F:\FIREDATA\global.zip file.
  4. The script unzips the global.zip file to the VIIRSFIREDATA folder. This creates a folder structure that contains the following files:
    • J1_VIIRS_C2_Global_24h.shp
    • J1_VIIRS_C2_Global_24h.dbf
    • J1_VIIRS_C2_Global_24h.prj
    • J1_VIIRS_C2_Global_24h.shx
  5. The script converts the MODIS_C6_1_Global_24h.shp shapefile to fire.json using the ogr2ogr command-line tool. The ogr2ogr tool is a command-line tool that can be used to convert between different spatial data formats. In this case, the ogr2ogr tool is being used to convert the MODIS_C6_1_Global_24h.shp shapefile to the GeoJSON format. The GeoJSON format is a lightweight open standard for representing geographic data.
  6. The script uploads the fire.json file to the Goestools server using the WinSCP command-line tool. The WinSCP command-line tool is a command-line tool that can be used to transfer files between a local computer and a remote server. In this case, the WinSCP tool is being used to upload the fire.json file to the Goestools server.
  7. The script displays a message indicating whether the upload was successful or not.
::Batch file to download, unzip, convert geospatial data to polygons, and upload to goestools
::  DEPENDECIES: 
:: 	POWERSHELL
::	OGR2OGR
::	WINSCP
::	GOESTOOLS

:: Change Directory to working folder

	cd C:\USERDIRECTORY
	
:: REMOVE EXISTING fire.json FILE (OGR2OGR DOES NOT ALLOW OVERWRITING FILE.)

	del "F:\FIREDATA\fire.json" /s /f /q
	
:: DOWNLOAD THE 24 HOUR FIRE DATA
:: NOTE THIS IS GLOBAL DATA, A USA (Conterminous) and Hawaii FIRE DATA FILE CAN BE SUBSTITUTED FOR THE ONE IN THE NEXT LINE
:: THAT LINK IS https://firms.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_USA_contiguous_and_Hawaii_24h.zip
	
	powershell -Command Invoke-WebRequest https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Global_24h.zip -OutFile C:\USERDIRECTORY\global.zip
	
:: UNZIP THE ARCHIVE

	powershell Expand-Archive C:\USERDIRECTORY\global.zip -DestinationPath C:\USERDIRECTORY -force
	
:: CONVERT THE SHAPEFILE MODIS_C6_1_Global_24h.shp TO fire.json

	ogr2ogr -f "GeoJSON" -dialect SQLite -sql "select ST_Buffer(geometry,0.01) from MODIS_C6_1_Global_24h" fire.json MODIS_C6_1_Global_24h.shp


@echo off

:: UPLOAD THE MOST RECENT fire.json TO GOESTOOLS

:: NOTE: Winscp can auto generate this script by runnung winscp GUI looging into your pi, then
:: select 'Session' 'Generate URL/code"

"C:\Program Files (x86)\WinSCP\WinSCP.com" ^
  /log="C:\USERDIRECTORY\FireDatausradioguy.log" /ini=nul ^
  /command ^
    "open sftp://piusername:piuserpassword@pi_IP adress/ -hostkey=""your pi host key""" ^
    "put -latest ""C:\USERDIRECTORY\fire.json" " ^

    "exit"

set WINSCP_RESULT=%ERRORLEVEL%
if %WINSCP_RESULT% equ 0 (
  echo Success
) else (
  echo Error
)

exit /b %WINSCP_RESULT%

Save the file as a batch script, such as shp2json.bat

I run the batch automatically by calling it from within windows using the Windows Task scheduler. https://docs.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page

The script can also be run as needed by running the batch file.

Creating the New Goestools Process

Instead of editing goesr-goesproc.conf file within goestools I chose to write a short handler that, again, can be run as needed (during fire season).

Create a .conf file using a text editor such as Notepad or better yet Notepad++. This process creates imagery using bands 02 and 13

The output files are saved in the usual fashion, but under a ‘fire’ directory under M1, M2 and FD directories. You can change this to work with different bands if needed. This will generate Full-Color Imagery, with the fire hotspots appearing as Orange dots or clusters Insert the following into the file:

# GOES-16 ABI FIRE HOTSPOT LAYERING
# FIRE DATA FROM https://firms.modaps.eosdis.nasa.gov/usfs/
[[handler]]
type = "image"
origin = "goes16"
regions = [ "fd", "m1", "m2" ]
channels = [ "ch02", "ch13" ]
directory = "./goes16/{region:short|lower}/fire/{time:%Y-%m-%d}"
filename = "GOES16_{region:short}_FCFIREMAP_{time:%Y%m%dT%H%M%SZ}"
format = "jpg"
json = false

  [handler.remap.ch02]
  path = "/usr/share/goestools/wxstar/wxstar_goes16_ch02_curve.png"

  [handler.lut]
  path = "usr/share/goestools/wxstar/wxstar_goes16_lut.png"

  [[handler.map]]
  path = "/usr/share/goestools/ne/ne_50m_admin_0_countries_lakes.json"

  [[handler.map]]
  path = "/usr/share/goestools/ne/ne_50m_admin_1_states_provinces_lakes.json"

  [[handler.map]]
  path = "/home/pi/fire.json"
  color = "#FF6700"

NOTE: You will need to change the ‘origin’, ‘directory’, and ‘filename’ to 18 to work with GOES 18.

Save the file as goesfireproc.conf

Copy the file to /home/pi/

Make sure you sudo reboot the pi to enable the new handler,

Then use the command

goesproc -c goesfireproc.conf -m packet --subscribe tcp://127.0.0.1:5004 --out /home/pi/goes

On the pi to start the process.

Future Ideas

Use the plotting function to maybe plot hurricanes, cyclones, etc using geospatial data.

Acknowledgments

Thanks to Pieter Noordhuis (https://github.com/pietern/goestools) for building an open-source utility -goestools- for receiving, and decoding signals from GOES satellites.

Thanks to Keith Jenkins from Cornell University GIS & Geospatial Applications Librarian for assistance with the ogr2ogr conversion.

Thanks to Dr. Michele Tobias Geospatial Data Specialist, UC Davis Library for assistance on testing with QGIS software.

Regional Settings

If you do not want full global data you can substitute the following links for regional coverage (as shown on map).

You need to update the script portion shown below with the new links (shown with BOLD) and shapefile names.

::Batch file to download, unzip, convert geospatial data to polygons, and upload to goestools

	
	powershell -Command Invoke-WebRequest https://firms.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Global_24h.zip -OutFile C:\USERDIRECTORY\global.zip
	
:: UNZIP THE ARCHIVE

	powershell Expand-Archive C:\USERDIRECTORY\global.zip -DestinationPath C:\USERDIRECTORY -force
	
:: CONVERT THE SHAPEFILE MODIS_C6_1_Global_24h.shp TO fire.json

	ogr2ogr -f "GeoJSON" -dialect SQLite -sql "select ST_Buffer(geometry,0.01) from MODIS_C6_1_Global_24h" fire.json MODIS_C6_1_Global_24h.shp
World24h   https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Global_24h.zip
Canada24h  https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Canada_24h.zip
Alaska24h   https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Alaska_24h.zip
USA (Conterminous) and Hawaii24h  https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_USA_contiguous_and_Hawaii_24h.zip
Central America24h   https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Central_America_24h.zip
South America24h   https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_South_America_24h.zip
Europe24h   https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Europe_24h.zip
North and Central Africa24h   https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Northern_and_Central_Africa_24h.zip
Southern Africa24h  https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Southern_Africa_24h.zip
Russia and Asia24h  https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Russia_Asia_24h.zip
South Asia24h  https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_South_Asia_24h.zip
South East Asia24h   https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_SouthEast_Asia_24h.zip
Australia and New Zealand 24h https://firms2.modaps.eosdis.nasa.gov/data/active_fire/modis-c6.1/shapes/zips/MODIS_C6_1_Australia_NewZealand_24h.zip