goesproc¶
Process stream of packets (VCDUs) or list of LRIT files.
Options¶
|
Path to configuration file |
|
Process stream of VCDU packets or pre-assembled LRIT files |
|
Address of nanomsg publisher |
|
Overwrite existing output files |
If mode is set to packet, goesproc reads VCDU packets from the
specified path(s). To process real time data you can either setup a pipe
from the decoder into goesproc (e.g. use /dev/stdin as path argument),
or use --subscribe to consume packets directly from goesrecv.
To process recorded data you can specify a list of files that contain
VCDU packets in chronological order.
If mode is set to lrit, goesproc finds all LRIT files in the specified
paths and processes them sequentially. You can specify a mix of files
and directories. Directory arguments expand into the files they
contain that match the glob *.lrit*. The complete list of LRIT files
is sorted according to their time stamp header prior to processing it.
Configuration¶
The configuration file uses TOML syntax. You can
find an example configuration (that includes stanzas for every
available handler) in ./etc/goesproc.conf.
Handlers¶
Different products use different handlers with different configuration options. For example, there is an NWS image handler, NWS text handler, EMWIN handler, GOES N-series handler, GOES R-series handler, Himawari handler, etc.
TODO – describe configuration options for every handler
Example¶
For example, with the following configuration file:
[[handler]]
type = "image"
product = "goes15"
region = "fd"
channels = [ "VS" ]
crop = [ -2373, 2371, -1357, 1347 ]
filename = "GOES15_%r_%c_%t"
Running goesproc against a directory with GOES-15 LRIT files results in a number of image files of the full disk visual channel:
$ goesproc --config example.conf --mode lrit ./out/images/goes15/fd
Writing ./GOES15_FD_VS_20170820-210600.png
Writing ./GOES15_FD_VS_20170821-000600.png
Writing ./GOES15_FD_VS_20170821-030600.png
Writing ./GOES15_FD_VS_20170821-060600.png
Writing ./GOES15_FD_VS_20170821-090600.png
Writing ./GOES15_FD_VS_20170821-120600.png
Writing ./GOES15_FD_VS_20170821-150600.png
Writing ./GOES15_FD_VS_20170821-180600.png
...
You can now use these image files however you like. For example, to produce a GIF from 8 consecutive full disk images, you can use the following ImageMagick commands:
$ mogrify -resize '640x480>' *.png
$ convert -loop 0 -delay 50 *.png GOES15_FD_VS_20170821.gif
And get:
Sample configuration¶
# Example configuration file for goesproc
#
# This tool is designed to run on streaming data (live or recorded)
# and product whatever is listed in this file. A single product can be
# processed multiple times (e.g. with different contrast curves,
# different scale, or different annotations) by listing multiple
# handlers for that same product.
#
# GOES-16 mesoscale region 1 imagery is stored at ./goes16/m1/YYYY-MM-DD
# The pattern specified in {time:XXX} is extrapolated using strftime(3).
# It can be used more than once if needed.
[[handler]]
type = "image"
origin = "goes16"
region = "m1"
dir = "./goes16/m1/{time:%Y-%m-%d}"
# GOES-16 full disk originals.
[[handler]]
type = "image"
origin = "goes16"
region = "fd"
dir = "./goes16/fd/{time:%Y-%m-%d}"
# GOES-16 full disk, channel 2, with contrast curve applied.
# The section [handler.remap] below applies to this handler.
[[handler]]
type = "image"
origin = "goes16"
region = "fd"
channels = [ "ch02" ]
directory = "./goes16/fd/{time:%Y-%m-%d}"
filename = "{filename}_contrast"
# Check out https://github.com/hdoverobinson/wx-star_false-color for a
# GOES-16 contrast curve and false color LUT. Uncomment the snippet
# below and update the path to point to the contrast curve file.
[handler.remap.ch02]
path = "./wx-star.com_GOES-R_ABI_C02_contrast-curve.pgm"
# GOES-16 full disk, channel 2 and 13 to produce a false color image.
# The sections [handler.remap] and [handler.lut] below apply to this handler.
[[handler]]
type = "image"
origin = "goes16"
region = "fd"
channels = [ "ch02", "ch13" ]
directory = "./goes16/false-color/fd/{time:%Y-%m-%d}"
[handler.remap.ch02]
path = "./assets/wxstar_goes16_ch02_curve.png"
[handler.lut]
path = "./assets/wxstar_goes16_lut.png"
# The raw GOES-15 full disk images are a mix of left
# and right aligned images. The image headers contain
# navigational data and we can use this to produce
# identical crops regardless of alignment.
[[handler]]
type = "image"
origin = "goes15"
region = "fd"
crop = [ -2373, 2371, -1357, 1347 ]
directory = "./goes15/fd/{time:%Y-%m-%d}"
filename = "GOES15_{region:short}_{channel:short}_{time:%Y%m%dT%H%M%SZ}"
# The HRIT stream on the GOES-R series includes EMWIN data.
# This handler writes EMWIN files and decompresses them if applicable
# (a subset of EMWIN files is compressed as ZIP file).
[[handler]]
type = "emwin"
directory = "./emwin/{time:%Y-%m-%d}"
# NWS text messages
#
# The AWIPS Product Identifier of the message can be used in the
# filename pattern. See https://www.weather.gov/tg/awips for a
# description of the fields in this identifier. They are usable
# through the following keywords (in "{awips:...}"):
#
# * t1t2
# * a1a2
# * ii
# * cccc
# * yy
# * gggg
# * bbb
# * nnn
# * xxx
#
[[handler]]
type = "text"
origin = "nws"
directory = "./nws/{time:%Y-%m-%d}"
filename = "{time:%Y%m%dT%H%M%SZ}_{awips:nnn}{awips:xxx}"
# NWS images
#
# Files received through HRIT do not have an LRIT timestamp header and
# goesproc tries to parse their timestamp from their file name. This
# format is not specified anywhere so it might change at any point in
# time. If the timestamp cannot be parsed from the file name, goesproc
# defaults to 1970-01-01. @pietern pinged NOAA (Seth) about this.
#
# NWS image files received through LRIT (GOES-15) have a timestamp.
#
[[handler]]
type = "image"
origin = "nws"
directory = "./nws/{time:%Y-%m-%d}"
filename = "{time:%Y%m%dT%H%M%SZ}_{filename}"
# Miscellaneous text.
[[handler]]
type = "text"
origin = "other"
directory = "./text/{time:%Y-%m-%d}"
filename = "{time:%Y%m%dT%H%M%SZ}_{filename}"