The following applies to all programs in the suite:
Vagabond programs are cross-platform command-line programs used to create and search a database of GPS track logs. Their binary executables are available for download from this site in three major versions (Linux, MacOSX and Windows). The files they create or operate on (specifically, the binary-format .trax files) can be freely moved between the systems, provided their names do not violate the naming conventions of the operating system to which they are transferred (i.e., do not use spaces and use only Latin alphanumerics in the file and directory names!) Keep also in mind that on some systems file and directory names are case-sensitive, and on others they are not).
The programs are "command-line" programs. The command line will usually
(with the exception of the newlineGpx, see below) consist, in addition to
the program name, of one or two arguments specifying the names of input
and output files and a number of "flags" that control in more detail the
execution of the program. If the command line arguments contain spaces or
special characters, they must be given as quote-strings. Flags, unlike
arguments (note that flags are NOT considered "command line arguments")
must start with a "-" (dash) character, followed by the flag-type letter,
optionally followed by an "=" sign, optionally followed by some numeric
or alphanumeric value (which can optionally be a quote-string) associated
with the flag. While the command line arguments are "positional" (first,
second...) the flags can occur anywhere on the line, in any order. One or
more space characters separate the components (i.e., program name, flags,
arguments) of the command line. An example of such command line might look
something like:
progAbcDef /dir/sub_dir/inputFile.xyz -m -t=123 outputFile.tuv -n="string fgh"
Several Vagabond programs can read and perform some required action on either a single file, or on all files in one directory ("folder") with a specific, context dependent extension (typically .gpx or .jpg, in either upper or lower case). When the name that can represent either a single file, or all such files in one directory is given as a command-line argument, the following important rule must be observed:
When the argument ends with a directory separator character ("\" in DOS/Windows, "/" in Unix/Linux/MacOSX) it will be assumed that it is the name of a directory, otherwise it will be assumed it is the name of a single file.
The time-stamp strings in text files produced by GPS devices will almost universally be UTC times in ISO 8601 format, i.e., "2008-11-05T13:15:30Z" for November 5, 2008, 14:15:30 ("2 PM.."). Times written by camera in exif data of a jpg file will as a rule include no indication of the time zone and will look something like "2008-11-05 13:15:30".
The programs that produce text files will write the text in a series of output lines to "standard output"; normally the console or "terminal window". Shell re-direction (">" and ">>" characters followed by a file name) can be used to capture such text in a disk file for later processing (printing, sorting, editing or to be used as input to another program).
Trax database (a file that usually has ".trax" extension) is a central file in the Vagabond suite, and, since its format is fully documented, it might be operated on by other programs and applications. It is a binary-format file, designed to efficiently store a very large number of tracks and their associated points and make possible quick searches in this collection. When this file is created, the name of the file is commonly given as the second command-line argument (the first argument typically specifies the input used for their creation). When a .trax file is specified to a program that creates a textual representation of it, it will be the first argument, and when it is given to a program that performs searches in it the file name will be specified as a text value by an -x command line flag (e.g., -x=myTravels_2010.trax). All programs will respond to a command line with no arguments, but one of two flags: -h to output a short help text, and -v to output the program version information.
dumpGpx program is used to extract data from a .gpx file created by or downloaded from a GPS device. Extracted text data is written to the standard output in one-item-per-line format. First and only command line argument is either the name of .gpx file, or a directory with a number of .gpx files to be processed in sequence. Items are written in the order in which they occur in the file(s), but the order in which multiple files are fetched from the directory is undefined.
The program can extract the following items:
An example of a command line invoking dumpGpx program might be:
dumpGpx -w \myLogs\logs2010\gps60_abc.gpx > waypoints.list
(to extract waypoints in simple text lines from the GPS file, gps60_2010_23.gpx, and save the output in waypoints.list file).
verifyGpx program traverses all .gpx files in a directory (a presumed future input to the gpxToTrax program) and detects and reports instances vhere two .gpx files cover the same time period.
Trax database represents the time/location log of a single GPS logging device. Time overlap of logs in two different files is only acceptable if the movements one object (person, vehicle...) is logged interchangably with more than one device, but at different times. If this was the case, time overlap between files is will not result in (undesirable) time overlap of tracks in .trax datase. This can be verified by runnig dumpTrax with -o flag.
An example follows:
verifyGpx \myGpsLogs\2008\
(To detect and report time overlaps between GPS logs in all .gpx files found in ...\2008\ directory).
newlineGpx program is an exception: like all the other programs here, it is a "command line" program, but unlike those, it takes no arguments. It is a simple text "filter" program that reads from the standard input and writes to the standard output. It modifies the input text by adding a newline character before each <trk..., <trkseg... and <trkpt... tag in a .gpx file. This is useful if a .gpx file produced by a GPS device that does not separate track points into separate lines needs to be manually edited.
The input and output are commonly redirected from and to a disk file using the common < and > command line notation. For instance:
newlineGpx < trackXYZ.gpx > trackXYZ_lines.gpx
(To create a modified copy of trackXYZ.gpx file).
gpxToTrax program creates the central file of this suite of programs: Trax Database (.trax file).
The Trax database consists of all the tracks (and their points) that the program was presented with on input, each stored complete with its time-ordered list of points that constitute the track. It is assumed that tracks do not overlap in time. One file may contain a large number of tracks, possibly covering a period of several weeks, months or even years. Storage of one point (location, elevation and time) takes in the order of 20 characters, yet the format in which it is stored makes possible very fast searching. As an illustration, if a device was generating GPS movement log around the clock, with the average density of one point every 5 seconds, the resulting .trax file would be only about 10 megabytes for one month, and between 100 and 150 megabytes for the whole year.
Input to the program consists of a single .gpx file or all .gpx files residing in the same directory. The program assumes that all track points in one .gpx file are ordered in time, and that any gap of more than five minutes means that a recording was turned off and a new track started at the time of the first recorded point following the time gap. Since the program ignores all content of .gpx file other than time-ordered sequence of track points, the input may be not only a .gpx file produced by a GPS device, but also text file "manufactured" by some process, as long as the format is the same as the output of a dumpGpx program (in ".gpx-like" format, flag -g, see above), as long as the points in each such file are ordered by time.
The program takes two command line arguments: first is the .gpx file or the name of a directory with a number of gpx files that all together constitute the input. The second argument is the name of the .trax file to be created. An example follows:
gpxToTrax \myGpsLogs\2008\ \travels\2008\y2008.trax
(To produce a .trax file consisting of all GPS log recordings (.gpx files) found in ...\2008\ directory and create y2008.trax database file).
dumpTrax program writes to standard output a textual representation of a .trax file. The program takes a single command line argument, the name of the .trax file to "dump".
By default, the program creates a textual representation of the file header and the track and point database records. This data, captured in a text file, can be used to verify the correct internal structure of the file, perhaps by comparing it to the dump of input .gpx files, used to create the database.
Alternatively, the output can be modified with a -g flag. In this case the output text will consists of time-ordered track point records in a format that mimics a .gpx file and track points in it. While such file will still not have all the information that could be expected in a GPS device produced .gpx file, it can be used as one of the .gpx input files in a subsequent run of the gpxToTrax program. This provides a method for expanding an existing .trax database with newly obtained GPS tracks/logs.
The internal consistency of tracks in the file can be verified by running the program with -o. This will perform a traverse of all tracks, and ckeck for and report their instances of time overlap between tracks. The overlap must be avoided since the determination of a location of an event the time of which is covered by two (or more) tracks is undetermined.
exifDataList program is used to extract one of a number of different meta-data items from the exif section of a .jpg file or all .jpg files in a single directory. The extracted item is written to standard output, one line for each input file. The line consists of the file name, written as a quote-string and the item.
The program can extract the following exif metadata items:
In this system, the intended use of exifDataList program is to prepare a list of .jpg file names with the times when the picture was taken. This list can then be used to find the location where the picture was taken, by searching the database of GPS logs (i.e., .trax file) for the location of the device (GPS and camera, assuming they are in the reasonable proximity of each other) at the time recorded by the camera.
Note however that the times in .gpx files (created by the GPS receiver) will be UTC times, while the time that the camera writes in the image file will depend on the camera clock setting and, in some cases will also depend on camera features. It is therefore suggested to set the camera clock to UTC time. Alternatively, the difference between two clocks can be supplied to the program doing the search. (cf. exifTimeDelta and findLocations programs). Note also that times in .gpx files will be very precise, while the camera clocks are typically much less precise and drift over time.
In most cases, the output file will not be ordered on the filenames. If the .jpg names are camera-generated, this is the order in which the pictures were taken. If desired, the file might be sorted by "piping" it through the operating system "sort" program. For instance:
exifDataList \myPictures\january06\ | sort > \work\jan06times.list
(To extract the exif timestamps from all .jpg files in \myPictures\january06\ directory, and capture the text file consisting of file names and time-stamps sorted by jpg file names into jan06times.list file in the \work\ directory).
exifTimeDelta program can be used to conveniently determine the difference between UTC time and the camera clock if the UTC time of an image is known by some method independent of the camera clock. (A simple method to obtain this is by taking a picture of the GPS device display which shows UTC time or, if the GPS will not display UTC time, local GPS time and knowledge of the time zone to which the GPS device is set).
The time difference is reported in the same format as expected by the findLocations program: seconds, positive if the number of seconds must be added to the camera time to obtain UTC time, negative if the number must be subtracted. If the camera clock was set to UTC time, this difference will reflect only the error of the camera clock setting at the time the photograph was taken and if the camera clock was set to local time the difference will include the hourly differential between local and UTC time. (Since it is expressed in seconds, this will be a rather large number, the modulo 3600 (number of seconds in an hour) of which will be the camera clock error).
First command line argument is the name of the file, the second is the time/date known UTC time. The format of given UTC time might be either the same as commonly used by GPS devices (e.g., 2008-10-06T03:46:19Z), or a quote string in the conventional time/date format (e.g., "2008-10-06 03:46:19").
An example of a command line invoking exifTimeDelta program might be:exifTimeDelta \myPhotos\southPacific2008\IMG30073.jpg "2008-10-06 03:46:19"
(To determine the camera clock error at the moment when the picture of the GPS display, which shows UTC time 2008-10-06 03:46:19, was taken).
findLocations program reads a text file that consists of lines with two items on each line: the first is a name of an object, and the second is its time-stamp. The time-stamp is a date/time string (for instance, output from the exifDataList program, see above) that specifies a point in time when the object (for instance, a camera and .jpg file created by it) for which the program must find the most likely location by searching the .trax database. The result of the search is written to the standard output, again with the same object name as the first item on each line, but with the geographic coordinates instead of the time as the second item and an estimate of the accuracy of the determined location as the third.
The time search can yield one of three possible results:
Location accuracy estimates are given as meters on Earth's surface. The accuracy determination process assumes (optimistically !) that the locations in the .trax file are error-less, and that the accuracy increases as the time-stamp of the object gets closer to the time of an individual track point. If the object was located between tracks (i.e., when the GPS device was inactive, but the gap between the tracks was of short length and duration) the accuracy estimate will be decreased (or the number increased) by a factor of ten and will be signed negative in order to flag the location of such an object as being of substantially lesser accuracy than the location of objects that were timed when the GPS logging was active.
The fully qualified .trax file to be searched is given by an -x=... command-line flag (see example below).
The program provides a method to adjust for the time difference between the times in the .trax database (which are assumed to be accurate UTC times) and the times kept by the timepiece (for instance, a camera clock) that was used to produce the object time-stamps. This difference can be the result of a simple imprecise setting and drift over a period of time, or the fact that the timepiece was set to the time of a local time-zone instead of UTC time; or some combination of all those factors. The differential can be specified to the program using -d flag (see the example below and the exifTimeDelta program description above), as a signed integer number of seconds. This number is added to the object time-stamp in order to obtain the equivalent UTC time, to be used when searching the .trax file.
The location units will by default be in decimal degrees. This can be modified by -m flag: if given, the program output will be in degrees and decimal minutes: a format often used on GPS displays.
An example of findLocations command line follows:
findLocations -x=\travels\2008\y2008.trax -d=-36 julyjpgtimes.list
(To produce a standard output text file of object names and their locations (latitude, longitude and elevetion) with accuracy estimates. Trax database is \travels\2008\y2008.trax file, and the camera clock was known to be 36 seconds behind the UTC time for all images (.jpg files) listed in the input text lines. Timestamps of objects are given in julyjpgtimes.list file.
The following is an example of findLocations input file, created by exifDataList program:
p1030295.jpg "2009:07:13 10:51:03"
p1030296.jpg "2009:07:13 10:52:07"
p1030297.jpg "2009:07:13 10:52:32"
p1030298.jpg "2009:07:13 10:52:54"
p1030299.jpg "2009:07:13 10:53:40"
p1030300.jpg "2009:07:13 10:54:41"
p1030301.jpg "2009:07:13 10:55:51"
p1030302.jpg "2009:07:13 10:56:37"
p1030303.jpg "2009:07:13 10:57:35"
p1030304.jpg "2009:07:13 11:00:45"
p1030305.jpg "2009:07:13 11:02:29"
p1030306.jpg "2009:07:13 11:03:07"
p1030307.jpg "2009:07:13 11:03:47"
p1030308.jpg "2009:07:13 11:04:48"
p1030309.jpg "2009:07:13 11:04:56"
p1030310.jpg "2009:07:13 11:05:12"
p1030311.jpg "2009:07:13 11:05:40"
p1030312.jpg "2009:07:13 11:06:04"
p1030313.jpg "2009:07:13 11:06:47"
The findLocations program using this file as input might output something like this:
"p1030295.jpg" "unknown (Time before the first track)"
"p1030296.jpg" "unknown (Time before the first track)"
"p1030297.jpg" " 41.071593 -112.137935" 1072.4 5.7
"p1030298.jpg" " 41.071552 -112.138745" 1046.5 9.5
"p1030299.jpg" " 41.072404 -112.139800" 1074.3 9.4
"p1030300.jpg" " 41.074298 -112.136719" 1097.8 22.2
"p1030301.jpg" " 41.074317 -112.131653" 1043.4 9.7
"p1030302.jpg" " 41.075555 -112.131454" 1006.6 5.0
"p1030303.jpg" " 41.075161 -112.131039" 1048.2 -181.6
"p1030304.jpg" " 41.073871 -112.129680" 1002.5 -501.5
"p1030305.jpg" " 41.073165 -112.128936" 1038.4 -265.5
"p1030306.jpg" " 41.072907 -112.128664" 1054.5 -179.3
"p1030307.jpg" " 41.072636 -112.128377" 1042.8 -88.6
"p1030308.jpg" " 41.072415 -112.131727" 1006.1 11.2
"p1030309.jpg" " 41.072429 -112.132355" 1062.8 9.8
"p1030310.jpg" " 41.072405 -112.134444" 1038.4 17.6
"p1030311.jpg" " 41.072050 -112.136075" 1030.6 11.0
"p1030312.jpg" " 41.071493 -112.134715" 1101.9 15.2
In the above example, pictures p1030295.jpg and ...296.jpg were taken before the GPS was turned on, and p1030303.jpg to ...307.jpg were taken while the GPS logging was stopped for a short time.
traxToKml program reads the Trax database and creates a text file in KML format, containing the geographical representation of GPS tracks. This file can be further edited using any text editor, or used as input for graphical depiction and further editing in geographical applications that support KML file format.
locationsToKml program reads the location list file (created by findLocations program) and creates a text file in KML format, containing the geographical representation of locations from which the photograph were taken. This file can be further edited using any text editor, or used as input for graphical depiction and further editing in geographical applications that support KML file format.
The program accepts two command-line flags: -n=... and -d=..., to specify, respectively, a description and the name of the Kml "document". Descrption will be written to the output file only if given in this manner. If no name is given, the base name of the input file will be used as the KML "document name").
wptTpGpx program reads a free-format text file that of object names and their locations in a form commonly used for large text files of known cardinal points (in GPS parlance, "way-points"). Such file will consist of one "free-form" line per point, with latitude, longitude, elevation, (time ?) and name. Items are blank or comma delimited. Anything on a line starting with "#" is comment. Latitude and longitude are decimal degrees, elevation is meters above or below the sea-level, time is a UTC time string, and the name is an alpha quote string. Elevation and time can be literal "UNKNOWN". Program reads the file given as the first (and only) command line argument and writes to the standard output which can be re-directed into a disk file (cf., newlineGpx)
Back to the Vagabond [main] page