= Introduction: Maps Tell Stories * Accelerometers are the way forward Chris Dodo * Maps Tell Stories ** There is a common misperception that cartography must be an exact science. ** The reality is that "close" is good enough for many purposes. ** This is a three-hour crash course intended to explode that misconception. * Big Bend Three Ways * Big Bend Three Ways * Big Bend Three Ways * Maps Tell Stories ** The Big Bend maps tell different stories about the same place. ** In an increasingly digital and increasingly mobile age, where attention is at a premium, we need maps that tell the right stories. ** We want to show you tools to help you tell your own stories with maps. ** You don't have to be an expert. # last year we had the middle earth map here, and there are a couple of other good ones, like the communities gather around stories slide...but I guess this is 'intro to open source gis, so time to get to content. * Kayak Trips on Chesapeake Bay ** Robert Woodard started with a flat file ** When he hit 1000 waypoints he 'hacked a little perl' to put waypoints into MySQL, and a PHP app to feed WorldKit with RSS. ** Grew his database to 14,000 waypoints using this foundation to import USGS data. ** http://www.kayaktrips.net/geo/ * Compsite aerial photos * more aerial photos * Another view * Telling your own story ** Mapping with a Spreadsheet and a GPS ** Mapping with GD::Graph ** The GPS Plotter Application ** Kayaking on the Chesapeake * Creating maps doesn't have to be hard ** ... or elegant! ** Speaks to the fundamental simplicity of cartography as a story telling medium. ** Also speaks to the tension between precision and ubiquity in geospatial data and applications... ** "Aircraft navigators have a special interest in maintaining a positive height vector above the surface." * With a small amount of work you can get a lot more sophisticated ** MapQuest, MapBlast, etc. ** TopoZone ** TIGER Mapsurfer (http://tiger.census.gov/cgi-bin/mapsurfer) ** worldKit * An apology to our international friends *img img/earth.jpg ** A lot of this talk is necessarily US-centric. We're sorry about that. ** But, hey, we're American! What did you expect? :) ** The US has some of the best sources of free GIS data in the world, making it an easy target for a talk on GIS ** We believe that freely available geo data is a public good -- viva la revolucíon! ** Nearly all of the concepts and software we discuss will apply to other parts of the world, even if the data sources don't = Ellipsoids and Datums and Projections, oh my! * A round world (sort of) ** Our planet isn't merely not regular in shape, it's not even regularly irregular! ** The Earth's shape is usually approximated with an ellipsoid, or the volume described by rotating an ellipse about an axis in space ** Ellipsoidal parameters include: *** Semi-major axis (equatorial radius ~ 6,378km in GRS80) *** Semi-minor axis (polar radius ~ 6,356km in GRS80) *** Flattening ( (major - minor) / major ) *** Eccentricity (deformation from spherical shape) ** The science of estimating and measuring the Earth's shape is called geodesy ** Ellipsoid differs from geoid, an empirical model of the Earth's surface based on gravity measurement * Common ellipsoid models ** Bessel 1841 (widely used in Europe) ** Airy 1858 (Great Britain) ** Clark 1866 (US prior to 1983) ** International 1967 ** Geodetic Reference System 1980 ** World Geodetic System 1984 (increasingly common worldwide) * Projections Any projection of the surface of an ellipsoid on to a flat plane will result in distortion of the ellipsoid surface. Three broad classes of projection make tradeoff between different features: ** Conformal (preserves angles) ** Equidistant ** Equivalent (preserves area) * Projection surfaces There are 3 primary shapes used to project the Earth's surface onto a plane: ** Azimuthal (flat plane at an angle) ** Cylindrical ** Conic *img img/aaMaps_M3_29G.gif (http://www.eoascientific.com/cartography/aaMaps_M3_surface_Z.htm) * Alteration of the projection surface ** Translation *** Tangent *** Secant (intersecting) ** Rotation *** Transverse (right angles) *** Oblique (any angle) Points or lines where the projection surface touches the ellipsoid are called standard points (or lines) and do not distort at those points. * Some commonly used projections ** Cylindrical projections *** Distorts shape in favor of direction and distance *** Mercator (conformal) *** Peters (equivalent or equal-area) *** Transverse Mercator **** Pushes distortion to edges, good for N-S regions **** e.g. British National Grid **** other nat'l grid systems * Some commonly used projections ** Cylindrical projections *** Universal Transverse Mercator (UTM) **** Divides the world into 6 degree longitudinal zones **** Projects each zone into Transverse Mercator **** Provides horizontal coordinate system for the whole world *** Pseudo-cylindrical projections **** Uses curved meridians to correct area distortion **** Often used for world maps **** Examples include Mollweide, Robinson * More commonly used projections ** Conic projections *** Distorts distance in favor of shape and direction *** Widely used in US and other E-W regions *** Examples include Albers Equal-Area, Lambert Conformal ** Azimuthal projections *** Typically distort area in favor of distance and direction *** Often used to map air travel routes or to map polar regions *** Examples include Azimuthal Equidistant, Lambert Azimuthal Equal-Area, Orthographic * More commonly used projections ** Dymaxion and other projections *** Other tradeoffs ** Unprojected maps *** Treat latitude and longitude as rectangular coordinates *** Everything gets distorted *** Easy to draw! * Geodetic Datums ** Definition of the cartesian origin of a geographic coordinate system with respect to a particular ellipsoid model ** Often include a "false" easting or northing to insure positive coordinate values ** Numerically identical coordinates referenced to different geodetic datums may appear at different places on a map ** The difference can sometimes be be hundreds of meters! * Common Geodetic Datums ** Over a hundred geodetic datums commonly used around the world *** World Geodetic System 1984 (WGS84) *** North American Datum 1983 (NAD83) *** North American Datum 1927 (NAD27) *** Ordinance Survey Great Britain 1936 *** etc. ** NAVSTAR GPS relies on WGS84 ** NAD83 and WGS84 are identical for most practical purposes * Coordinate Systems ** Latitude, longitude, and height ** Universal Transverse Mercator (UTM) ** National grid systems ** US State Plane system ** Earth-centered 3D cartesian and polar systems * The Univeral Transverse Mercator system ** Each UTM zone has a number and letter ** Zone numbers represent 6° longitudinal slices extending from 80° S to 84° N ** Zone letters designate 8° zones north and south away from the equator. * More about the Univeral Transverse Mercator system ** UTM measures easting in meters from the zone origin (with a 500km offset) ** ... and northing in meters from the Equator (with a 10,000km offset in the Southern Hemisphere) ** UTM allows you to treat individual zones as more or less rectangular ** (Actually, distances measured within a zone should be scaled by 0.9996 to equalize distortion across the zone) * Converting Lat/Long to Lat/Long ** Classic problem of radix conversion *** O'Reilly & Associates at 38.41061° N 122.84093° W **** 38° + ( .41601 x 60 )' = 38° 24.9606' **** 38° 24' + ( .9606 x 60 ) = 38° 24' 57.636" *** 38° 24' 57.636" N **** 38 + (24 / 60) + (57.636 / 3600) = 38.41061° ** Don't forget: Longitudes west of the Prime Meridian and latitudes south of the Equator are often represented by negative values! * Converting Universal Transverse Mercator to Lat/Long and back ** Spherical trigonometry to transform and rotate points through space ** Good luck doing this one on a pocket calculator! ** Enter PROJ.4, a cartographic projections library 
cs2cs +proj=latlong +datum=NAD83 +to +proj=utm +zone=10 +datum=NAD27
** cs2cs can also do DMS to decimal and back: 
cs2cs -f "%.6f" +proj=latlong +to +proj=latlong
** PROJ.4 lives at http://remotesensing.org/proj/ * Converting WGS84 to NAD27 and back ** Uses a bit of spherical trig called the Standard Molodensky transform ** PROJ.4 does this for us, too 
cs2cs +proj=latlong +datum=WGS84 +to +proj=latlong +datum=NAD27
** PROJ.4 knows about lots of other datums and projections, as well = Global Positioning Systems * Global Positioning Systems (GPS) ** Global positioning triangulates an unknown location on Earth by correlating time signals from multiple satellites in orbit ** NAVSTAR launched by USAF from 1978 to 1994 ** NAVSTAR constellation employs 24 satellites in 12 medium earth orbits staggered such that five to eight are overhead anywhere on Earth at any time ** NAVSTAR operates in the 1.2 and 1.5 GHz microwave bands * GPS in Theory ** Each GPS satellite transmits its orbital location plus a time-seeded pseudorandom sequence ** A receiver can use this plus the mostly constant speed of light to plot its own location along the edge of a sphere with a radius proportional to the transmission time ** When three of these spheres are plotted, the receiver will be found at the point or region of intersection between them ** A fourth sphere is needed to find elevation * GPS in Practice ** Local bias *** Microwave obstructions *** Multipath and RF noise *** Incorrect local time *** Receiver motion ** More general difficulties *** Ephemeris inaccuracies *** Ionospheric delay *** Selective Availability (off since Oct 2000) * Global Positioning in the Real World ** GPS can be a reliable timekeeping source ** GPS must correct for relativistic time dilation -- about 38 microseconds a day ** Enhancements to GPS *** Differential GPS (DGPS) *** Wide-Area Augmentation System (WAAS) *** European Geostationary Navigation Overlay (EGNOS) ** Other global positioning systems *** LORAN-C *** GLONASS *** Gailleo (due in 2007) * GPS Data Protocols ** Plain ASCII text *** Lat, long, (maybe) altitude, and time ** NMEA-0183 *** ASCII "sentences" over RS-232 at 4800 baud, 8N1 *** Published by the National Marine Electronics Association *** Live navigation and ephemeris data *** Write-only ** Proprietary formats *** e.g. Garmin's GRMN format *** Used to fetch and store tracklogs, waypoints, base maps, etc. *** Read/write * Getting data from your GPS ** Command Line utilities *** Garnix *** gpstrans *** gpsutil ** APIs *** GPS::Garmin *** pygarmin * Real-time GPS with gpsd ** Reads NMEA-0183 from a serial port ** Republishes via a simple protocol over TCP/IP ** Intended to make free navigation programs hardware independent ** You can map remote objects in real-time over the 'Net! ** Available from http://pygps.org/gpsd/ * What can you with your GPS data? ** Real-time navigation (gpsdrive) ** Plot it on a basemap ** Use it to georeference other stuff * Options for plotting your tracklogs on a base map ** Naive graphing methods (discussed earlier) ** Web-based options like PointMapper, worldKit, etc. ** Simple mapping programs like drawmap ** A full-scale GIS, like GRASS * Geotag your photos from a GPS tracklog ** Time: The universal foreign key ** Adjust for bias in clock difference, time zone ** Linear interpolation between tracklog points * Making your own Garmin base maps ** Get existing aerial photos or base maps ** Add features by hands or from GPS data ** Render the new features to Garmin .IMG format ** We arrived at Burning Man at 5am and found our camp within minutes. ** Tools *** GPSmapper: http://gps.chrisb.org/ *** GPSMapEdit: http://kgy.narod.ru/util/mapedit/mapedit_e.htm = Geographic Information Systems and GIS Data Sources * Geographic Information Systems ** Layers ** Scale and extents ** Data elements *** Points *** Lines *** Areas ** Data Models *** Raster data model *** Vector data model *** "Site" and point data ** "Raster is faster, vector is corrector." -- Joe Berry * Common GIS data formats ** USGS Spatial Data Transfer Standard (SDTS) ** GeoTIFF ** Arc/Info formats ** ESRI shapefiles ** US Census TIGER/Line ** Geographic Markup Language * Manipulating GIS data formats ** Geospatial Data Abstraction Library (GDAL) ** OGR Simple Feature Library ** GDAL converts raster data, OGR handles vector ** Both can be found at http://www.remotesensing.org/gdal/ * US Zip Codes ** No official standard for geocoding ** Can be calculated based on the 'centroid' of the zip code ** US Census Gazetteer has reasonable location and population data for ZIP codes, as well as counties, metro areas, and inhabited places ** Find it at http://www.census.gov/geo/www/gazetteer/places.html * British Post Codes ** The more digits, the more precise your location ** In urban areas, post codes can cover areas as small as 20 meters across ** Free UK postal outcode database at http://www.jibble.org/ukpostcodes/ ** See Tom Coates at the break for more info (Thanks Tom!) * Geographic Name Information Service (GNIS) ** GNIS is a freely downloadable gazetteer of places in the USA. ** Includes lat, long, county name, and the following features:
airport, arch, area, arroyo, bar, basin, bay, beach, bench, bend, bridge, building, canal, cape, cemetery, channel, church, civil, cliff, crater, crossing, dam, falls, flat, forest, gap, geyser, glacier, gut, harbor, hospital, island, isthmus, lake, lava, levee, locale, military, mine, oilfield, other, park, pillar, plain, range, rapids, reserve, reservoir, ridge, school, sea, slope, spring, stream, summit, swamp, tower, trail, tunnel, valley, well, woods
** One can very easily make a REST-ful web service out of this! ** For places outside the US, try the GEOnet Names Server at http://earth-info.nima.mil/gns/html/ * Topologically Integrated Geographic Encoding and Referencing system (TIGER) ** Collected by Census Bureau every ten years or so ** Contains lines and polygons of most relevant features ** Often inaccurate and incomplete, but at least it's there ** You can map street maps with it ** You can build a pretty good street address geocoder with it ** It's a HUGE amount of data ** Available at http://www.census.gov/geo/www/tiger/ * Other sources of (sometimes) free US GIS data ** USGS Digital Line Graph (DLG) ** USGS Digital Raster Graphics (DRG) ** USGS Digital Elevation Models (DEM) ** USGS Digital Orthophoto Quads (DOQ) ** worldKit MapProxy: http://brainoff.com/worldkit/mapproxy/ * US-based sources of free global GIS data ** NIMA Digital Terrain Elevation Data (DTED) ** NOAA Coastline Extractor ** NASA Earth Observatory ** Virtual Terrain Project at http://vterrain.org/Locations/ is a good starting point for finding sources of free data for the US and the world = GRASS, an Open Source GIS * Geographic Resources Analysis Support System ** Download and install GRASS ** Download sample data (use the demo data as a learning aid) ** GRASS commands are just command line programs ** GRASS has a GUI that wraps these programs * Installing GRASS ** Download appropriate binary package from http://www.geog.uni-hannover.de/grass/download.html ** On Mac, use the install script (also in Fink under unstable) ** Debian users can apt-get install grass ** Or you can build from source... * Download Sample GRASS Data ** Lots of samples at http://www.geog.uni-hannover.de/grass/data.html ** Download global_grass4data.tar.gz and spearfish_grass50data.tar.gz ** mv global/ spearfish/ /usr/local/grass5 (or your database path) * A Simple GRASS Session ** Starting GRASS Database/Location/Map Sets ** The simple viewer: d.dm & * More useful commands ** Manage monitors with d.mon start=x0 ** Show available layers with g.list ** Show info with [vrs].info ** Display layers with d.rast, d.vect and d.sites ** d.zoom click to mark first corner, middle click to zoom in/out * Creating a New GRASS Location ** Coordinate System ** Zone for UTM, other parameters for your projection ** Coordinates of the region ('extents') and grid resolution ** A text description * Working with Raster Data ** Importing and exporting raster data ** Viewing and managing raster map layers ** r.poly -c input=nations output=nations ** Plotting line of sight from a DEM ** Plotting a wireless viewshed from a DEM ** Plotting the effects of rising sea levels * Shuttle Radar Topgraphy Mission (SRTM) ** SRTM and Landsat 7 data http://srtm.usgs.gov/srtmimagegallery/ * Shuttle Radar Topgraphy Mission (SRTM) ** Digital Elevation Model (DEM) Coverage ** 1 arc second (30 m) coverage of the US ** 3 arc second (90 m) coverage of most of the world ** Lot's of nulls and other data issues ...but nearly ubiquitous! * Fetching SRTM Data ** ftp://edcsgs9.cr.usgs.gov ** Documentation ** /pub/data/srtm/What_are_these.txt ** /pub/data/srtm/Documentation ** US Data ** /pub/data/srtm/United_States_1arcsec/1arcsec ** N38W122.hgt.zip * Loading SRTM Data ** 1 arc second data has 3601 rows x 3601 columns ** Lat/Long of file is the center of the lower left arc-second ** Create a new location ** Rename *.hgt *.bil ** r.in.bin -b input=N38W123.bil output=N38W123 bytes=2 title="N38W123 1 arc second SRTM" north=39 south=38 east=-122 west=-123 r=3601 c=3601 ** r.colors map=N38W123 color=grey.eq (or any color map!) * Loading SRTM Data - A script * Working with SRTM Data - A demonstration ** Display ** Zoom in ** r.profile * Working with Vector Data ** Importing and exporting vector data ** Viewing and analyzing vector data ** Shading vector regions from other data sources ** Making contour maps from topographical data * Working with Site/Point Data ** Importing and exporting site/point data ** Viewing and managing site data ** Coordinate conversions with m.ll2u ** Convert tracklogs and waypoints to vector and raster layers * GNIS as an example of Site Data ** Getting GNIS ** Loading GNIS data into GRASS with s.in.ascii ** Displaying GNIs ** d.sites.labels * Automating tasks with GRASS ** Make HTML Image Map ** Load SRTM ** You have a command line, and so can use any scripting tool you wish! 
sudo mysql nocat < get_interested.sql | /home/rich/wa/gps/dd2ll.pl > temp.ll
m.ll2u -z spheroid=wgs84  input=temp.ll | adjust.pl > file.utm
s.in.ascii input=file.utm sites=nocat_interested
* Some other GRASS tricks ** Mapping Great Circle routes ** Reprojecting raster data ** Make an HTML image map from a vector layer ** Exporting GRASS maps to PostScript ** Creating a 2D shaded elevation map ** Mapping a long-distance wireless network in 3D ** 3D visualization with NVIZ * Making countour maps ** r.contour N38W124 step=10 minlevel=0 output=contour.test ** The Power of r.mapcalc ** r.shade.rel.sh ** Make an HTML image map from a vector layer ** The power of Voronoi diagrams to capture your closest click ** A script to make HTML Image Maps * When you don't need all of GRASS: libgrass5 ** libgrass5 implements the guts of GRASS without the chrome ** Offers a way of leveraging GRASS functionality in your own applications ** Or a way of retooling selected GRASS program to run in standalone form ** Available from http://gdal.velocet.ca/projects/grass/ = Other GIS Projects * When you just want a GIS viewer: Thuban ** Thuban is a straightforward GIS layer browser ** Implemented in wxPython: cross-platform, easy to extend ** Reads ESRI shapefiles, GeoTIFF, PostGIS ** Plans to support GDAL/OGR for a wider ranger of formats ** Available at http://thuban.intevation.org/ * MapServer: a GIS browser for the web "MapServer is an Open Source development environment for building spatially enabled Internet applications... MapServer is not a full-featured GIS system, nor does it aspire to be. It does, however, provide enough core functionality to support a wide variety of web applications." ** http://mapserver.gis.umn.edu/ * Features of MapServer ** Supports ESRI shapefile, TIFF/GeoTIFF, GIF, PNG, JPEG and more ** Quadtree spatial indexing for shapefiles ** Fully customizable, template driven output ** Feature selection by item/value, point, area or another feature ** Support for tiled raster and vector data * More Features of MapServer ** TrueType font support ** Automatic legend and scalebar building ** Scale dependent feature drawing and application execution ** Feature labeling including label collision mediation ** Mapscript bindings for Perl, Python, Tcl, Guile, and Java * Installing and Configuring MapServer ** The HTML template ** The 'map' file ** Adding Layers to MapServer ** MapServer Workbench: a Tcl/Tk MapServer configuration tool ** Improving MapServer with a bit of JavaScript * Practical applications of MapServer ** Plotting earthquake locations ** Plotting wardriving data ** Importing TIGER/Line data ** Getting MapServer to show features at the right scale * Virtual Terrain Project ** "The goal of VTP is to foster the creation of tools for easily constructing any part of the real world in interactive, 3D digital form." ** Merges GIS with OpenGL for a 3D modelling environment ** Models not just terrain, but vegetation and man-made features ** Really impressive -- if you have accelerated OpenGL ** MIT licensed, but you have to email Ben to ask for the source! ** Check it out at http://vterrain.org = Emerging Standards for GIS Data Interchange * The OpenGIS consortium ** The source for standards among the serious GIS geeks ** Simple Feature Specification ** Well-Known Text and Binary representations ** Geographic Markup Language (GML) ** Web Feature Service (WFS) ** Web Mapping Service (WMS) * Applications for OpenGIS standards ** Web Services *** Microsoft's terraservice.net *** GeoServer - a transactional GML server ** Spatial databases *** Simple Feature Specification for SQL *** PostGIS *** MySQL Spatial Extensions * What is the Resource Description Framework (RDF)? ** A formal graph-based model for describing and accessing data ** An RDF model consists of statements about the world ** Each statement has a subject, predicate, and object ** RDF allows the construction of domain-specific vocabularies with specific, well-defined semantics ** RDF is not XML but is commonly represented with XML * Why RDF is useful for geospatial applications ** The fire hydrant example ** Dan Brickley says:
"The RDF data model [is] a handy mechanism for mixing independently created data vocabularies... Unlike vanilla XML, RDF vocabularies can be freely mixed together in data without prior agreement. So you often see ad-hoc combinations of Dublin Core, RSS1, MusicBrainz, RDF-calendar, FOAF, Wordnet, thesaurus, Geo-info etc etc..."
* Geo namespace ** Described at http://www.w3.org/2003/01/geo/ ** Defines a single RDF Schema class, geo:Point ** Defines three properties: geo:lat, geo:long, geo:alt ** Intentionally kept as simple as possible * Geo RDF/XML example 
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
	xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#">
  <geo:Point>
    <geo:lat>55.701</geo:lat>
    <geo:long>12.552</geo:long>
  </geo:Point>
</rdf:RDF>
* Other relevant RDF vocabularies ** rdfgeom2d ** spacenamespace ** Locative packets ** Dublin Core * What to do with RDF ** Publish it ** Aggregate it *** RDF/XML files *** Naive SQL store *** Redland, Sesame, Jena ** Make maps with it *** worldKit *** RDFmapper / PointMapper *** Collaborative mapping -- come to the workshop! * worldKit ** Clickable, Flash-based interface to point data ** Configured with a simple XML file ** Plots RSS2.0 + ICBM, or RDF Locative packets ** Supports Dymaxion and polar projections ** Get it from http://www.brainoff.com/worldkit/ ** PointMapper is more interactive, but it's not Open Source (yet)! * Scalable Vector Graphics (SVG) ** XML vector graphic format ** Can be made dynamic with Javascript ** Batik SVG browser at http://xml.apache.org/batik/ ** The future of interactive maps on the web? = Building a Custom GIS Application * You say you want a revolution? Albert Einstein, when asked to describe radio, replied:
"You see, wire telegraph is a kind of a very, very long cat. You pull his tail in New York and his head is meowing in Los Angeles. Do you understand this? And radio operates exactly the same way: you send signals here, they receive them there. The only difference is that there is no cat."
** The NoCat Network is a free community network built with fixed-point 802.11b links. ** Microwave radio requires line-of-sight over long distances. ** Who can see whom? ** Suppose we have 100 participants. ** 100 x 99 ÷ 2 = 4,950 possible links! * maps.nocat.net ** Node database ** Geocoder ** Elevation profile ** Interactive flat map ** The goal: All you need is a compass and an antenna! * Node database ** Collect name, contact info, location ** If the user doesn't know their lat/long, geocode their address against TIGER ** Add their node to the database and calculate all the potential LoS on the fly * Profile calculations ** Uses 10m DEMs from BARD covering Sonoma County ** Borrowed source code from r.profile, linked against libgrass5, glued to perl with Inline::C ** Clearance values are cached in database ** Profile images are just line plots in GD::Graph! * True bearing and distance ** Now the user can view a table of potential peers, sorted by clearance ** Distance is easy in UTM: Pythagorean theorem x .9996 ** Bearing is pretty easy, too: arctan2(x, y) ** Also, the data can now be pulled out and viewed in MapServer * What the NoCat Map doesn't do ** Curvature of the Earth ** Fresnel zones ** Ground clutter ** Where's the hill? = Conclusion: Maps Can Tell Your Stories * Conclusions ** Maps can tell your stories ** Mapping and GIS aren't hard, they just look that way ** You don't have to be an expert ** You don't have to spend a fortune on software ** You (maybe) don't have to spend a fortune on data ** Go tell all your friends ** Thank you! * Accelerometers are the way forward Chris Dodo