Geo-spatial

Geo-spatial data and analysis

Overview & introduction

Historical context

Spatial data has been around for a long time, as we saw in lecture-1, the first data visualizers were map makers

Spatial Data: Overview

Spatial data, also known as geo-spatial data, is information about a physical object that can be represented by numerical values in a geographic coordinate system.

Spatial data represents:

  • location, size and shape of an object on planet Earth such as a building, lake, mountain or township
  • Spatial data may also include attributes that provide more information about the entity that is being represented
  • Geographic Information Systems (GIS) or other specialized software applications can be used to access, visualize, manipulate and analyze geospatial data
  • Can be another attribute in a standard tabular dataset

Latitude and longitude are not enough!

  • Coordinate pairs, are overly simplistic, and lose much of their meaning when treated independently
  • Typically we are more interested in location AND local attributes
  • In addition to having point locations, observations may often be associated with spatial lines, areas, or grid cells
  • spatial distances between observations are often not well represented by straight-line distances, but by great-circle distances, distances through networks, or by measuring the effort it takes in getting from \(\mathrm{A}\) to \(\mathrm{B}\)

What can we use Spatial analysis for?

  • Spatial analysis lets us ask, and find answers to, a wide range of questions in six broad categories:
    • Understanding where things and events are
    • Measuring sizes, shapes, and distributions of things or measurements
    • Analyzing relationships and interactions between places
    • Optimizing locations for facilities, or routes for transportation
    • Detecting and quantifying patterns and relationships between things or measurements
    • Making predictions based on existing or theoretical patterns and relationships
  • For today’s, we focus on visualizing data with a geographical/geo-spatial component.

Geo-spatial: use cases

  • Using visualizations to understand relationships over space (and also time)
  • Typically related to data linked with locations or coordinate systems
  • There are many applications
    • geographic maps, geo-statistics and geographic information systems
    • medical images (x-rays, MRI, microscopy, pathology)
    • LIDAR/SONAR/Remote sensing
    • Epidemiology
    • Environmental science
    • meteorology/climatology
    • urban planning

Applications

Fun Aside: Physics

  • Much of physics is inherently “spatial” in nature. In some sense, physics is just a collection of models and laws that describe spatial kinematics of matter in 3D space.
  • This is a fun gif showing the solar system with a moving sun (as is the case)

Static sun solar system

Moving sun solar system (real case)

Coordinate Reference Systems

CRS

Spherical Coordinate system

  • We exist in 3D space, which is mathematically represented with coordinate systems
  • Not surprisingly, spherical coordinates are particularly useful for geo-data
  • Aside: You can “re-map” coordinates from one space to another with the following:
    \(\begin{aligned} & r=\sqrt{x^2+y^2+z^2} \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, \theta=\arccos \frac{z}{\sqrt{x^2+y^2+z^2}}=\arccos \frac{z}{r}=\arctan \frac{\sqrt{x^2+y^2}}{z} \\ & \varphi= \begin{cases}\arctan (y / x) & \text { if } x>0 \\ \arctan (y / x)+\pi & \text { if } x<0 \\ \frac{\pi}{2} & \text { else }\end{cases} \end{aligned}\)

Mathematical relationships

As with other data, Geo-spatial relationships typically fall into the following categories

Overview

  • Coordinate reference systems (CRS) are ways to represent the 3D spatial data of earth on a 2-dimensional surface. 
  • A spatial reference system defines a specific map projection, as well as transformations between different spatial reference systems.

CRS: Components

  • The CRS is a very important aspect of spatial data. For example, a location of (140, 12) on earth is useless without units (meters, KM, miles, degrees) and an origin point.
  • The coordinate reference system is made up of several key components:
    • Coordinate system: The X, Y grid upon which your data is overlayed and how you define where a point is located in space.
    • Horizontal and vertical units: The units to define the grid along the \(x, y\) (and \(z\) ) axis.
    • Datum: A modeled version of the shape of the Earth which defines the origin used to place the coordinate system in space.
    • Projection Information: The mathematical equation used to flatten objects that are on a round surface(e.g. the Earth) so you can view them on a flat surface (e.g. your computer screens or a paper map).
      • Different planar coordinate reference systems are referred to as projections. Examples are ‘Mercator’, ‘UTM’, ‘Robinson’, ‘Lambert’, and ‘Albers’.

Good resource: click here

CRS: Examples:

  • There are MANY CRS and components of CRS
  • They are typically tagged with EPSG tags to identify them
    • Note: EPSG=“European Petroleum Survey Group”
  • CRS definitions will typically consist of a “stack” of dependent specifications, as exemplified in the following table:
  • This is a vast field of study, if you specialize in geo-spatial data, then you would need to learn more, however, for now this is enough.

CRS Units: longitude & latitude

  • Geo-spatial data is often represented in spherical coordinates, with the radius constant
  • A common choice of units in a CRS is degree’s (longitude & latitude)

Exercise

It’s good to be able to estimate locations on earth from longitude & latitude. For example:

CRS Units: Minutes to degrees

London: latitude 51.509865, and longitude -0.118092. (degrees)

CRS Units: Length

  • Units can also be distance measured from some point (e.g. meters)
  • When using a CRS, it is good to look it up to determine the units and other details

  • Attention to detail with units is always important, e.g. measuring area in “degrees squared” doesn’t make a lot of sense.
  • Sanity check: Earth/Circumference 40.075 million meters

CRS: Map projections

  • Projection: The mathematics used to flatten objects that are on a round surface
  • This is a specialized form of dimensionality reduction
  • map projection are either “equal area” (the scale of the map is constant) or “conformal” (the shapes of the geographic features are as they are seen on a globe)
  • Map projects have to be one or the other, they can’t be both

Projection: Examples

CRS: Map projections distortions

Mappings from 3D to 2D always leave artifacts and distortions.

Map projections: Size comparison

Geo-spatial data

Vector and raster GIS data

We’ve seen raster & vector graphics before, but, the concepts are also important for geo-data

Raster GIS Formats Raster data is made of pixels (grid cells) with associated values (continuous) or class (discrete). They are usually regularly spaced squares but don’t have to be.

Vector GIS Formats: Not made up of grids of pixels. Instead, vector graphics are comprised of vertices and paths (points, lines, and polygons).

source: click here

Multi-layer Raster LIDAR

In addition to raster and vector data, there is also LiDAR data (also known as point clouds) and 3D data. LiDAR data is data that is collected via satellites, drones, or other aerial devices. 3D data is data that extends the typical latitude and longitude 2-D coordinates and incorporates elevation and or depth into the data. While complex, this data is rich with information and can be used to solve a variety of problems pertaining to the Earth’s surface.

Geo-spatial file-formats

There are MANY formats for storing Geo-spatial data, here are some popular choices.

Resource: click here for more information on formats

Geo-Json: components

  • GeoJSON is an open standard format designed for representing simple geographical locations, along with their non-spatial attributes.
  • GeoJSON consists of the following parts:
  • Geometry object: point, line, or polygon composing location information. Points are [x, y] or [x, y, z]. They may be [longitude, latitude ] or [eastings, northings]. Elevation is an optional third number.

Good resource: https://developer.here.com/blog/an-introduction-to-geojson

Geo-Json: Geometry primitives

Geo-Json: Geometry examples

Here are additional examples of geographic information stored in the Geo-Json format

Geo-Json: Multipart geometries

We can also store collections of geometric primitives in groups

Geo-Json: components

  • Feature object: This is the geometry object & the associated meta data. This can be anything associated with that location or region
  • FeatureCollection: Basically just a list of feature objects. This can be used to collect different categories of features

Optional: Demonstration

Shape files

  • The shape-file is an alternative to GeoJSON, it is BY FAR the most common geospatial file, since it is used by GIS.
  • The term “shapefile” is quite common, but the format consists of a collection of files with a common filename prefix, stored in the same directory.
  • Mandatory files
    • shp - Main file: a direct access, variable-record-length file in which each record describes a shape with a list of its vertices.
    • shx - Index file: each record contains the offset of the corresponding main file record from the beginning of the main file. The index file (.shx) contains a 100-byte header followed by 8-byte, fixed-length records.
    • dbf - dBASE Table file: a constrained form of DBF that contains feature attributes with one record per feature. The one-toone relationship between geometry and attributes is based on record number. Attribute records in the dBASE file must be in the same order as records in the main file.
  • One downside, the files are stored as binary objects and are not human readable

Interactive mapping packages

Java-script libraries

Leaflet.js

  • Popular open-source JS library for embedding mobile-friendly interactive maps.
  • Very light-weight, at around 38 KB of JS, it has all the mapping features most developers ever need.
  • Leaflet.js provides the mapping tools, the data comes from map data-bases.

Map database: OpenStreetMap

  • OpenStreetMap (OSM) is a collaborative project to create a free editable geographic database of the world by the OpenStreetMap Foundation (OSMF).
  • It is licensed under the Open Data Commons Open Database License (ODbL)
  • OpenStreetMap.org provides a map interface based on the Leaflet JavaScript library
  • Google also has it’s own map data-base as an alternative to OSM

Leaflet.js alternative \(\rightarrow\) Mapbox GL JS

  • Mapbox GL JS is a client-side JavaScript library for building web maps and web applications with Mapbox’s modern mapping technology.
  • You can use Mapbox GL JS to display Mapbox maps in a web browser or client, add user interactivity, and customize the map experience in your application.
  • Similar to OpenStreetMap, the Mapbox also provides a popular geographic database

Source: [click here for more)(https://docs.mapbox.com/mapbox-gl-js/guides/)

Leaflet.js Alternatives

GDAL

Geo-spatial visualization

Introduction to geo-spatial visualization in Python.

Python: Common Geo-spatial packages

Note: This is not in any particular order

Geo-pandas

  • The most useful way to map geospatial data in Python is geopandas.
    • This works very similarly to the sf package in \(\mathrm{R}\)
    • All the data in one pandas. DataFrame-like object
    • One column for the simple features geometry specification (called geometry)
    • Other columns for data about each geometry, that you can then represent within a map
  • There are some basic plotting functions with Geo-pandas, built on MPL
  • Note: We cover GeoPandas more in the lab, so we only show the basics here
  • For more information click here

Geo-pandas: Data Frames

  • The core data structure in GeoPandas is the geopandas.GeoDataFrame
  • It is a modified data-frame to store geometry and perform spatial operations.
  • geopandas.GeoSeries, a subclass of pandas.Series, handles the geometries

Geo-pandas: Data Frames example-1

Input data: GeoJson

Resultant data frame

Geo-pandas: Data Frames example-2

Input data: GeoJson

Map

Data frame

Geopandas: CRS

  • GeoSeries.crs attribute, stores information about the projection and CRS

Code example: Python

Code
import geopandas
import matplotlib.pyplot as plt

# GET DATA
PATH = geopandas.datasets.get_path("naturalearth_lowres")

# READ
gdf = geopandas.read_file(PATH)

# PLOT: W/ MATPLOTLIB
fig, ax = plt.subplots()
ax = gdf.plot("name", legend=False, ax=ax)
ax.set_title("WGS84 (lat/lon)")
plt.show()
# plt.clf()

CRS: Units and Measures

  • Again the CRS units are very important, as they effect anything you measure
Code
import geopandas 

#GET DATA
PATH = geopandas.datasets.get_path("naturalearth_lowres")

# READ
gdf = geopandas.read_file(PATH)

gdf = gdf.to_crs("EPSG:3395")   #CHANGE CRS TO 3395

#MEASURE THE AREA OF EACH POLYGON:  access the GeoDataFrame.area
gdf["area"]=gdf.area; #print(gdf)
#SANITY CHECK GOOGLE: Tanzania/Area --> 364,900 mi^2 9.450867e+11 m^2
print(gdf.loc[gdf['name'] == "Tanzania"])
#NEED TO BE CAREFUL WITH MULTI-POLYGON COUNTRIES (AREAS INCORRECT)
      pop_est continent      name iso_a3  gdp_md_est  \
1  58005463.0    Africa  Tanzania    TZA       63177   

                                            geometry          area  
1  POLYGON ((3774143.866 -105050.440, 3792946.708...  9.459693e+11  
  • ALWAYS check to make sure your numbers make sense and are meaningful!

Code example: Change CRS

Code
# DROP ANTARTICA (CAUSES PROBLEMS)
gdf = gdf[(gdf.name != "Antarctica") & (gdf.name != "Fr. S. Antarctic Lands")]

# #CHANGE CRS TO 3395
# gdf = gdf.set_crs("EPSG:3395",allow_override=True)     #Set Projection
gdf = gdf.to_crs("EPSG:3395")  # Re-Project
print(gdf.crs)

# REPLOT
fig, ax = plt.subplots()
ax = gdf.plot("name", legend=False, ax=ax)
ax.set_title("EPSG:3395 (Mercator)")
plt.show()

# # #CONVERT BACK
# gdf = gdf.to_crs("EPSG:4326")      #Re-Project
# print(gdf.crs)
# fig, ax = plt.subplots()
# ax = gdf.plot("name", legend=False,ax=ax)
# ax.set_title("WGS84 (lat/lon)");
# plt.show(); #plt.clf()
EPSG:3395

Geo-pandas: Visualization

  • Let’s use some inbuilt data to do some maps
Code
import pandas as pd
import matplotlib.pyplot as plt
import geopandas as gp
world = gp.read_file(gp.datasets.get_path('naturalearth_lowres'))
print(world.head())
print(world.columns)
       pop_est      continent                      name iso_a3  gdp_md_est  \
0     889953.0        Oceania                      Fiji    FJI        5496   
1   58005463.0         Africa                  Tanzania    TZA       63177   
2     603253.0         Africa                 W. Sahara    ESH         907   
3   37589262.0  North America                    Canada    CAN     1736425   
4  328239523.0  North America  United States of America    USA    21433226   

                                            geometry  
0  MULTIPOLYGON (((180.00000 -16.06713, 180.00000...  
1  POLYGON ((33.90371 -0.95000, 34.07262 -1.05982...  
2  POLYGON ((-8.66559 27.65643, -8.66512 27.58948...  
3  MULTIPOLYGON (((-122.84000 49.00000, -122.9742...  
4  MULTIPOLYGON (((-122.84000 49.00000, -120.0000...  
Index(['pop_est', 'continent', 'name', 'iso_a3', 'gdp_md_est', 'geometry'], dtype='object')

Geo-pandas: Choropleth

Code
world['gdp_per_cap'] = world.gdp_md_est / world.pop_est
fig, ax = plt.subplots(1,1)
world.plot(column = 'pop_est', 
  ax=ax, legend=True,
  legend_kwds={'label': 'Population by Country',
               'orientation': 'horizontal'})
plt.show()

Geo-pandas: Borders

Code
world.boundary.plot();

Folium

  • Folium is a Python library which wraps around leaflet.js to create Leaflet maps.
  • Due to the interactivity, Folium is useful for dashboard creation, e.g. w/ Streamlit . 

Example: initialize a map:

Code
import folium
from IPython.display import display

# INITIALIZE MAP 
m = folium.Map(location=[38.907852, -77.072807])

# SAVE
file_name="img/map-0.html"
m.save(file_name)

# RE-LOAD AS AN IFRAME
from IPython.display import IFrame
IFrame(src=file_name, width=800, height=400)

Folium layers: Markers and circles

Markers can be added to folium maps with folium.Marker and add_to functions. After creating you map you will ble able to add the marker, specifying the desired location, as in the example below.

Code
# https://python-charts.com/spatial/interactive-maps-folium/
import folium

m = folium.Map(location = [34.06, -118.25], zoom_start = 8)

# Add a marker to the map
folium.Marker(location = [34.06, -118.25]).add_to(m)

# Circle with a radius of 50km
folium.Circle(
    location = [33.56, -116.85],
    radius = 50000,
    color = "black",
    fill = True,
    fill_color = "blue").add_to(m)


# SAVE
file_name="img/map-1.html"
m.save(file_name)

# RE-LOAD AS AN IFRAME
from IPython.display import IFrame
IFrame(src=file_name, width=800, height=400)

Folium layers: Polygons from GeoJSON

It is possible to add polygons to the folium maps with the GeoJson function. You just need to pass the geojson file to data, set a name to the layer with name and add optional styling with style_function.

Code
# https://python-charts.com/spatial/interactive-maps-folium/
import folium

m = folium.Map(location = [38.34, -0.49], zoom_start = 13)

# Geojson url
geojson_url = "https://raw.githubusercontent.com/R-CoderDotCom/data/main/sample_geojson.geojson"

# Desired styles
style = {'fillColor': 'red', 'color': 'blueviolet'}

# Geojson
folium.GeoJson(data = geojson_url, name = "geojson",
    style_function = lambda x:style).add_to(m)

# SAVE
file_name="img/map-2.html"
m.save(file_name)

# RE-LOAD AS AN IFRAME
from IPython.display import IFrame
IFrame(src=file_name, width=800, height=400)

Folium plugin: Heat map

To create a heat map you can import HeatMap from folium.plugins and then use the HeatMap function in order to display your data.

Code
import folium
from folium import plugins
from folium.plugins import HeatMap
import numpy as np

# Sample longitudes and latitudes
data = np.array([np.random.normal(35, 2, size = 100),
                 np.random.normal(-80, 2, size = 100)]).T

m = folium.Map([35, -80], zoom_start = 5)

HeatMap(data).add_to(m)

# SAVE
file_name="img/map-3.html"
m.save(file_name)

# RE-LOAD AS AN IFRAME
from IPython.display import IFrame
IFrame(src=file_name, width=800, height=400)

Folium plugin: Mini map

  • You can add a mini map to folium adding plugins.MiniMap() to the map. By default, it will display a small map at the bottom right.
Code
# SOURCE: https://python-charts.com/spatial/interactive-maps-folium/
import folium
from folium import plugins

m = folium.Map(location = [34.06, -118.25], zoom_start = 8)

plugins.MiniMap().add_to(m)

# SAVE
file_name="img/map-4.html"
m.save(file_name)

# RE-LOAD AS AN IFRAME
from IPython.display import IFrame
IFrame(src=file_name, width=800, height=400)

Folium plugin: Cluster marker

  • In case you have too many markers you can cluster them with MarkerCluster depending on the zoom level. You can also click over the clusters to zoom over them.
Code
# https://python-charts.com/spatial/interactive-maps-folium/
import folium
from folium import plugins
import numpy as np

# RANDOMLY SAMPLE LONGITUDES AND LATITUDES
data = np.array([np.random.uniform(low = 25, high = 50, size = 100),
                 np.random.uniform(low = -125, high = -70, size = 100)]).T

# INITIALIZE MAP
m = folium.Map([40, -100], zoom_start = 3)

# ADD POINTS 
plugins.MarkerCluster(data).add_to(m)

# SAVE
file_name="img/map-5.html"
m.save(file_name)

# RE-LOAD AS AN IFRAME
from IPython.display import IFrame
IFrame(src=file_name, width=800, height=400)

Plotly: Geo-maps

Plotly: Choropleth-1

As we have seen, A choropleth map displays divided geographical areas or regions that are colored in relation to a numeric variable

Code
import plotly.express as px

df = px.data.election()
geojson = px.data.election_geojson()

print(df["district"][2])
print(geojson["features"][0]["properties"])

df = px.data.election()
geojson = px.data.election_geojson()

fig = px.choropleth(
    df,
    geojson=geojson,
    color="Bergeron",
    locations="district",
    featureidkey="properties.district",
    projection="mercator",

)
fig.update_geos(fitbounds="locations", visible=False)
fig.update_layout(margin={"r": 0, "t": 0, "l": 0, "b": 0})
# fig.show()

# SAVE
file_name="img/plotly-1.html"
fig.write_html(file_name)

# RE-LOAD AS AN IFRAME
from IPython.display import IFrame
IFrame(src=file_name, width=800, height=400)
11-Sault-au-Récollet
{'district': '11-Sault-au-Récollet'}

Plotly: Choropleth-2

Code
from urllib.request import urlopen
import json
import pandas as pd
import plotly.express as px
from plotly import offline

# GET DATA
url1 = "https://raw.githubusercontent.com/plotly/datasets/master/geojson-counties-fips.json"
with urlopen(url1) as response:
    counties = json.load(response)

# print(counties)
# print(type(counties))

url2 = "https://raw.githubusercontent.com/plotly/datasets/master/fips-unemp-16.csv"
df = pd.read_csv(
    url2,
    dtype={"fips": str},
)


fig = px.choropleth(
    df,
    geojson=counties,
    locations="fips",
    color="unemp",
    color_continuous_scale="Viridis",
    range_color=(0, 12),
    scope="usa",
    labels={"unemp": "unemployment rate"},
)
out = fig.update_layout(margin={"r": 0, "t": 0, "l": 0, "b": 0})
# fig.show()
# offline.plot(out, filename="PLOTLY-1.html")

# SAVE
file_name="img/plotly-2.html"
out.write_html(file_name)

# RE-LOAD AS AN IFRAME
from IPython.display import IFrame
IFrame(src=file_name, width=800, height=400)

# out.show()

Graphical dis-honesty

As we’ve seen before, it’s important to use linear encodings for honest representations!

Example: The 2016 US Presidential Election Results

Plotly: Heat-map

  • Heat maps represent large sets of continuous data on a map using a color spectrum.
  • Unlike chloropleths, heat map colors don’t represent discrete geographical boundaries.
Code
import pandas as pd
import plotly.express as px

# GET DATA
df = pd.read_csv(
    "https://raw.githubusercontent.com/plotly/datasets/master/earthquakes-23k.csv"
)

# LOOK AT DATA STRUCTURE
print(df.head(1))

# GENERATE PLOT
fig = px.density_mapbox(
    df,
    lat="Latitude",
    lon="Longitude",
    z="Magnitude",
    radius=10,
    center=dict(lat=0, lon=180),
    zoom=0,
    mapbox_style="stamen-terrain",
)
# fig.show()

# SAVE
file_name="img/plotly-3.html"
fig.write_html(file_name)

# RE-LOAD AS AN IFRAME
from IPython.display import IFrame
IFrame(src=file_name, width=1000, height=500)
         Date  Latitude  Longitude  Magnitude
0  01/02/1965    19.246    145.616        6.0

Plotly: Dot map

  • A dot map (also called dot distribution map or dot density map) uses a dot to indicate the presence of a variable.
  • Dot maps are essentially scatterplots on a map and are used to show spatial patterns.

North American Precipitation Map

Code
import plotly.graph_objects as go

import pandas as pd

df = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/2015_06_30_precipitation.csv')

scl = [0,"rgb(150,0,90)"],[0.125,"rgb(0, 0, 200)"],[0.25,"rgb(0, 25, 255)"],\
[0.375,"rgb(0, 152, 255)"],[0.5,"rgb(44, 255, 150)"],[0.625,"rgb(151, 255, 0)"],\
[0.75,"rgb(255, 234, 0)"],[0.875,"rgb(255, 111, 0)"],[1,"rgb(255, 0, 0)"]

fig = go.Figure(data=go.Scattergeo(
    lat = df['Lat'],
    lon = df['Lon'],
    text = df['Globvalue'].astype(str) + ' inches',
    marker = dict(
        color = df['Globvalue'],
        colorscale = scl,
        reversescale = True,
        opacity = 0.7,
        size = 2,
        colorbar = dict(
            titleside = "right",
            outlinecolor = "rgba(68, 68, 68, 0)",
            ticks = "outside",
            showticksuffix = "last",
            dtick = 0.1
        )
    )
))

fig.update_layout(
    geo = dict(
        scope = 'north america',
        showland = True,
        landcolor = "rgb(212, 212, 212)",
        subunitcolor = "rgb(255, 255, 255)",
        countrycolor = "rgb(255, 255, 255)",
        showlakes = True,
        lakecolor = "rgb(255, 255, 255)",
        showsubunits = True,
        showcountries = True,
        resolution = 50,
        projection = dict(
            type = 'conic conformal',
            rotation_lon = -100
        ),
        lonaxis = dict(
            showgrid = True,
            gridwidth = 0.5,
            range= [ -140.0, -55.0 ],
            dtick = 5
        ),
        lataxis = dict (
            showgrid = True,
            gridwidth = 0.5,
            range= [ 20.0, 60.0 ],
            dtick = 5
        )
    ),
    title='US Precipitation 06-30-2015<br>Source: <a href="http://water.weather.gov/precip/">NOAA</a>',
)
# fig.show()

# SAVE
file_name="img/plotly-4.html"
fig.write_html(file_name)

# RE-LOAD AS AN IFRAME
from IPython.display import IFrame
IFrame(src=file_name, width=1000, height=450)

Dot map: word of caution

  • Dots are often used in graphs, charts, and maps to accurately locate individual observations and phenomena, but that’s not the case here. If you read a dot density map that way, it’ll look like there were fatalities everywhere in Florida, and that lightning strikes become much less deadly as soon as you cross the border with Georgia or Alabama.

  • In a dot density map, though, each dot represents one observation, but dots aren’t located where those observations were made; instead, dots are distributed to maximize coverage and, if the placement algorithm is well designed and manually tweaked, it’ll avoid absurd placement —such as dots over lakes, rivers, or unpopulated regions.

Plotly: Bubble maps

  • Bubble maps summarize your data by creating different sized circles that represent the value of the information in each geographic area.
  • The larger the bubble, the higher the value of your data in that area.
Code
import plotly.graph_objects as go

import pandas as pd

df = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/2014_us_cities.csv')
df.head()

df['text'] = df['name'] + '<br>Population ' + (df['pop']/1e6).astype(str)+' million'
limits = [(0,2),(3,10),(11,20),(21,50),(50,3000)]
colors = ["royalblue","crimson","lightseagreen","orange","lightgrey"]
cities = []
scale = 5000

fig = go.Figure()

for i in range(len(limits)):
    lim = limits[i]
    df_sub = df[lim[0]:lim[1]]
    fig.add_trace(go.Scattergeo(
        locationmode = 'USA-states',
        lon = df_sub['lon'],
        lat = df_sub['lat'],
        text = df_sub['text'],
        marker = dict(
            size = df_sub['pop']/scale,
            color = colors[i],
            line_color='rgb(40,40,40)',
            line_width=0.5,
            sizemode = 'area'
        ),
        name = '{0} - {1}'.format(lim[0],lim[1])))

fig.update_layout(
        title_text = '2014 US city populations<br>(Click legend to toggle traces)',
        showlegend = True,
        geo = dict(
            scope = 'usa',
            landcolor = 'rgb(217, 217, 217)',
        )
    )

# SAVE
file_name="img/plotly-5.html"
fig.write_html(file_name)

# RE-LOAD AS AN IFRAME
from IPython.display import IFrame
IFrame(src=file_name, width=1000, height=450)

Plotly: Hex map

  • A hex map geographic representation consisting of hexagonal tiles arranged in a honeycomb pattern, commonly used in strategy games and mapping applications.
  • To use mapbox-styles requiring a mapbox token, set the token with plotly.express. px.set_mapbox_access_token(open(".mapbox_token").read())
  • Alternatively you can use fig.update_layout(mapbox_style="open-street-map") without a token
Code
import pandas as pd
import plotly.figure_factory as ff

df = px.data.carshare()

import plotly.express as px

fig = ff.create_hexbin_mapbox(
    data_frame=df, lat="centroid_lat", lon="centroid_lon",
    nx_hexagon=10, opacity=0.9, labels={"color": "Point Count"},
)
fig.update_layout(margin=dict(b=0, t=0, l=0, r=0))
fig.update_layout(mapbox_style="open-street-map")  # 
fig.update_layout(margin={"r":0,"t":0,"l":0,"b":0})

# SAVE
file_name="img/plotly-6.html"
fig.write_html(file_name)

# RE-LOAD AS AN IFRAME
from IPython.display import IFrame
IFrame(src=file_name, width=1000, height=300)

Possible geo-visualization issues

Some immediate gotcha’s when working with geo-spatial data and trying to create visualizations

  • Layers won’t match up
  • Points will not show up on the right location on the grid
  • Distances won’t be correct
  • Circles turn to ellipses for no apparent reason
  • Geographic elements look weird

Geo-spatial visualization

Introduction to geo-spatial visualization in R.

Geo-spatial packages: R

  • Visualizing spatial and geographic data is a specialized area on its own
  • \(\mathrm{R}\) has increasing capabilities in this regard, and its increasingly mature.

Geospatial data

  • \(\mathrm{sf}\), \(\mathrm{sp}\)
  • raster, spData
  • rnaturalearthdata

Geospatial visualization

  • ggplot + ggmap + geom_sf
  • tmap
  • leaflet

Augmenting/Wrangling your data

  • Geocoding and reverse geocoding
    • Geocoding is a process of converting an address or a name of a place into its coordinates.
    • Reverse geocoding performs just an opposite task: returns an address or a description of a place based on its coordinates.
  • Spatial joins
    • Joins attributes from one feature to another based on the spatial relationship.
    • The target features and the joined attributes from the join features are written to the output feature class.

Simple features package

SF object methods

  • The sf package provides simple features access for R, and helps to store and process geographic data within the tidyverse framework, while linking to several geospatial packages that are standard in the geography world.
  • we can use the follow to see the available methods provided for sf objects
Code
library(sf); 
#LIST METHODS FOR SF OBJECTS
print(methods(class = "sf"))
 [1] $<-                          [                           
 [3] [[<-                         aggregate                   
 [5] as.data.frame                cbind                       
 [7] coerce                       dbDataType                  
 [9] dbWriteTable                 filter                      
[11] identify                     initialize                  
[13] merge                        plot                        
[15] print                        rbind                       
[17] show                         slotsFromS3                 
[19] st_agr                       st_agr<-                    
[21] st_area                      st_as_s2                    
[23] st_as_sf                     st_as_sfc                   
[25] st_bbox                      st_boundary                 
[27] st_break_antimeridian        st_buffer                   
[29] st_cast                      st_centroid                 
[31] st_collection_extract        st_convex_hull              
[33] st_coordinates               st_crop                     
[35] st_crs                       st_crs<-                    
[37] st_difference                st_drop_geometry            
[39] st_filter                    st_geometry                 
[41] st_geometry<-                st_inscribed_circle         
[43] st_interpolate_aw            st_intersection             
[45] st_intersects                st_is                       
[47] st_is_valid                  st_join                     
[49] st_line_merge                st_m_range                  
[51] st_make_valid                st_minimum_rotated_rectangle
[53] st_nearest_points            st_node                     
[55] st_normalize                 st_point_on_surface         
[57] st_polygonize                st_precision                
[59] st_reverse                   st_sample                   
[61] st_segmentize                st_set_precision            
[63] st_shift_longitude           st_simplify                 
[65] st_snap                      st_sym_difference           
[67] st_transform                 st_triangulate              
[69] st_union                     st_voronoi                  
[71] st_wrap_dateline             st_write                    
[73] st_z_range                   st_zm                       
[75] transform                   
see '?methods' for accessing help and source code

ggplot: choropleth of life expectancy

If you look at world, it also provides life expectancy estimates from 2014 (World Bank). The data set is tidy, with one row corresponding to one country. We’ll use our known ggplot2 way of filling things in.

Code
library(ggplot2)
library(sf)
library(spData)

# ggplot(data = world) +  geom_sf() # a special geometry for plotting maps
ggplot(data = spData::world) + geom_sf(aes(fill = lifeExp)) # a special geometry for plotting maps

ggplot(data = spData::world) + 
  geom_sf(aes(fill = lifeExp)) + # a special geometry for plotting maps
  viridis::scale_fill_viridis('Life Exp', option='plasma', 
                              trans='sqrt', labels = scales::label_number_si())

Example: another choropleth

Code
#MODIFIED from https://r-spatial.github.io/sf/articles/sf1.html
library(sf); 

#FIND PATH ON COMPUTER 
path=system.file("shape/nc.shp", package="sf")
# print(path)

#READ SHAPE FILE 
nc <- st_read(system.file("shape/nc.shp", package="sf"))

#EXPLORE
# print(c("class",class(nc)))
# print(nc)
# print(nc[9:15], n = 3)

#CONVERT TO REGULAR DF
nc.no_sf <- as.data.frame(nc)
# print(class(nc.no_sf))

#GET GEOMETRY COMPONENT 
nc_geom <- st_geometry(nc)
# print(nc_geom)

#SF PLOTING FEATURE: EXAMPLE-1
# par(mar = c(0,0,1,0))
# reset = FALSE: we want to add to a plot with a legend
plot(nc[1], reset = FALSE) 
plot(nc[1,1], col = 'grey', add = TRUE)

# #SF PLOTING FEATURE: EXAMPLE-2:
# par(mar = c(0,0,1,0))
# (w <- which(sapply(nc_geom, length) > 1))
# ## [1]  4 56 57 87 91 95
# plot(nc[w,1], col = 2:7)
Reading layer `nc' from data source 
  `/Library/Frameworks/R.framework/Versions/4.2/Resources/library/sf/shape/nc.shp' 
  using driver `ESRI Shapefile'
Simple feature collection with 100 features and 14 fields
Geometry type: MULTIPOLYGON
Dimension:     XY
Bounding box:  xmin: -84.32385 ymin: 33.88199 xmax: -75.45698 ymax: 36.58965
Geodetic CRS:  NAD27

R: Leaflet

The easiest ways to overlay data on street maps is with the leaflet package.

library(leaflet)
library(leaflet.providers)
library(htmlwidgets)
load("data/exdata.rda")
fig <- leaflet(gpx) %>% addTiles() %>% addCircleMarkers(~Longitude, ~Latitude, radius=1)

# htmlwidgets::saveWidget(fig, "img/leaflet-R.html", selfcontained=TRUE)
saveWidget(fig, file="img/leaflet-R.html")

Cartogram

A cartogram is a map in which the geometry of regions is distorted in order to convey the information of an alternate variable. The region area will be inflated or deflated according to its value.

  1. Start with a basic plot
  2. Distort geometry based on chosen variable
  3. Add color and now you have both a choropleth and a cartogram

References

  • https://python-charts.com/spatial/interactive-maps-folium/
  • https://cran.r-project.org/web/packages/sp/index.html
  • https://cran.r-project.org/web/packages/sf/index.html
  • https://keen-swartz-3146c4.netlify.app/
  • https://bookdown.org/robinlovelace/geocompr/
  • https://www.paulamoraga.com/book-geospatial/sec-spatialdataandCRS.html
  • https://spatialanalysis.github.io/tutorials/
  • https://www.jessesadler.com/post/geocoding-with-r/
  • http://https://www.r-graph-gallery.com/index.html
  • https://community.rstudio.com/t/modern-geospatial-data-analysis-with-r-workshop-rstudio-conf-2020/49098

Disclaimer: This document is not an original work. It contains content from many sources and is for educational purposes only.

Break

Lets take a 10 minute break before moving onto the lab.