Getting Started: Loading, Processing, Visualizing, and Storing Data¶
This short notebook demostrates how you can get started with landlensdb can be used to load, process, visualize, and store street-view data from local file directories.
Pre-requisites¶
Before getting started, you will need to have PostgreSQL and PostGIS installed.
PostgreSQL and PostGIS¶
Once PostgreSQL and PostGIS are installed, or if they are already installed, then you will need to have a PostGIS enabled PostgreSQL database to work with. To create one, use:
createdb <database_name> && psql <database_name> -c "CREATE EXTENSION POSTGIS"
Be sure to replace <database_name> with the name you want to call your database. For example, it could be:
createdb landlens && psql landlens -c "CREATE EXTENSION POSTGIS"
Once this is done, you should be ready to proceed with the tutorial.
If you don't have landlensdb installed, you can do so with pip install landlensdb.
import os
import geopandas as gpd
from landlensdb.handlers.image import Local
from landlensdb.process.snap import snap_to_road_network
from landlensdb.handlers.db import Postgres
from landlensdb.geoclasses.geoimageframe import GeoImageFrame
1. Loading images from local directory¶
To load images from a local directory, simply call the load_images function while providing the source directory to read from. Currently, only jpeg images are supported and it is best to provide the full path to the images.
relative_path = "../example_data/360_images"
absolute_path = os.path.abspath(relative_path)
images = Local.load_images(absolute_path, create_thumbnails=False)
images.head()
The resulting image is a GeoImageFrame, which is a simple extension of a Pandas GeoDataFrame with a few required column definitions and additional methods for visualization and data verification.
Processing Images¶
Now that we have loaded some data, we can perform some simple processing on the images. Check the documentation for the current processing functions available. Here is an example of how landlensdb can be used to snap images to road networks.
Snapping to a Road Network¶
First, we need a road network to snap your images to. landlensdb also offers a helper function to download road networks from Open Street Map within a given bounding box. However, you can also load your own road (or path) network if you have one. We will use our own in this case since these images are taken along an unmapped forest trail.
network = gpd.read_file('../example_data/trail.geojson')
Then, calling the snap_to_road_network will snap all points to the closest road network (within the provided threshold distance) and will create a new geometry column in the GeoImageFrame falled snapped_geometry to represent this new point.
snap_to_road_network(images, 100, network, realign_camera=True).head()
Visualizing Images¶
landlensdb provides a simple way to visualize its GeoImageFrames interactively using Folium. The map method of a GeoImageFrame will plot all images as markers on a map and will display the image on click along with any metadata set using the additional_properties argument as well as markers for any provided additional geometry.
images.map(
additional_properties=['altitude', 'camera_type'],
additional_geometries=[
{'geometry': 'snapped_geometry', 'angle': 'snapped_angle', 'label': 'Snapped'},
]
)
Storing Images¶
GeoImageFrame data can be stored in a variety of formats. Given that it is built on GeoPandas the GeoDataFrame class, it will take any geodataframe method to save data. For instance, to save a table as a geopackage, we simply call:
images.to_file('./images_tutorial.gpkg')
However, in the current version when reading a saved vector format it is important to then initialize the GeoDataFrame as a GeoImageFrame if you want to make use of the features of landlensdb. For example:
images_gdf = gpd.read_file('./images_tutorial.gpkg')
images = GeoImageFrame(images_gdf)
Saving to a PostgreSQL Database¶
landlensdb also offers functionality to store data in a PostGIS enabled PostgreSQL database. This is done by extending the to_postgis method of GeoPandas. There are some constraints, such as unique image_urls, that are automatically applied when storing data, as well as some data validity checks -- see the documentation for details.
To save a GeoImageFrame to a PostgreSQL table, you will need to first initiate a connection to a PostgreSQL database. You can do this using the ImageDB class:
db_con = Postgres("postgresql://localhost:5432/landlens")
This database must already exist and have PostGIS loaded.
Then, you can save using to_postgis:
images.to_postgis("tutorial", db_con.engine, if_exists="replace")
Updating an Existing Table¶
When saving to PostgreSQL, you can choose to handle existing tables. to_postgis offers the same fail, replace and append methods that GeoPandas offers, however, append requires that all data going in will not conflict with any existing data. Instead, it is possible to "upsert" (insert and update) data into existing tables using the upsert_images class method of Image_DB. You may choose to either update conflicting records or skip them by declaring "update" or "nothing" in the conflict argument of the function.
db_con.upsert_images(images, "tutorial", conflict='update')
Querying an Existing Table¶
It is also possible to load and filter data from existing postgres connections. landlensdb offers simple filter functions to query and filter tables to provide a subset of the data. This can be important when working with very large datasets. For example, to load all images with an altitude greater than 50:
high_altitude_images = db_con.table("tutorial").filter(altitude__gt=170).all()
high_altitude_images.map(
additional_properties=['altitude', 'camera_type']
)
Finally, you can save the map as an html file if you like:
high_alt_map = high_altitude_images.map(
additional_properties=['altitude', 'camera_type']
)
high_alt_map.save('./high_alt_map.html')