Setup and Configuration

The pannotator application can be opened in any web browser. Once running, settings can be accessed using the cog icon at the top corner of the Annotation Panel. The drop down window contains three panels: a ‘Main Settings’ tab, a ‘Lookups’ tab, and an ‘About This Software’ tab (Figure 2, left). In the Main Settings tab you can change general settings. These are grouped into 1) Layout Settings, including the size of the Mapping, Image and Annotation Panels, turning on or off popup alert windows, and altering the theme for the entire application, 2) Mapping Panel Settings, which includes setting the background map, icons and markers for points, rectangles and polygons drawn on maps, and the colour, weight and opacity of the stroke (outline) and the fill of polygons and rectangles, 3) Image Panel Settings, which includes the same functionality for drawing and formatting shapes but on 360-degree flattened equirectangular images in the Image Panel, and the capacity to turn off borders and fill on the exported version of cropped sub-images (‘Show Polygon Stroke/Fill in Cropped Image Export’ checkboxes), and 4) Annotation Panel Settings which includes choosing a custom user name lookup file (Figure 2, centre & Figure 3, right) and defining the format of the annotation export file (e.g., .csv or .rds).

Main Settings (left and centre) and Lookups (right). The Lookups are tables that appear as dropdown menus in the Annotation Panel (below) and are used to add data annotations to images, markers and drawn geometries.

In the ‘Lookups’ Panel (Figure 2, right) you can specify up to four data fields that will be used to annotate the images. The name of the data field name can be specified in the ‘Lookup Label’ text boxes. The categories for the data field are specified in an accompanying .csv file which can be linked using the ‘Lookup csv File’ browse buttons. The Lookup csv file has two columns: 1) a ‘display’ column and 2) a ‘value’ column (Figure 3, left). The text in the ‘display’ column will appear in the Lookup drop down menu and the ‘value’ will saved to the annotation file. This way your ‘display’ category can differ from the data field in the export file if needed. An accompanying .pdf help file can be uploaded to assist the annotator using the Lookup Help File browse buttons. The help file can then be accessed using buttons created in the Annotation Panel (see below).

Example .csv lookup files; user name (left) and general lookup variable, in this case plant species (right).

Note: once changes have been made to settings and configurations you can update them by clicking the ‘Apply Settings’ button.

Warning! If you change and update Lookup tables while in the process of annotating images, the structure of the annotation dataframe and exported annotation data will also change. This may affect the validity of data.

Viewing Mode

In Viewing Mode the Pannellum web viewer loads an equirectangular image selected from those in the .kmz file using the ‘Image to Annotate’ dropdown menu and renders it in interactive 360-degree viewing mode. The panel has buttons for zoom in, zoom out and full screen mode and also contains the compass bearing of the direction in which the image is being viewed (if present in metadata).

The 360-degree Viewing Mode allows the user to extract target information from each selected image. Figure 6 (below) shows several key ecological applications, including species identification and mapping (left), estimation of vertically projected ground-layer vegetation cover (centre), and estimation of crown health for individual trees (right). In each case data annotation is performed in the Annotation Panel using help files as visual and/or descriptive aids (see below).

Estimation of vertically projected ground layer cover may be assisted by overlaying sampling templates constructed from reference images (eg., camera images containing plots of specified size laid out using markers or tape etc.) onto each sample image. In this example (Figure 6 centre) a 10 m diameter plot is shown, centred on the vertically projected camera location. The plot has been broken into 4, 20 and 100 equal areal increments (90, 18 and 3.6 degrees) to assist with cover estimation of scattered and/or small taxa. The two orthogonal lines that divide the plot have been marked with 1 m increments to assist the user in estimating size and cover while taking into account image distortion.

One key benefit of sampling in transects or in a regular pattern is that objects may be viewed from multiple perspectives for identification and/or classification. This is advantageous when sampling objects with obstructed views or large, three-dimensional objects that are not entirely visible from a single perspective.

Common ecological applications using the image panel. Left: species identification and mapping. Four candidate species are labelled (A-D). The plants labelled C and D can be located by dropping a pin in the Mapping Panel. Centre: cover estimation of ground layer vegetation. Cover may be assisted by comparison with example 25%, 5% and 1% areas. Right: Crown health classification. In this example the intent is to estimate the percentage of the potential crown that has healthy green leaves. Images can be drawn on and cropped and exported in Drawing Mode (below) .

Drawing Mode

When the Drawing Mode is enabled by clicking the ‘Switch to Drawing Mode’ button the Leaflet plugin loads a flattened equirectangular image in the Image Panel (i.e., not in 360-degree viewing format; Figure 7). A toolbar allows the user to draw point markers, rectangles and more complex polygons onto the equirectangular image, in the same way as the Mapping Panel. This allows the user to select sub-images (‘patches’ or ‘crops’) or locations on the original image that are of interest (e.g., vegetation or landforms; Figure 7). Any drawn geometric object can be moved and rectangles and polygons can be re-drawn. The display of geometric objects added in Drawing Mode (e.g., icon colour, marker colour, and the colour and opacity of the stroke (border) and fill of each rectangle or polygon) can be altered from the Image Panel Settings in the Main Settings tab.

Drawing Mode allows the user to draw points, rectangles and polygons on flattened equirectangular images.

When a point or shape object is drawn onto the equirectangular image an ID number is created (based on timestamp) and added to the underlying annotation dataframe, along with its geometry (based on pixel-location(s)). A linked dropdown menu also opens in the Annotation Panel for adding data based on user-defined lookup tables (see below). Geometry icons and IDs are shown in both Annotation dropdowns and in the Image Panel (ID if rolled over, ID and icon if clicked). The metadata containing ID, geometry and annotations are automatically saved to the annotation dataframe and all records can be exported as a .csv/.rds using the ‘Export All Records’ button in the Annotation Panel.

Finally, rectangle and polygon geometries drawn on a selected image can be exported using the ‘Export Cropped Polygon Images’ button. This generates a cropped image file (.png) of the underlying equirectangular image based on the bounding box of the drawn geometries. If the geometry stroke (border) and fill are enabled the image is cropped to a bounding box with a small buffer, and if it is disabled the image is cropped to the exact dimensions of the geometry bounding box (Figure 7). The stroke and fill of polygons on the exported version of cropped sub-images can be turned on and off using the ‘Show Polygon Stroke/Fill in Cropped Image Export’ checkboxes in the Image Panel Settings of the Main Settings (Figure 2, centre).

The name of a cropped image consists of the associated geometric object’s ID appended to the original 360-degree image name, while the geolocation of the underlying image is embedded in the metadata. Cropped images are saved in a user-selected folder.

Adding Data Records

A new data record is added to a selected equirectangular image by clicking the ‘Add A Whole Image Annotation’ button (Figure 8). In the example provided the presence of the plant species (desert oak, Allocasuarina decaisneana) is added to the image annotation. Dropdown menus for adding records are user defined according to the requirements of the project. Multiple new records can be added to each whole image; these are clustered in the Mapping Panel and exploded by clicking the cluster icon.

Adding a record to the whole image marker. The process is the same for points, rectangles and polygons drawn on maps or 360-degree equirectangular images.

When point markers, rectangles or polygons are added to an image in either the Mapping Panel (e.g., on a satellite image) or the Image Panel (to an equirectangular image) a new data record is automatically generated in the Annotation Panel. All records contain an icon type (point-map, point-whole-image-annotation and polygon-map for objects in the Mapping Panel and point-360 and polygon-360 for objects in the Image Panel) to which the specific dropdown label name is appended (e.g., polygon-360-Species), and also an object ID (timestamp). Individual annotation records can be deleted using the ‘trash’ button and shown or hidden using the chevron button.

Help files

Data records are added with the assistance of help files accessed using the associated button. In this example the help file provided contains images of species present in the study area (Figure 8). Help files developed by the user can be uploaded as required. Examples include reference images for species identity (Figure 9, top row), cover classes (Figure 9, centre row) and size class (Figure 9, bottom row).

Example help files for collection of ecological data from 360 degree images.

Exporting Data

Data may be exported in .csv and .rds formats. An example .rds file viewed in R is provided below (Figure 9). The file contains user name, ID, image file, feature type (image marker or point marker), geometry and the four data fields specified in the Annotation Panel drop down menu.

Exported data file containing annotations appended to camera image metadata.

Rectangle and polygon geometries drawn on a selected image in Drawing Mode of the Image Panel can be exported using the ‘Export Cropped Polygon Images’ button. This generates a cropped image file(s) (.png) which is/are saved to a user-defined folder.

Visualising and Validating Data

Below is a simple example of R code for visualising a sample of exported data using the R package mapview. Quarto files that contain a complete set of code for extracting and visualising species distribution data, cover data and tree crown health data, and that validate data extracted from images compared to data collected in the field are provided in the accompanying pannotator_examples and pannotator_data_validation files.

This example code generates a plot of scores on the variable dd2 (percentage of tree crown with live leaves) using the package mapview.

library(dplyr)
library(mapview)
library(RColorBrewer)
library(sf)


df_annotation <- readRDS("C:/user_1_annotations.rds") # read in the .rds file
df_annotation <- st_as_sf(df_annotation, wkt = "geometry",crs = 4326) #define 
#the geometry

df_annotation$dd2 <- as.numeric(df_annotation$dd2) # dd2 = -999 where crowns 
# have not been assessed for health (NA); range = 0 for no live leaves (entirely
# dead) to 100 for entire crown healthy with green leaves

df_annotation <- subset(df_annotation, dd2 > -1 ) # select only records where 
# Allocasuarina crowns have been assessed for health; that is, excluding NA records


mapviewOptions(basemaps = c("Esri.WorldImagery"),
               vector.palette = colorRampPalette(c("red","orange", "yellow", "green")),
               layers.control.pos = "topright")


mapview(df_annotation, zcol = "dd2 ", na.rm = TRUE)

Map generated using mapview.

This example code generates a plot of scores of the relationship between the number of plant species (i.e., species richness) recorded in a field survey (x axis) and camera survey (y axis) of the same plots (n = 79). It also fits a linear model to the relationship, plots the line of best fit, and prints the model statistics.

#read in the species data file#

species_data <- read_csv("Calibration_species.csv", show_col_types = FALSE)

# confirm that there are 79 plots of species data
cat("The number of rows in the dataframe is", nrow(species_data), "\n")


# now determine the relationship between plot-level species counts in the field survey versus camera survey counts. abline adds a linear model to the plot   #

plot(species_data$No_Field_species, species_data$No_Camera_species,
     main = "Plot-level species richness",
     xlab = "No. of species (Field Survey)",
     ylab = "No. of species (Camera Survey)",
     pch = 16,  # Use filled circles as data points
     col = "black",  # Set point color 
     ylim = c(0, 8),  # Set y-axis limits
     xlim = c(0, 8))  # Set x-axis limits
abline(lm(No_Camera_species ~ No_Field_species, data = species_data), col = "red")

# view the linear model statistics #
model <- lm(No_Camera_species ~ No_Field_species, data = species_data)

model_summary <- summary(model)
print(model_summary)

Relationship between number of plant species recorded in a field survey (x axis) and camera survey (y axis) of the same plots.