Access Data Stored in a Postgresql Database
Overview
Perform some temporal aggregation (by day and by week) of the amount of data entry completed in Postgresql, and plot the results in R. See resulting figure at the bottom of the page. Note that this requires the Rdbi and RdbiPgSQL packages. Hints on installing these packages can be found on this page...
Weekly Aggregation hints from the psql manual page
SELECT week, count(week) AS enteredFROM(SELECT pedon_id, creation_date, extract( week FROM creation_date) AS weekFROM descriptionORDER BY creation_date ASC) AS aGROUP BY a.weekORDER BY week;X
Daily Aggregation hints from the psql manual page
SELECT doy, count(doy) AS enteredFROM(SELECT pedon_id, creation_date, extract( doy FROM creation_date) AS doyFROM descriptionORDER BY creation_date ASC) AS aGROUP BY a.doyORDER BY doy;X
R Example
##### load the samme data in from PgSQL #####library(Rdbi)library(RdbiPgSQL)# conn becomes an object which contains the DB connection:conn <- dbConnect(PgSQL(), host="localhost", dbname="xxx", user="xxx", password="xxx")# see if the connection works (should report the list of table(s) if table(s) are existing):# dbListTables(conn)## create an object which contains the SQL query:query <- dbSendQuery(conn, "select pedon_id, hz_number, name, top, bottom, ((bottom - top)/2 + top) as avgdepth, matrix_wet_color_hue, matrix_wet_color_value, matrix_wet_color_chroma, matrix_dry_color_hue, matrix_dry_color_value, matrix_dry_color_chroma from horizon order by pedon_id, hz_number ")# fetch data according to query:x <- dbGetResult(query)# create an object which contains the SQL query:query <- dbSendQuery(conn, "select doy, count(doy) as entered from (select pedon_id, creation_date, extract( doy from creation_date) as doy from description order by creation_date asc ) as a group by a.doy order by doy;")# fetch data according to query:y <- dbGetResult(query)# setup plot environmentpar(mfrow=c(2,1))# plot cumulative progress, by weekplot(x$week, cumsum(x$entered), type='b', xlab='Week', ylab='Pedon Forms Completed', main='Weekly Progress')# plot cumulative progress, by dayplot(y$doy, cumsum(y$entered), type='b', xlab='Day of Year', ylab='Pedon Forms Completed', main='Daily Progress')X
Pedon entry progress
Software
- General Purpose Programming with Scripting Languages
- LaTeX Tips and Tricks
- PostGIS: Spatially enabled Relational Database Sytem
- PROJ: forward and reverse geographic projections
- GDAL and OGR: geodata conversion and re-projection tools
- R: advanced statistical package
- Access Data Stored in a Postgresql Database
- Additive Time Series Decomposition in R: Soil Moisture and Temperature Data
- Aggregating SSURGO Data in R
- Cluster Analysis 1: finding groups in a randomly generated 2-dimensional dataset
- Color Functions
- Comparison of Slope and Intercept Terms for Multi-Level Model
- Comparison of Slope and Intercept Terms for Multi-Level Model II: Using Contrasts
- Creating a Custom Panel Function (R - Lattice Graphics)
- Customized Scatterplot Ideas
- Estimating Missing Data with aregImpute() {R}
- Exploration of Multivariate Data
- Interactive 3D plots with the rgl package
- Making Soil Property vs. Depth Plots
- Numerical Integration/Differentiation in R: FTIR Spectra
- Plotting XRD (X-Ray Diffraction) Data
- Using lm() and predict() to apply a standard curve to Analytical Data
- Working with Spatial Data
- Comparison of PSA Results: Pipette vs. Laser Granulometer
- GRASS GIS: raster, vector, and imagery analysis
- Generic Mapping Tools: high quality map production