• Welcome
  • I Introduction
  • 1 Project Overview
  • 2 Contributor Covenant Code of Conduct
  • 3 About the PEcAn Book
    • 3.1 General Feedback/Comments/Suggestions
    • 3.2 Editing this book
    • 3.3 How to create your own version of Documentation
  • II Tutorials, Demos and How To’s
  • 4 Install PEcAn
    • 4.1 Docker
    • 4.2 Installation From r-universe
    • 4.3 (Advanced) Native install
  • 5 Tutorials
    • 5.1 Demo 01: Basic Run PEcAn
    • 5.2 Demo 02: Sensitivity and Uncertainty Analysis
      • 5.2.1 Run Specification
      • 5.2.2 Additional Outputs:
    • 5.3 Other Vignettes
      • 5.3.1 Simple Model-Data Comparisons
      • 5.3.2 Data Assimilation Concepts
      • 5.3.3 Parameter Data Assimilation
      • 5.3.4 State-Variable Data Assimilation
      • 5.3.5 PEcAn: Testing the Sensitivity Analysis Against Observations”
    • 5.4 Advanced User Guide
      • 5.4.1 Submitting Workflow from Command Line
  • 6 Basic Web workflow
    • 6.1 Site and model selection
      • 6.1.1 Selecting a model
      • 6.1.2 Selecting a site
      • 6.1.3 Site Groups
      • 6.1.4 Using existing sites
      • 6.1.5 Adding a new site
      • 6.1.6 Troubleshooting
    • 6.2 Model configuration
      • 6.2.1 Choosing meteorology
      • 6.2.2 Met workflow
      • 6.2.3 Troubleshooting meteorological conversions
    • 6.3 Selecting Plant Functional Types (PFTs) and other parameter groupings.
      • 6.3.1 Using existing PFTs
      • 6.3.2 Creating new PFTs
      • 6.3.3 Choosing initial vegetation
      • 6.3.4 US FIA
      • 6.3.5 Spin up
      • 6.3.6 Selecting a soils product
      • 6.3.7 Soil texture, depth, and physical parameters
      • 6.3.8 Other model inputs
  • 7 More on the PEcAn Web Interface
    • 7.1 Additional web configuration
      • 7.1.1 Web interface setup
      • 7.1.2 Advanced Setup
      • 7.1.3 Editing model configurations
    • 7.2 Settings-configured analyses
      • 7.2.1 Parameter data assimilation (PDA)
      • 7.2.2 State data assimilation (SDA)
      • 7.2.3 Running SDA on remote
      • 7.2.4 Restart functionality in SDA
      • 7.2.5 State Data Assimilation Methods
      • 7.2.6 MultiSettings
      • 7.2.7 Benchmarking
  • 8 Developer guide
    • 8.1 Updating PEcAn Code and Bety Database
      • 8.1.1 Updating PEcAn
    • 8.2 Git and GitHub Workflow
      • 8.2.1 Using Git
      • 8.2.2 Recommended Git Workflow
      • 8.2.3 Useful Git tools
      • 8.2.4 Advanced
      • 8.2.5 References
      • 8.2.6 GitHub use with PEcAn
    • 8.3 Coding Practices
      • 8.3.1 Coding Style
  • 9 dplyr
    • 9.0.1 Logging
    • 9.0.2 Package Data
    • 9.0.3 Documenting functions using roxygen2
    • 9.1 Testing
      • 9.1.1 Unit testing
      • 9.1.2 Integration testing
      • 9.1.3 Continuous Integration
    • 9.2 Download and Compile PEcAn
      • 9.2.1 Download, compile and install PEcAn from GitHub
      • 9.2.2 PEcAn Testrun
    • 9.3 Directory structure
      • 9.3.1 Overview of PEcAn repository as of PEcAn 1.5.3
      • 9.3.2 Generic R package structure:
  • III Topical Pages
  • 10 PEcAn standard formats
    • 10.1 Defining new input formats
    • 10.2 Time Standard
    • 10.3 Input Standards
      • 10.3.1 Meteorology Standards
      • 10.3.2 Soils and Vegetation Inputs
    • 10.4 Output Standards
  • 11 The PEcAn XML
    • 11.1 Core configuration
      • 11.1.1 Top-level structure
      • 11.1.2 info: Run metadata
      • 11.1.3 outdir: Output directory
      • 11.1.4 database: PEcAn database settings
      • 11.1.5 pft: Plant functional type selection
      • 11.1.6 meta.analysis: Trait Meta Analysis
      • 11.1.7 model: Model configuration
      • 11.1.8 run: Run Setup
      • 11.1.9 host: Host information for remote execution
    • 11.2 Advanced features
      • 11.2.1 ensemble: Ensemble Runs
      • 11.2.2 sensitivity.analysis: Sensitivity analysis
      • 11.2.3 Parameter Data Assimilation
      • 11.2.4 Multi-Settings
      • 11.2.5 (experimental) State Data Assimilation
      • 11.2.6 State Variables for State Data Assimilation
      • 11.2.7 Observation Preparation for State Data Assimilation
      • 11.2.8 (experimental) Benchmarking
      • 11.2.9 Remote data module
  • 12 PEcAn workflow (web/workflow.R)
    • 12.1 Read Settings
    • 12.2 Input Conversions
    • 12.3 Input Data
    • 12.4 Initial Conditions
      • 12.4.1 CONUS (NEON/FIA/BADM) Initial Conditions.
      • 12.4.2 North America (NA) Initial Conditions.
    • 12.5 Meteorological Data
      • 12.5.1 Downloading Raw data (Description of Process)
      • 12.5.2 Converting raw data to PEcAn standard
      • 12.5.3 Downscaling and gapfilling (optional)
      • 12.5.4 Converting from PEcAn standard to model-specific format
    • 12.6 Input phenological Data
    • 12.7 Traits
    • 12.8 Meta Analysis
    • 12.9 Model Configuration
    • 12.10 Run Execution
    • 12.11 Post Run Analysis
  • 13 PEcAn Models
    • 13.1 BioCro
      • 13.1.1 Introduction
      • 13.1.2 PEcAn configuration file additions
      • 13.1.3 Model specific input files
      • 13.1.4 Model configuration files
      • 13.1.5 Installation notes
    • 13.2 CLM
    • 13.3 DALEC
    • 13.4 ED2
      • 13.4.1 Introduction
      • 13.4.2 PEcAn configuration file additions
      • 13.4.3 PFT configuration in ED2
      • 13.4.4 Model specific input files
      • 13.4.5 Model configuration files
      • 13.4.6 Installation notes
    • 13.5 GDAY
    • 13.6 LDNDC
      • 13.6.1 Introduction
      • 13.6.2 Model specific input files
      • 13.6.3 Installation notes
    • 13.7 LINKAGES
    • 13.8 LPJ-GUESS
    • 13.9 MAESPA
    • 13.10 PRELES
    • 13.11 SiPNET
    • 13.12 STICS
  • 14 Available Meteorological Drivers
    • 14.1 Ameriflux
    • 14.2 AmerifluxLBL
    • 14.3 Fluxnet2015
    • 14.4 NARR
    • 14.5 CRUNCEP
    • 14.6 CMIP5
    • 14.7 NLDAS
    • 14.8 GLDAS
    • 14.9 PalEON
    • 14.10 FluxnetLaThuile
    • 14.11 Geostreams
    • 14.12 ERA5
    • 14.13 ICOS Drought 2018
    • 14.14 ICOS Ecosystem Archive
    • 14.15 Download GFDL
    • 14.16 CM3
    • 14.17 ESM2M & ESM2G
  • 15 PEcAn Project API
    • 15.1 Introduction
    • 15.2 Authentication
    • 15.3 RESTful API Endpoints
    • 15.4 Examples:
      • 15.4.1 Prerequisites to interact with the PEcAn API Server
      • 15.4.2 GET /api/ping
      • 15.4.3 GET /api/status
      • 15.4.4 GET /api/models/
      • 15.4.5 GET /api/models/{model_id}
      • 15.4.6 GET /api/sites/
      • 15.4.7 GET /api/sites/{site_id}
      • 15.4.8 GET /api/pfts/
      • 15.4.9 GET /api/pfts/{pft_id}
      • 15.4.10 GET /api/formats/
      • 15.4.11 GET /api/formats/{format_id}
      • 15.4.12 GET /api/inputs/
      • 15.4.13 GET /api/inputs/{input_id}
      • 15.4.14 GET /api/workflows/
      • 15.4.15 POST /api/workflows/
      • 15.4.16 GET /api/workflows/{id}
      • 15.4.17 GET /api/workflows/{id}/status
      • 15.4.18 GET /api/workflows/{id}/file/{filename}
      • 15.4.19 GET /api/runs/
      • 15.4.20 GET /api/runs/{run_id}
      • 15.4.21 GET /api/runs/{run_id}/input/{filename}
      • 15.4.22 GET /api/runs/{run_id}/output/{filename}
      • 15.4.23 GET /api/runs/{run_id}/graph/{year}/{y_var}
  • 16 Database synchronization
    • 16.1 How does it work?
    • 16.2 Set up
    • 16.3 Fetch latest data
    • 16.4 Sharing data
    • 16.5 Automation
    • 16.6 Database maintentance
      • 16.6.1 Reindexing the database
      • 16.6.2 Vacuuming the database
    • 16.7 Troubleshooting
      • 16.7.1 Username and password
      • 16.7.2 db_hba.conf file
    • 16.8 Network Status Map
    • 16.9 Tasks
  • 17 Standalone tools (modules)
    • 17.1 Loading Data in PEcAn
      • 17.1.1 Inputs
      • 17.1.2 Output
      • 17.1.3 Example
    • 17.2 Remote data module
      • 17.2.1 Configuring remote_process
  • 18 Shiny
    • 18.1 Testing the Shiny Server
    • 18.2 Debugging Shiny Apps
    • 18.3 Checking Log Files
  • 19 Adding to PEcAn
    • 19.1 Adding An Ecosystem Model
      • 19.1.1 Using PEcAn Database
      • 19.1.2 Define MODEL_TYPE
      • 19.1.3 MACHINE
      • 19.1.4 MODEL
      • 19.1.5 FORMATS
      • 19.1.6 MODEL_TYPE -> Formats
      • 19.1.7 INPUTS
      • 19.1.8 Add Plant Functional Types (PFTs)
      • 19.1.9 Adding Priors for Each Variable
      • 19.1.10 Interface Modules
    • 19.2 Example met conversion wrapper function
    • 19.3 Adding input data
      • 19.3.1 Input records in BETY
      • 19.3.2 Create a database file record for the input data
      • 19.3.3 Creating a new Input record in BETY
      • 19.3.4 Adding a new input converter
    • 19.4 Pecan Data Ingest via Web Interface
      • 19.4.1 Loading Data
      • 19.4.2 Local Upload Example
      • 19.4.3 3. Creating a format record
    • 19.5 Creating a new format
      • 19.5.1 Formats in BETY
      • 19.5.2 Creating a new format in BETY
    • 19.6 Creating a new benchmark reference run
    • 19.7 Editing records
  • 20 Troubleshooting and Debugging PEcAn
    • 20.1 Cookies and pecan web pages
    • 20.2 Warning: mkdir() [function.mkdir]: No such file or directory
    • 20.3 After creating a new PFT the tag for PFT not passed to config.xml in ED
    • 20.4 Debugging
      • 20.4.1 Using tests/workflow.R
    • 20.5 Useful scripts
  • 21 BETY Database Administration
    • 21.1 Best practices
    • 21.2 Backup of BETY database
    • 21.3 Restore of BETY database
  • 22 Workflow modules
    • 22.1 Overview
    • 22.2 Load Settings
      • 22.2.1 read.settings("/home/pecan/pecan.xml")
    • 22.3 Query Database
      • 22.3.1 get.trait.data()
    • 22.4 Meta Analysis
      • 22.4.1 run.meta.analysis()
    • 22.5 Write Configuration Files
      • 22.5.1 write.configs(model)
    • 22.6 Start Runs
      • 22.6.1 start.runs(model)
    • 22.7 Get Model Output
      • 22.7.1 get.model.output(model)
    • 22.8 Ensemble Analysis
      • 22.8.1 run.ensemble.analysis()
    • 22.9 Sensitivity Analysis, Variance Decomposition
      • 22.9.1 run.sensitivity.analysis()
    • 22.10 Glossary
  • 23 Installation details
    • 23.1 AWS Setup
      • 23.1.1 Convert PEcAn VM
      • 23.1.2 Set up an account on AWS
      • 23.1.3 Install EC2 command line tools
      • 23.1.4 Create an AWS S3 ‘bucket’ to upload VM to
      • 23.1.5 Upload
      • 23.1.6 Configuring the VM
      • 23.1.7 Set up multiple instances (optional)
      • 23.1.8 Booting the VM
    • 23.2 Shiny Setup
      • 23.2.1 Install the Shiny R package and Shiny server
      • 23.2.2 Modify the shiny configuration file
      • 23.2.3 Set the Apache proxy
      • 23.2.4 Enable and start the shiny server, and restart apache
      • 23.2.5 Troubleshooting
      • 23.2.6 Further reading
    • 23.3 Thredds Setup
      • 23.3.1 Install the Tomcat 8 and Thredds webapp
      • 23.3.2 Update the catalog
      • 23.3.3 Troubleshooting
      • 23.3.4 Further reading
    • 23.4 OS Specific Installations
      • 23.4.1 Ubuntu
      • 23.4.2 CentOS/RedHat
      • 23.4.3 Mac OSX
      • 23.4.4 Installing BETY
      • 23.4.5 Install Models
      • 23.4.6 Installing data for PEcAn
  • 24 Docker
    • 24.1 Introduction to Docker?
      • 24.1.1 What is Docker?
      • 24.1.2 Working with Docker
      • 24.1.3 docker-compose
    • 24.2 Quick-start docker install
    • 24.3 The PEcAn docker install process in detail
      • 24.3.1 Configure docker-compose
      • 24.3.2 Initialize PEcAn (first time only)
      • 24.3.3 Troubleshooting
    • 24.4 PEcAn Docker Architecture
      • 24.4.1 Overview
      • 24.4.2 PEcAn’s docker-compose
      • 24.4.3 Top-level structure
      • 24.4.4 traefik
      • 24.4.5 portainer
      • 24.4.6 minio
      • 24.4.7 thredds
      • 24.4.8 postgres
      • 24.4.9 rabbitmq
      • 24.4.10 bety
      • 24.4.11 docs
      • 24.4.12 web
      • 24.4.13 executor
      • 24.4.14 monitor
      • 24.4.15 Model-specific containers
    • 24.5 Models using Docker
      • 24.5.1 Model information
      • 24.5.2 Model build
      • 24.5.3 Common problems
      • 24.5.4 Debugging missing libraries
    • 24.6 Building and modifying images
      • 24.6.1 Local development and testing with Docker
    • 24.7 Troubleshooting Docker
      • 24.7.1 “Package not available” while building images
    • 24.8 Migrating PEcAn from VM to Docker
      • 24.8.1 Running BETY as a docker container
    • 24.9 The PEcAn Docker API
    • 24.10 RabbitMQ
      • 24.10.1 Producer – PEcAn.remote::rabbitmq_post_message
      • 24.10.2 Consumer – receiver.py
      • 24.10.3 RabbitMQ and the PEcAn web interface
      • 24.10.4 RabbitMQ in the PEcAn XML
      • 24.10.5 RabbitMQ configuration in Dockerfiles
      • 24.10.6 Case study: PEcAn web interface
  • 25 Remote execution with PEcAn
    • 25.1 Basics of SSH
    • 25.2 SSH authentication – password vs. SSH key
    • 25.3 SSH tunneling
    • 25.4 SSH tunnels and PEcAn
    • 25.5 Basic remote execute functions
    • 25.6 Remote model execution with PEcAn
    • 25.7 XML configuration
    • 25.8 Configuration for PEcAn web interface
    • 25.9 Running PEcAn code for remotely
    • 25.10 Special case: geo.bu.edu
  • 26 Data assimilation with DART
  • IV Appendix
  • 27 Miscellaneous
    • 27.1 TODO
    • 27.2 Using the PEcAn download_file() function
  • 28 FAQ
  • 29 PEcAn Project Used in Courses
    • 29.1 University classes
      • 29.1.1 GE 375 - Environmental Modeling - Spring 2013, 2014 (Mike Dietze, Boston University)
    • 29.2 Summer Courses / Workshops
      • 29.2.1 Annual summer course in flux measurement and advanced modeling (Mike Dietze, Ankur Desai) Niwot Ridge, CO
      • 29.2.2 Assimilating Long-Term Data into Ecosystem Models: Paleo-Ecological Observatory Network (PalEON) Project
      • 29.2.3 Integrating Evidence on Forest Response to Climate Change: Physiology to Regional Abundance
      • 29.2.4 Ecological Society of America meetings
    • 29.3 Selected Publications
  • 30 Package Dependencies
    • 30.1 Executive Summary: What to usually do
    • 30.2 Big Picture: What’s possible to do
    • 30.3 Declaring Dependencies: Depends, Suggests, Imports
    • 30.4 Importing Functions: Use Roxygen
    • 30.5 Loading Code: Don’t… But Use requireNamespace When You Do
    • 30.6 Installing dependencies: Let the machines do it
  • 31 Testing with the testthat package
    • 31.1 List of Expectations
    • 31.2 Basic use of the testthat package
    • 31.3 Data for tests
    • 31.4 Settings
    • 31.5 Helper functions for unit tests
  • 32 devtools package
  • 33 singularity
  • 34 SiBCASA

The Predictive Ecosystem Analyzer

8 Developer guide

  • Update BETY
  • Update PEcAn Code
  • PEcAn and Git
  • Coding Practices