-
Notifications
You must be signed in to change notification settings - Fork 0
Use R to save table into Excel
It is a convenient to use GNU R to prepare model parameters and analyze output values. There are two different R APIs which we can use for openM++ models:
- openMpp package: simple and convenient specially for desktop users, upstream and downstream analysis;
-
oms
JSON web-service API: preferable choice to run models on computational clusters and in cloud.
There is also an excelent R package created by Matthew T. Warkentin available at: mattwarkentin.github.io/openmpp.
Below is an example how to use oms
JSON web-service to read multiple output table values from multiple model runs and save it into XLSX file:
- using
RiskPaths
demo model - reading model run names and output table names from input Excel file as on screenshots below
- for each table retriving output values for all model runs
- retrieving model runs metadata: run name, description, notes, date and time
- retrieving output tables metadata: name, description and notes
- saving each table output values as separate Excel workbook sheet
- saving all model runs metadata and tables metadata as separate sheets
Input Excel workbook:
Output Excel workbook:
#
# Read multiple tables from multiple model runs and save it as XLSX file
# Also save model runs metadata and tables metadata (name, description, notes) into .csv files
# Model run names and table names are coming from another input XLSX file
#
# If any of library below is not installed then do:
# install.packages("jsonlite")
# install.packages("httr")
# install.packages("readxl")
# install.packages("writexl")
#
library("jsonlite")
library("httr")
library("readxl")
library("writexl")
# Include openM++ helper functions from your $HOME directory
#
source("~/omsCommon.R")
#
# Model digest of RiskPaths version 3.0.0.0: "d90e1e9a49a06d972ecf1d50e684c62b"
# We MUST use model digest if there are multiple versions of the model published.
# We can use model name if only single version of the model is published.
#
md <- "d90e1e9a49a06d972ecf1d50e684c62b"
# oms web-service URL from file: ~/oms_url.txt
#
apiUrl <- getOmsApiUrl()
# model runs can be identified by digest, by run stamp or by run name
# run digest is unique and it preferable way to identify model run
# run names are user friendly may not be unique
#
# read model run names from some XLSX file,
# it must have sheet name = "RunNames" with A column "RunNames"
#
rn <- read_xlsx(
"model-runs-to-read-and-tables-to-read.xlsx",
sheet = "RunNames",
col_types = "text"
)
# read table names from some XLSX file,
# it must have sheet name = "TableNames" with A column "TableNames"
#
tn <- read_xlsx(
"model-runs-to-read-and-tables-to-read.xlsx",
sheet = "TableNames",
col_types = "text"
)
# get table information
#
rsp <- GET(paste0(
apiUrl, "model/", md, "/text"
))
if (http_type(rsp) != 'application/json') {
stop("Failed to get first model info")
}
jr <- content(rsp)
tTxt <- jr$TableTxt
tableInfo <- data.frame()
for (t in tTxt) {
for (tbl in tn$TableNames)
{
if (t$Table$Name == tbl) {
ti <- data.frame(
TableName = tbl,
TableDescription = t$TableDescr,
TableNotes = t$TableNote
)
tableInfo <- rbind(tableInfo, ti)
break
}
}
}
# get run information
#
runInfo <- data.frame()
for (run in rn$RunNames)
{
rsp <- GET(paste0(
apiUrl, "model/", md, "/run/", URLencode(run, reserved = TRUE), "/text"
))
if (http_type(rsp) != 'application/json') {
stop("Failed to get first run info of: ", run)
}
jr <- content(rsp)
ri <- data.frame(
ModelName = jr$ModelName,
ModelVersion = jr$ModelVersion,
RunName = jr$Name,
SubCount = jr$SubCount,
RunStarted = jr$CreateDateTime,
RunCompleted = jr$UpdateDateTime,
RunDescription = "",
RunNotes = ""
)
if (length(jr$Txt) > 0) {
ri$RunDescription <- jr$Txt[[1]]$Descr
ri$RunNotes <- jr$Txt[[1]]$Note
}
runInfo <- rbind(runInfo, ri)
}
# for each table do:
# combine all run results and write it into some .xlsx file
#
shts <- list(
RunInfo = runInfo,
TableInfo = tableInfo
)
for (tbl in tn$TableNames)
{
allCct <- NULL
isFirst <- TRUE
for (run in rn$RunNames)
{
cct <- read.csv(paste0(
apiUrl, "model/", md, "/run/", URLencode(run, reserved = TRUE), "/table/", tbl, "/expr/csv"
))
# build a pivot table data frame:
# use first run results to assign all dimensions and measure(s)
# from all subsequent model run bind only expr_value column
if (isFirst) {
allCct <- rbind(allCct, cct)
isFirst <- FALSE
} else {
cval <- data.frame(expr_value = cct$expr_value)
allCct <- cbind(allCct, cval)
}
# use run name for expression values column name
names(allCct)[names(allCct) == 'expr_value'] <- run
}
shts[[ tbl ]] <- allCct
}
write_xlsx(shts, paste0("output-tables-data.xlsx"))
- Windows: Quick Start for Model Users
- Windows: Quick Start for Model Developers
- Linux: Quick Start for Model Users
- Linux: Quick Start for Model Developers
- MacOS: Quick Start for Model Users
- MacOS: Quick Start for Model Developers
- Model Run: How to Run the Model
- MIT License, Copyright and Contribution
- Model Code: Programming a model
- Windows: Create and Debug Models
- Linux: Create and Debug Models
- MacOS: Create and Debug Models
- MacOS: Create and Debug Models using Xcode
- Modgen: Convert case-based model to openM++
- Modgen: Convert time-based model to openM++
- Modgen: Convert Modgen models and usage of C++ in openM++ code
- Model Localization: Translation of model messages
- How To: Set Model Parameters and Get Results
- Model Run: How model finds input parameters
- Model Output Expressions
- Model Run Options and ini-file
- OpenM++ Compiler (omc) Run Options
- OpenM++ ini-file format
- UI: How to start user interface
- UI: openM++ user interface
- UI: Create new or edit scenario
- UI: Upload input scenario or parameters
- UI: Run the Model
- UI: Use ini-files or CSV parameter files
- UI: Compare model run results
- UI: Aggregate and Compare Microdata
- UI: Filter run results by value
- UI: Disk space usage and cleanup
- UI Localization: Translation of openM++
-
Highlight: hook to self-scheduling or trigger attribute
-
Highlight: The End of Start
-
Highlight: Enumeration index validity and the
index_errors
option -
Highlight: Simplified iteration of range, classification, partition
-
Highlight: Parameter, table, and attribute groups can be populated by module declarations
- Oms: openM++ web-service
- Oms: openM++ web-service API
- Oms: How to prepare model input parameters
- Oms: Cloud and model runs queue
- Use R to save output table into CSV file
- Use R to save output table into Excel
- Run model from R: simple loop in cloud
- Run RiskPaths model from R: advanced run in cloud
- Run RiskPaths model in cloud from local PC
- Run model from R and save results in CSV file
- Run model from R: simple loop over model parameter
- Run RiskPaths model from R: advanced parameters scaling
- Run model from Python: simple loop over model parameter
- Run RiskPaths model from Python: advanced parameters scaling
- Windows: Use Docker to get latest version of OpenM++
- Linux: Use Docker to get latest version of OpenM++
- RedHat 8: Use Docker to get latest version of OpenM++
- Quick Start for OpenM++ Developers
- Setup Development Environment
- 2018, June: OpenM++ HPC cluster: Test Lab
- Development Notes: Defines, UTF-8, Databases, etc.
- 2012, December: OpenM++ Design
- 2012, December: OpenM++ Model Architecture, December 2012
- 2012, December: Roadmap, Phase 1
- 2013, May: Prototype version
- 2013, September: Alpha version
- 2014, March: Project Status, Phase 1 completed
- 2016, December: Task List
- 2017, January: Design Notes. Subsample As Parameter problem. Completed
GET Model Metadata
- GET model list
- GET model list including text (description and notes)
- GET model definition metadata
- GET model metadata including text (description and notes)
- GET model metadata including text in all languages
GET Model Extras
GET Model Run results metadata
- GET list of model runs
- GET list of model runs including text (description and notes)
- GET status of model run
- GET status of model run list
- GET status of first model run
- GET status of last model run
- GET status of last completed model run
- GET model run metadata and status
- GET model run including text (description and notes)
- GET model run including text in all languages
GET Model Workset metadata: set of input parameters
- GET list of model worksets
- GET list of model worksets including text (description and notes)
- GET workset status
- GET model default workset status
- GET workset including text (description and notes)
- GET workset including text in all languages
Read Parameters, Output Tables or Microdata values
- Read parameter values from workset
- Read parameter values from workset (enum id's)
- Read parameter values from model run
- Read parameter values from model run (enum id's)
- Read output table values from model run
- Read output table values from model run (enum id's)
- Read output table calculated values from model run
- Read output table calculated values from model run (enum id's)
- Read output table values and compare model runs
- Read output table values and compare model runs (enun id's)
- Read microdata values from model run
- Read microdata values from model run (enum id's)
- Read aggregated microdata from model run
- Read aggregated microdata from model run (enum id's)
- Read microdata run comparison
- Read microdata run comparison (enum id's)
GET Parameters, Output Tables or Microdata values
- GET parameter values from workset
- GET parameter values from model run
- GET output table expression(s) from model run
- GET output table calculated expression(s) from model run
- GET output table values and compare model runs
- GET output table accumulator(s) from model run
- GET output table all accumulators from model run
- GET microdata values from model run
- GET aggregated microdata from model run
- GET microdata run comparison
GET Parameters, Output Tables or Microdata as CSV
- GET csv parameter values from workset
- GET csv parameter values from workset (enum id's)
- GET csv parameter values from model run
- GET csv parameter values from model run (enum id's)
- GET csv output table expressions from model run
- GET csv output table expressions from model run (enum id's)
- GET csv output table accumulators from model run
- GET csv output table accumulators from model run (enum id's)
- GET csv output table all accumulators from model run
- GET csv output table all accumulators from model run (enum id's)
- GET csv calculated table expressions from model run
- GET csv calculated table expressions from model run (enum id's)
- GET csv model runs comparison table expressions
- GET csv model runs comparison table expressions (enum id's)
- GET csv microdata values from model run
- GET csv microdata values from model run (enum id's)
- GET csv aggregated microdata from model run
- GET csv aggregated microdata from model run (enum id's)
- GET csv microdata run comparison
- GET csv microdata run comparison (enum id's)
GET Modeling Task metadata and task run history
- GET list of modeling tasks
- GET list of modeling tasks including text (description and notes)
- GET modeling task input worksets
- GET modeling task run history
- GET status of modeling task run
- GET status of modeling task run list
- GET status of modeling task first run
- GET status of modeling task last run
- GET status of modeling task last completed run
- GET modeling task including text (description and notes)
- GET modeling task text in all languages
Update Model Profile: set of key-value options
- PATCH create or replace profile
- DELETE profile
- POST create or replace profile option
- DELETE profile option
Update Model Workset: set of input parameters
- POST update workset read-only status
- PUT create new workset
- PUT create or replace workset
- PATCH create or merge workset
- DELETE workset
- POST delete multiple worksets
- DELETE parameter from workset
- PATCH update workset parameter values
- PATCH update workset parameter values (enum id's)
- PATCH update workset parameter(s) value notes
- PUT copy parameter from model run into workset
- PATCH merge parameter from model run into workset
- PUT copy parameter from workset to another
- PATCH merge parameter from workset to another
Update Model Runs
- PATCH update model run text (description and notes)
- DELETE model run
- POST delete model runs
- PATCH update run parameter(s) value notes
Update Modeling Tasks
Run Models: run models and monitor progress
Download model, model run results or input parameters
- GET download log file
- GET model download log files
- GET all download log files
- GET download files tree
- POST initiate entire model download
- POST initiate model run download
- POST initiate model workset download
- DELETE download files
- DELETE all download files
Upload model runs or worksets (input scenarios)
- GET upload log file
- GET all upload log files for the model
- GET all upload log files
- GET upload files tree
- POST initiate model run upload
- POST initiate workset upload
- DELETE upload files
- DELETE all upload files
Download and upload user files
- GET user files tree
- POST upload to user files
- PUT create user files folder
- DELETE file or folder from user files
- DELETE all user files
User: manage user settings
Model run jobs and service state
- GET service configuration
- GET job service state
- GET disk usage state
- POST refresh disk space usage info
- GET state of active model run job
- GET state of model run job from queue
- GET state of model run job from history
- PUT model run job into other queue position
- DELETE state of model run job from history
Administrative: manage web-service state
- POST a request to refresh models catalog
- POST a request to close models catalog
- POST a request to close model database
- POST a request to delete the model
- POST a request to open database file
- POST a request to cleanup database file
- GET the list of database cleanup log(s)
- GET database cleanup log file(s)
- POST a request to pause model run queue
- POST a request to pause all model runs queue
- PUT a request to shutdown web-service