Based on a configuration file or list this functions creates a connectors() object with
a connector for each of the specified datasources.
The configuration file can be in any format that can be read through read_file(), and
contains a list. If a yaml file is provided, expressions are evaluated when parsing it
using yaml::read_yaml() with eval.expr = TRUE.
See also vignette("connector") on how to use configuration files in your project,
details below for the required structure of the configuration.
Usage
connect(
config = "_connector.yml",
metadata = NULL,
datasource = NULL,
set_env = TRUE,
logging = FALSE
)Arguments
- config
character path to a connector config file or a list of specifications
- metadata
list Replace, add or create elements to the metadata field found in config
- datasource
character Name(s) of the datasource(s) to connect to. If
NULL(the default) all datasources are connected.- set_env
logical Should environment variables from the yaml file be set? Default is TRUE.
- logging
logical Add logs to the console as well as to the whirl log html files. See details with this vignette.
Details
The input list can be specified in two ways:
A named list containing the specifications of a single connectors object.
An unnamed list, where each element is of the same structure as in 1., which returns a nested connectors object. See example below.
Each specification of a single connectors have to have the following structure:
Only name, metadata, env and datasources are allowed.
All elements must be named.
name is only required when using nested connectors.
datasources is mandatory.
metadata and env must each be a list of named character vectors of length 1 if specified.
datasources must each be a list of unnamed lists.
Each datasource must have the named character element name and the named list element backend
For each connection backend.type must be provided
Examples
config <- system.file("config", "default_config.yml", package = "connector")
config
#> [1] "/home/runner/work/_temp/Library/connector/config/default_config.yml"
# Show the raw configuration file
readLines(config) |>
cat(sep = "\n")
#> # A example of the configuration file for FS and Database
#> metadata:
#> trial: "demo_trial"
#> root_path: !expr system.file(package = "connector")
#> extra_class: "test2"
#>
#> datasources:
#> - name: "adam"
#> backend:
#> type: "connector_fs"
#> path: "{metadata.root_path}/{metadata.trial}/adam"
#> extra_class: "{metadata.extra_class}"
#> - name: "sdtm"
#> backend:
#> type: "connector_dbi"
#> drv: "RSQLite::SQLite()"
#> dbname: ":memory:"
# Connect to the datasources specified in it
cnts <- connect(config)
#> ────────────────────────────────────────────────────────────────────────────────
#> Connection to:
#> → adam
#> • connector_fs
#> • /home/runner/work/_temp/Library/connector/demo_trial/adam and test2
#> ────────────────────────────────────────────────────────────────────────────────
#> Connection to:
#> → sdtm
#> • connector_dbi
#> • RSQLite::SQLite() and :memory:
cnts
#> <connectors>
#> $adam <test2>
#> $sdtm <connector_dbi>
# Content of each connector
cnts$adam
#> <test2/connector_fs>
#> Inherits from: <connector>
#> Registered methods:
#> • `create_directory_cnt.connector_fs()`
#> • `download_cnt.connector_fs()`
#> • `list_content_cnt.connector_fs()`
#> • `read_cnt.connector_fs()`
#> • `remove_cnt.connector_fs()`
#> • `remove_directory_cnt.connector_fs()`
#> • `upload_cnt.connector_fs()`
#> • `write_cnt.connector_fs()`
#> Specifications:
#> • path: /home/runner/work/_temp/Library/connector/demo_trial/adam
cnts$sdtm
#> <connector_dbi>
#> Inherits from: <connector>
#> Registered methods:
#> • `disconnect_cnt.connector_dbi()`
#> • `list_content_cnt.connector_dbi()`
#> • `read_cnt.connector_dbi()`
#> • `remove_cnt.connector_dbi()`
#> • `tbl_cnt.connector_dbi()`
#> • `write_cnt.connector_dbi()`
#> Specifications:
#> • conn: <SQLiteConnection>
# Overwrite metadata informations
connect(config, metadata = list(extra_class = "my_class"))
#> → Replace some metadata informations...
#> ────────────────────────────────────────────────────────────────────────────────
#> Connection to:
#> → adam
#> • connector_fs
#> • /home/runner/work/_temp/Library/connector/demo_trial/adam and my_class
#> ────────────────────────────────────────────────────────────────────────────────
#> Connection to:
#> → sdtm
#> • connector_dbi
#> • RSQLite::SQLite() and :memory:
#> <connectors>
#> $adam <my_class>
#> $sdtm <connector_dbi>
# Connect only to the adam datasource
connect(config, datasource = "adam")
#> ────────────────────────────────────────────────────────────────────────────────
#> Connection to:
#> → adam
#> • connector_fs
#> • /home/runner/work/_temp/Library/connector/demo_trial/adam and test2
#> <connectors>
#> $adam <test2>
# Connect to several projects in a nested structure
config_nested <- system.file("config", "_connector_nested.yml", package = "connector")
readLines(config_nested) |>
cat(sep = "\n")
#> # A example of the configuration file nested connectors
#> - name: "study1"
#> metadata:
#> trial: "demo_trial"
#> root_path: !expr system.file(package = "connector")
#> datasources:
#> - name: "adam_fs"
#> backend:
#> type: "connector_fs"
#> path: "{metadata.root_path}/{metadata.trial}/adam"
#>
#> - name: "study2"
#> metadata:
#> trial: "demo_trial"
#> root_path: !expr system.file(package = "connector")
#> datasources:
#> - name: "adam_fs"
#> backend:
#> type: "connector_fs"
#> path: "{metadata.root_path}/{metadata.trial}/adam"
cnts_nested <- connect(config_nested)
#> ────────────────────────────────────────────────────────────────────────────────
#> Connection to:
#> → adam_fs
#> • connector_fs
#> • /home/runner/work/_temp/Library/connector/demo_trial/adam
#> ────────────────────────────────────────────────────────────────────────────────
#> Connection to:
#> → adam_fs
#> • connector_fs
#> • /home/runner/work/_temp/Library/connector/demo_trial/adam
cnts_nested
#> <nested_connectors>
#> $study1 <connectors>
#> $study2 <connectors>
cnts_nested$study1
#> <connectors>
#> $adam_fs <connector_fs>