Based on a configuration file or list this functions creates a connectors() object with
a Connector for each of the specified datasources.
The configuration file can be in any format that can be read through read_file(), and
contains a list. If a yaml file is provided, expressions are evaluated when parsing it
using yaml::read_yaml() with eval.expr = TRUE.
See also vignette("connector") on how to use configuration files in your project,
details below for the required structure of the configuration.
Usage
connect(
config = "_connector.yml",
metadata = NULL,
datasource = NULL,
set_env = TRUE,
logging = zephyr::get_option("logging", "connector")
)Arguments
- config
character path to a connector config file or a list of specifications
- metadata
list Replace, add or create elements to the metadata field found in config
- datasource
character Name(s) of the datasource(s) to connect to. If
NULL(the default) all datasources are connected.- set_env
logical Should environment variables from the yaml file be set? Default is TRUE.
- logging
Add logs to the console as well as to the whirl log html files. Default:
FALSE.
Details
The input list can be specified in two ways:
A named list containing the specifications of a single connectors object.
An unnamed list, where each element is of the same structure as in 1., which returns a nested connectors object. See example below.
Each specification of a single connectors have to have the following structure:
Only name, metadata, env and datasources are allowed.
All elements must be named.
name is only required when using nested connectors.
datasources is mandatory.
metadata and env must each be a list of named character vectors of length 1 if specified.
datasources must each be a list of unnamed lists.
Each datasource must have the named character element name and the named list element backend
For each connection backend.type must be provided
Examples
config <- system.file("config", "_connector.yml", package = "connector")
config
#> [1] "/home/runner/work/_temp/Library/connector/config/_connector.yml"
# Show the raw configuration file
readLines(config) |>
cat(sep = "\n")
#> # A example of the configuration file for FS and Database
#> metadata:
#> trial: "demo_trial"
#> root_path: !expr tempdir()
#> extra_class: "test2"
#>
#> datasources:
#> - name: "adam"
#> backend:
#> type: "connector_fs"
#> path: "{metadata.root_path}/{metadata.trial}/adam"
#> extra_class: "{metadata.extra_class}"
#> - name: "sdtm"
#> backend:
#> type: "connector_dbi"
#> drv: "RSQLite::SQLite()"
#> dbname: ":memory:"
# Connect to the datasources specified in it
cnts <- connect(config)
#> ────────────────────────────────────────────────────────────────────────────────
#> Connection to:
#> → adam
#> • connector_fs
#> • /tmp/RtmpI13gam/demo_trial/adam and test2
#> ────────────────────────────────────────────────────────────────────────────────
#> Connection to:
#> → sdtm
#> • connector_dbi
#> • RSQLite::SQLite() and :memory:
cnts
#> <connectors>
#> $adam <test2>
#> $sdtm <ConnectorDBI>
# Content of each connector
cnts$adam
#> <test2/ConnectorFS>
#> Inherits from: <Connector>
#> Registered methods:
#> • `create_directory_cnt.ConnectorFS()`
#> • `download_cnt.ConnectorFS()`
#> • `download_directory_cnt.ConnectorFS()`
#> • `list_content_cnt.ConnectorFS()`
#> • `log_read_connector.ConnectorFS()`
#> • `log_remove_connector.ConnectorFS()`
#> • `log_write_connector.ConnectorFS()`
#> • `read_cnt.ConnectorFS()`
#> • `remove_cnt.ConnectorFS()`
#> • `remove_directory_cnt.ConnectorFS()`
#> • `tbl_cnt.ConnectorFS()`
#> • `upload_cnt.ConnectorFS()`
#> • `upload_directory_cnt.ConnectorFS()`
#> • `write_cnt.ConnectorFS()`
#> Specifications:
#> • path: /tmp/RtmpI13gam/demo_trial/adam
cnts$sdtm
#> <ConnectorDBI>
#> Inherits from: <Connector>
#> Registered methods:
#> • `disconnect_cnt.ConnectorDBI()`
#> • `list_content_cnt.ConnectorDBI()`
#> • `log_read_connector.ConnectorDBI()`
#> • `log_remove_connector.ConnectorDBI()`
#> • `log_write_connector.ConnectorDBI()`
#> • `read_cnt.ConnectorDBI()`
#> • `remove_cnt.ConnectorDBI()`
#> • `tbl_cnt.ConnectorDBI()`
#> • `write_cnt.ConnectorDBI()`
#> Specifications:
#> • conn: <SQLiteConnection>
# Overwrite metadata informations
connect(config, metadata = list(extra_class = "my_class"))
#> ℹ Replace some metadata informations...
#> ────────────────────────────────────────────────────────────────────────────────
#> Connection to:
#> → adam
#> • connector_fs
#> • /tmp/RtmpI13gam/demo_trial/adam and my_class
#> ────────────────────────────────────────────────────────────────────────────────
#> Connection to:
#> → sdtm
#> • connector_dbi
#> • RSQLite::SQLite() and :memory:
#> <connectors>
#> $adam <my_class>
#> $sdtm <ConnectorDBI>
# Connect only to the adam datasource
connect(config, datasource = "adam")
#> ────────────────────────────────────────────────────────────────────────────────
#> Connection to:
#> → adam
#> • connector_fs
#> • /tmp/RtmpI13gam/demo_trial/adam and test2
#> <connectors>
#> $adam <test2>
# Connect to several projects in a nested structure
config_nested <- system.file("config", "_nested_connector.yml", package = "connector")
readLines(config_nested) |>
cat(sep = "\n")
#> Warning: incomplete final line found on '/home/runner/work/_temp/Library/connector/config/_nested_connector.yml'
#> # A example of the configuration file nested connectors
#> - name: "study1"
#> metadata:
#> trial: "demo_trial"
#> root_path: !expr tempdir()
#> datasources:
#> - name: "adam_fs"
#> backend:
#> type: "connector_fs"
#> path: "{metadata.root_path}/{metadata.trial}/adam"
#>
#> - name: "study2"
#> metadata:
#> trial: "demo_trial"
#> root_path: !expr tempdir()
#> datasources:
#> - name: "adam_fs"
#> backend:
#> type: "connector_fs"
#> path: "{metadata.root_path}/{metadata.trial}/adam"
cnts_nested <- connect(config_nested)
#> Warning: incomplete final line found on '/home/runner/work/_temp/Library/connector/config/_nested_connector.yml'
#> ────────────────────────────────────────────────────────────────────────────────
#> Connection to:
#> → adam_fs
#> • connector_fs
#> • /tmp/RtmpI13gam/demo_trial/adam
#> ────────────────────────────────────────────────────────────────────────────────
#> Connection to:
#> → adam_fs
#> • connector_fs
#> • /tmp/RtmpI13gam/demo_trial/adam
cnts_nested
#> <nested_connectors>
#> $study1 <connectors>
#> $study2 <connectors>
cnts_nested$study1
#> <connectors>
#> $adam_fs <ConnectorFS>