ADR: Specifying pooled specifications

Status

Package	{mighty.metadata}
Status	Draft
Version	0.1.0
Description	How to specify and work with pooled specifications

Success criteria

Pooling of ADaM specifications is supported across the three use cases described in Alternatives Considered.
{mighty.metadata} supports a new subset field on row operations.
{mighty.metadata} provides a new resolve_subsets() function that resolves the row-operation subset logic.
Row-operation metadata is included in the define.xml generated by {mighty.toolbox}.
Standard components can be reused unmodified in pooled specifications.
Study-level metadata can be passed into component.with so components can access study context without inline YAML expressions.

Context

Clinical project teams often want to work on pooled specifications of ADaM domains across studies for the same development project.

The advantage of pooling specifications is that most derivations are the same across studies, and then pooling both increases efficiency and coherence across studies, by only having the CDS specify them once.

As a consequence, pooling only makes sense when the domains are mostly similar across the studies. If this is not the case, they should be specified individually.

When a derivation is the same across studies there are no limitations in the current setup, but if the derivation is conditional certain limitations exist.

Under Alternatives Considered we go through the different cases with conditional derivations, and which we are supporting already and which we need to develop support for.

Three use cases are considered:

Use case 1 — Only do something for a given study. Already supported by include: and resolve_includes().
Use case 2 — Same endpoint, different derivations, one script per study.
Use case 3 — Same endpoint, different derivations, one pooled script covering all studies.

Design Principles

To fit in the overall mightyverse the following principles have to be followed:

One YAML per ADaM domain.
No additional tooling needed; everything contained in a combination of {mighty}, {mighty.metadata} and {mighty.component}.
Individual component definitions and structure are not affected; they should always be as simple and modular as possible and focus only on the derivation being carried out.
Keep implementation as generic as possible; ensure no lock-in to current business logic.

An additional thought is that all testing and investigation leading up to this should not spread across the clinical projects. We need to ensure strict mighty compliance from the start to ensure we, in the long run, achieve our long-term quality and efficiency goals.

Decisions

To support all three use cases, {mighty.metadata} will be extended with:

Improved support for standard components in row operations with a new subset field.
A new resolve_subsets() function that resolves this logic.
A mechanism for passing study-level metadata into component.with, so components can access study context without inline YAML expressions.

In the wider mighty system we will ensure that:

Metadata from row operations is properly included in the define.xml generated by {mighty.toolbox}.

For each use case the recommended approach is:

Use case 1 — use the existing include: property (no changes needed).
Use case 2 — use row operations (see Alternatives Considered). For very limited study-specific changes, scripting via update_columns() is an acceptable alternative.
Use case 3 — use row operations with the new subset field.

Consequences

Changes to current content

{mighty.metadata} gains a subset field on row operations (schema update to inst/schema/adam.json) and a resolve_subsets() function.
{mighty.metadata} gains a way to inject study-level metadata into component.with values.
{mighty.toolbox} must integrate row-operation metadata into define.xml output.
{mighty.component} rendering must honor the subset field when producing code.

Alternatives Considered

Use case 1: Only do something for a given study

The simplest deviation across studies is when one study requires a derivation, but another study does not.

{mighty.metadata} supports inclusion/exclusion matching via the include: property on all operation levels (columns, parameters, rows):

columns:
  - id: MYVAR
    label: My label
    method: My method
    include: "{study_id == 'my_study_1'}"
    component:
      id: my_component

This is evaluated with resolve_includes():

When study_id = "my_study_1" the column is included, and MYVAR is added to the domain using the rendered code from my_component.
Otherwise the column is not included.

This use case is already supported. No changes needed.

Use case 2: Split into different scripts

Same endpoint, different derivation methods per study — pseudo specification:

columns:
  - id: MYVAR
    label: My label
    method: My method
    include: "{study_id == 'my_study_1'}"
    component:
      id: my_component

  - id: MYVAR
    label: My label
    method: My method
    include: "{study_id == 'my_study_2'}"
    component:
      id: my_other_component

Problem: Duplicate id entries are not allowed at domain level, and allowing them risks diverging metadata. The CDISC principle that the same name represents the same information makes it undesirable to loosen this requirement.

Alternative 1: Use scripting

Keep pooled YAML free of study-specific components and inject them in a study-specific script:

pooled.yml:

id: ADXX
columns:
  - id: MYVAR
    label: My label
    method: My general method

create_study_scripts.R:

study <- mighty.metadata::mighty_study()

study$ADXX <- study$ADXX |>
    mighty.metadata::update_columns(
      id = "MYVAR",
      component = list(id = "my_component")
    )

mighty.metadata::write_config(x = study, path = "/path/to/study/artifacts")

mighty::generate_adam_code(...)

Pros	Cons
No updates to mighty packages needed	Requires more coding by CDS
Full flexibility for the CDS	The pooled YAML does not contain all specifications

Alternative 2: Allow multiple component entries

Allow a list of components under columns.component:

columns:
  - id: MYVAR
    label: My label
    method: My method
    component:
      - id: my_component
        include: "{study_id == 'my_study_1'}"
      - id: my_other_component
        include: "{study_id == 'my_study_2'}"

Requires updating the ADaM schema and resolve_includes() to evaluate on the component level and ensure exactly one component remains after evaluation.

Pros	Cons
Keep specifications in the pooled YAML	Requires updates in mighty.metadata
Full flexibility for the CDS	Allowing multiple components and pushing validation of uniqueness is making validation harder

Alternative 3: Use row operations (recommended)

Initialize the column with an empty component, then conditionally overwrite it via row operations:

columns:
  - id: MYVAR
    label: My label
    method: My general method
    component:
      id: empty_numeric

rows:
  - id: MYVAR_STUDY1
    include: "{study_id == 'my_study_1'}"
    method: "Study 1: Doing this special derivation...."
    component:
      id: my_component

  - id: MYVAR_STUDY2
    include: "{study_id == 'my_study_2'}"
    method: "Study 2: Doing this other special derivation...."
    component:
      id: my_other_component

include: is already supported for rows, so this works today.

Pros	Cons
Keep specifications in the pooled YAML	Harder to get overview of a column when it is edited by row operations directly in the YAML¹
Full flexibility for the CDS	The generated script will first initialize the column, which might look a little weird
No updates to mighty packages needed
Validation stays consistent

Recommendation: Row operations offer the best overview when many study-specific changes are needed. For very limited changes scripting is acceptable.

Use case 3: Same script, different derivations

When creating a single pooled script for multiple studies (e.g. ISS), both derivations must be present in the script and restricted to their respective study rows:

# Using dplyr::case_when
ADXX <- ADXX |>
  dplyr::mutate(
    MYVAR = dplyr::case_when(
      STUDYID == "my_study_1" ~ my_component(...),
      STUDYID == "my_study_2" ~ my_other_component(...),
    )
  )

# Using admiral::restrict_derivation()
ADXX <- admiral::restrict_derivation(
  dataset = ADXX,
  derivation = my_component(...),
  filter = STUDYID == "my_study_1"
)

Problem: The same duplicate-id restriction from use case 2 applies, plus there is no logic in {mighty} or {mighty.component} to combine several components into one code block, and include cannot be evaluated on the dataset level. Scripting (use case 2 alternative 1) is not viable here.

Alternative 1: Handle in custom project components

Delegate the conditional logic to a custom component:

columns:
  - id: MYVAR
    label: My label
    method: My general method
    component:
      id: my/project/components/myvar.mustache

The custom component uses mustache conditional sections:

#' @title Mustache based conditional logic
#' @param study1 Do study 1 derivation?
#' @param study2 Do study 2 derivation?
#' @outputs MYVAR
#' @code
{{#study1}}
{{ domain }} <- admiral::restrict_derivation(
  dataset = {{ domain }},
  derivation = ...,
  filter = STUDYID == "my_study_1"
)
{{/study1}}

{{#study2}}
{{ domain }} <- admiral::restrict_derivation(
  dataset = {{ domain }},
  derivation = ...,
  filter = STUDYID == "my_study_2"
)
{{/study2}}

And in the spec:

columns:
  - id: MYVAR
    label: My label
    method: My general method
    component:
      id: my/project/components/myvar.mustache
      with:
        study1: true
        study2: false

Pros	Cons
Full flexibility for the CDS	Not supported with standard components
Validation stays consistent	Requires updating {mighty.metadata} to support study-level metadata in `component.with`
Mustache version produces nice scripts

Alternative 2: Combine multiple component entries

Extend the multiple-components approach with a subset field so the component doesn’t need to know about row filtering:

columns:
  - id: MYVAR
    label: My label
    method: My method
    component:
      - id: my_component
        include: "{study_id == 'my_study_1' || isTRUE(pooled)}"
        subset: "STUDYID == 'my_study_1'"
      - id: my_other_component
        include: "{study_id == 'my_study_2' || isTRUE(pooled)}"
        subset: "STUDYID == 'my_study_2'"

Each component is rendered as:

{{domain}}[with({{domain}}, {{subset}}), ] <-
    {{domain}}[with({{domain}}, {{subset}}), ] |>
      code_from_the_component()

Pros	Cons
Full flexibility for the CDS	Validation is less straightforward
Supports the use of standard components	Essentially doing row operations in a more complicated way
Nicely rendered scripts	Requires updating {mighty.metadata} and {mighty.component}

Alternative 3: Use row operations with `subset` (recommended)

Combine row operations with the new subset field so standard components can be reused:

columns:
  - id: MYVAR
    label: My label
    method: My general method
    component:
      id: empty_numeric

rows:
  - id: MYVAR_STUDY1
    include: "{study_id == 'my_study_1'}"
    subset: "STUDYID == 'my_study_1'"
    method: "Study 1: Doing this special derivation...."
    component:
      id: my_component

  - id: MYVAR_STUDY2
    include: "{study_id == 'my_study_2'}"
    subset: "STUDYID == 'my_study_2'"
    method: "Study 2: Doing this other special derivation...."
    component:
      id: my_other_component

Pros	Cons
Full flexibility for the CDS	Requires updating {mighty.metadata}
Validation is consistent	Lack of overview in the YAML
Supports the use of standard components
Nicely rendered scripts
Subset-specific method field ensures consistent documentation

Recommendation: Develop the row-operation approach with subset and ensure study-level metadata can easily be passed to other areas of the specification.

Implementation Details

`subset` field on row operations

A new optional subset field is added to row operations. Its value is an R expression evaluated in the context of the domain dataset.

The ADaM schema (inst/schema/adam.json) is extended by adding a subset property to the row definition:

"row": {
  "properties": {
    "...": "existing properties (id, include, method, component, depends)",
    "subset": {
      "description": "R expression restricting the row action to a subset of rows",
      "$ref": "#/definitions/mighty/filter"
    }
  }
}

#/definitions/mighty/filter is the existing filter-expression type, reused so validation is consistent with other filter fields.

When present, the associated component is rendered as:

{{domain}}[with({{domain}}, {{subset}}), ] <-
    {{domain}}[with({{domain}}, {{subset}}), ] |>
      code_from_the_component()

This is achieved by rendering any component with domain = {{domain}}[with({{domain}}, {{subset}}), ] in the presence of a subset entry, using the existing {mighty.component} rendering machinery.

`resolve_subsets()` and rendering

The subset field is resolved by rewriting the row’s component.with.domain before code generation, so it composes with the existing {mighty.component} rendering machinery — no changes to component internals are needed.

A minimal reference implementation:

resolve_subsets <- function(x) {
  for (i in seq_along(x)) {
    for (j in seq_along(x[[i]])) {
      x[[i]][[j]] <- resolve_subsets_entry(x[[i]][[j]])
    }
  }
  x
}

resolve_subsets_entry <- function(x) {
  for (r in seq_along(x)) {
    if (all(c("component", "subset") %in% names(x[[r]]))) {
      x[[r]][["component"]][["with"]][["domain"]] <- glue::glue_data(
        list(
          domain = x[[r]][["component"]][["with"]][["domain"]],
          subset = x[[r]][["subset"]]
        ),
        "{domain}[with({domain}, {subset}), ]"
      ) |>
        as.character()
      x[[r]][["subset"]] <- NULL
    }
  }
  x
}

Composed with resolve_includes() the full pipeline for a pooled build is:

study |>
  resolve_includes(info = list(study_id = "my_study_1", pooled = TRUE)) |>
  resolve_subsets() |>
  generate_adam_code(...)

For a single-study build the subset fields can be dropped rather than resolved, since no row filtering is needed:

study |>
  resolve_includes(info = list(study_id = "my_study_1", pooled = FALSE))
# then strip $subset from surviving row entries before generate_adam_code()

`define.xml` integration

{mighty.toolbox} will be updated so row-operation metadata is included in the generated define.xml. This restores the column-level overview that is otherwise weakened by moving conditional logic into row operations.

Testing Strategy

Unit tests in {mighty.metadata} for the new subset field and resolve_subsets().
Integration tests generating ADaM code from a pooled specification exercising all three use cases.
Integration tests in {mighty.toolbox} verifying that row-operation metadata appears correctly in define.xml.

Risks

Row-operation-driven column mutation reduces overview when reading the YAML directly. Mitigated by define.xml integration and tooling.
Validation of subset expressions is harder than validation of include: expressions since they operate on the domain dataset rather than study-level metadata.
Tighter coupling between {mighty.metadata} and {mighty.component} rendering.

Compliance Considerations

All development on GitHub using Pull Requests for merges to main branch, and standard ATMOS branch protection rules.
R CMD Check is required to pass on all relevant platforms before a PR is approved.

Aksel Thomsen

2026-07-29

Status

Success criteria

Context

Design Principles

Decisions

Consequences

Changes to current content

Alternatives Considered

Use case 1: Only do something for a given study

Use case 2: Split into different scripts

Alternative 1: Use scripting

Alternative 2: Allow multiple component entries

Alternative 3: Use row operations (recommended)

Use case 3: Same script, different derivations

Alternative 1: Handle in custom project components

Alternative 2: Combine multiple component entries

Alternative 3: Use row operations with `subset` (recommended)

Implementation Details

`subset` field on row operations

`resolve_subsets()` and rendering

`define.xml` integration

Testing Strategy

Risks

Compliance Considerations

ADR: Specifying pooled specifications

Aksel Thomsen

2026-07-29

Status

Success criteria

Context

Design Principles

Decisions

Consequences

Changes to current content

Alternatives Considered

Use case 1: Only do something for a given study

Use case 2: Split into different scripts

Alternative 1: Use scripting

Alternative 2: Allow multiple component entries

Alternative 3: Use row operations (recommended)

Use case 3: Same script, different derivations

Alternative 1: Handle in custom project components

Alternative 2: Combine multiple component entries

Alternative 3: Use row operations with subset (recommended)

Implementation Details

subset field on row operations

resolve_subsets() and rendering

define.xml integration

Testing Strategy

Risks

Compliance Considerations

Alternative 3: Use row operations with `subset` (recommended)

`subset` field on row operations

`resolve_subsets()` and rendering

`define.xml` integration