Skip to content

cleanup changelog for v1.0-tvst; add release note #40

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
stephanieshong merged 1 commit into from
Oct 14, 2026
Merged

cleanup changelog for v1.0-tvst; add release note #40

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 27 additions & 57 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,77 +4,47 @@ All notable changes to this project are documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

## Where to find the mapping files

- **Working version (use for ETL):** `redCapDataDictionaryWithExtension/value-set-mapping-common/_ai_readi_redcap_value_set_mapping_common.csv`. This file is actively maintained and may change after each release.
- **Frozen release version:** `manuscript-tvst2026/ai_readi_redcap_value_set_mapping_common.csv` (with frozen SSSOM and LinkML copies), matching release `v1.0-tvst` and the TVST 2026 manuscript.

## [Unreleased]
- prior versions are used for development and testing

## [v1.0-tvst] - 2026-10-14

First archived release, accompanying the manuscript *"A Generalizable Framework for Mapping Survey, Vision Care, and Cognitive Assessment Data to the OMOP Common Data Model in AI-READI"* (Translational Vision Science & Technology, 2026).
First archived release, accompanying the manuscript *"A Generalizable Framework for Mapping Survey, Vision Care, and Cognitive Assessment Data to the OMOP Common Data Model in AI-READI"* (Translational Vision Science & Technology, 2026). Archived on Zenodo: https://doi.org/10.5281/zenodo.23375723

### Added

- `manuscript-tvst2026/`: frozen mapping file, DuckDB and Databricks count scripts, OMOP vocabulary validation script, Figures 1–3 and Supplementary Tables S1–S2.
- AI-READI custom concepts 2005200630 (PhenX DMS total score) and 2005200631 (dvenvlocn).
- `manuscript-tvst2026/`: frozen mapping file, frozen SSSOM and LinkML copies, DuckDB and Databricks count scripts (`count_concept_map*.sql`, `run_counts.py`), OMOP vocabulary validation script, Figures 1–3 and Supplementary Tables S1–S2.
- AI-READI custom concepts:
- 2005200631 "Location of environmental sensor within home" for `dvenvlocn`, with qualifier 45762337 (Environmental sensor). It was previously mapped to SNOMED 284349006 (4109238), which does not describe sensor location.
- 2005200630 "Discrimination in Medical Settings (DMS) total score [PhenX]" for `pxdhc_sum`, with qualifier 2005200575 (PhenX SDOH Survey). It was previously unmapped.
- SSSOM mapping set and LinkML schema in `redCapDataDictionaryWithExtension/`.
- `deprecated/` folder with a README describing superseded files.

### Changed

- Standard concepts replaced after validation against the OMOP Standardized Vocabularies from non-standard (e.g., heart rate 3027018, LOINC 63954-2, LOINC 52810-9, SNOMED 1148423001). line 864,867 - non-standard vs. standard - use standard concept
Heart rate (pulse_vsorres)
4239408 (SNOMED 364075005) -> 3027018 (LOINC 8867-4, Heart rate) Standard Measurement-domain concept for a recorded vital sign
- Insulin mapped to LOINC 3695-4 (Mass/volume) with ng/mL units; 24 missing lab unit concepts added.
- Non-standard targets replaced with standard concepts after validation against the OMOP Standardized Vocabularies (v5.0, 27-Aug-2025):
- Heart rate (`pulse_vsorres`): 4239408 (SNOMED 364075005) → 3027018 (LOINC 8867-4, Heart rate).
- MoCA total score: 37174522 (SNOMED 1255891005) → 606671 (SNOMED 1148423001, Montreal Cognitive Assessment version 8.1).
- Closer standard equivalents:
- Substance use (`subrdosfr`): 1010004 (LOINC 68518-0) → 40766674 (LOINC 63954-2), closeMatch, confidence 0.75.
- Current medications, frequency: 4126015 (SNOMED 260864003) → 3042548 (LOINC 52810-9, Current Medication, Frequency).
- Disposition reason: 4260904 (SNOMED 410666004) → 44810920 (SNOMED 871401000000109), relatedMatch, confidence 0.5; no exact standard concept, flagged for review.
- Insulin mapped to LOINC 3695-4 (Mass/volume) with ng/mL units.
- Count scripts renamed `count_mapping_concepts*.sql` → `count_concept_map*.sql`; `run_counts.py` now prints UTF-8 output on Windows.
- Supplementary Table S1 now links the MoCA to https://mocacognition.com/paper; the MoCA 8.1 test form PDF was removed (copyright).
- `_v4` and `_v5` copies of the common mapping file moved to `deprecated/`; map2omop and open-source public versions regenerated from the common file.
- Top-level README, `CITATION.cff` and `codemeta.json` rewritten for this repository (previously template content).
- Superseded mapping files and folders moved to `deprecated/`.
- New custom concept created to support the following concept
Line 436, dvenvlocn (“Location of the environmental sensor in the home”): **dvenvlocn** → 2005200631 (436) and PhenX DMS → 2005200630 (1408)
It was mapped to SNOMED 4109238 (284349006), which doesn’t describe sensor location.
No suitable standard concept exists, so it now uses the new custom concept 2005200631, “Location of environmental sensor within home”, with qualifier 45762337 (Environmental sensor).
- Line 1408, pxdhc_sum (PhenX Discrimination in Medical Settings summary score):
It was previously unmapped.
It now uses the new custom concept 2005200630, “Discrimination in Medical Settings (DMS) total score [PhenX]”, with qualifier 2005200575 (PhenX SDOH Survey).
- concept map changed for MoCA total score/ 37174522 (SNOMED 1255891005) -> 606671 (SNOMED 1148423001), class Staging / Scale


### Fixed

- Swapped REDCap survey start and completion timestamp concepts.
- Excel-corrupted values: concept codes converted to scientific notation and answer ranges converted to dates.
- the swapped timestamp concepts (1098/1099) Survey timestamp concepts swapped back (lines 1098–1099)
In the PhenX racial and ethnic discrimination survey, the start and completion timestamps were mapped the wrong way round:
pxrdcmpts (completion) pointed to 2005200558 Survey Started Timestamp;
pxrdstartts (start) pointed to 2005200559 Survey Completed Timestamp.
- heart rate (864/867), MoCA total (647), line 1893, line 334, line 246
- 25 lab unit rows that had no UCUM concept
- the Excel-corrupted codes found in the mapping csv, fixed
- line 647 - failed standard validation test (non-standard vs standard) - use the standard concept
- Line 1893 - found a better term
Substance use item (subrdosfr)
1010004 (LOINC 68518-0) -> 40766674 (LOINC 63954-2)closeMatch, confidence 0.75 Closer standard equivalent
- Line 334 - found a better term
Current medications, frequency/ 4126015 (SNOMED 260864003)-> 3042548 (LOINC 52810-9, “Current Medication, Frequency”)
Question-level LOINC concept matches the item
- Line 246 - found a better term
Disposition reason
4260904 (SNOMED 410666004) ->
44810920 (SNOMED 871401000000109), relatedMatch, confidence 0.5
No exact standard concept; nearest related concept kept and was flagged for review

- We used to have the lab results in the separate mapping file, but they are all merged. In the merged list the UCUM unit concept id was blank in the target_concept_id column. They are now all filled in. 25 rows of imported lab result unit (line range: 506-613)
- Values corrupted by excel- found few values that were turned into a scientific notation when the code value was long ( i.e. 434941000124101, 859981000000107, 449921000124105, 873751000000103, 368441000000106)
For example, case of 4-Mar when it should be 3-4 and case of 2-Jan when it should be 1-2

- line 862,865,863,866 - concept id was entered into the code field - corrected.

- Concept names did not match the concept name in the OMOP vocabulary table - corrected

- Laterality qualifiers added for eye and foot measurements - missing qualifier values added
Snellen visual acuity rows were missing the qualifier concept id for right eye or left eye
Monofilament foot sensation rows were missing the laterality qualifier - right or left


## Latest Release version tag v1.0-tvst2026
- the latest release version of the concept map csv file is ai_readi_redcap_value_set_mapping_common.csv - tag v1.0-tvst-2026
- fronzen release copy of the concept map csv file is located in the manuscript-tvst2026/
- working version is located in the redCapDataDictionaryWithExtension/value-set-mapping-common/
Please note, the working directory contains the ai_readi_redcap_value_set_mapping_common.csv, which should be used for ETL. This file is working file so it can change after each release. The fronzen release version is in the manuscript-tvst2026 folder.
- REDCap survey start/completion timestamp concepts swapped back for the PhenX racial and ethnic discrimination survey (`pxrdstartts` → 2005200558 Survey Started Timestamp; `pxrdcmpts` → 2005200559 Survey Completed Timestamp).
- Lab result unit rows (25) that had a blank UCUM `TARGET_CONCEPT_ID` after the lab mappings were merged into the common file are now filled in.
- Excel-corrupted values restored: long concept codes converted to scientific notation (e.g., 434941000124101, 859981000000107, 449921000124105, 873751000000103, 368441000000106) and answer ranges converted to dates (e.g., "4-Mar" → "3-4", "2-Jan" → "1-2").
- Concept IDs that had been entered in the concept code field corrected.
- Concept names that did not match the OMOP vocabulary concept name corrected.
- Missing laterality qualifiers added: Snellen visual acuity rows (Right eye / Left eye) and monofilament foot sensation rows (Structure of right foot / Structure of left foot).
15 changes: 15 additions & 0 deletions release_notes_v1.0-tvst.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
Archived on Zenodo: https://doi.org/10.5281/zenodo.23375723

First archived release, accompanying the manuscript *"A Generalizable Framework for Mapping Survey, Vision Care, and Cognitive Assessment Data to the OMOP Common Data Model in AI-READI"* (Translational Vision Science & Technology, 2026).

**What's in this release**
- `manuscript-tvst2026/`: the frozen REDCap-to-OMOP mapping file (2,004 rows) used in the manuscript, frozen SSSOM and LinkML copies, count scripts that reproduce every number in the paper (`count_concept_map.sql`, `run_counts.py`), the OMOP vocabulary validation script, Figures 1–3 and Supplementary Tables S1–S2.
- AI-READI custom vocabularies (AIREADI, AIREADI-Vision, AIREADI-MOCA) submitted to the OHDSI Vocabulary Working Group.

**Key numbers (Table 2):** 2,906 curated elements; 1,900 mapped (65.4%): 1,412 to 380 standard concepts and 488 to 343 AI-READI custom concepts.

**Reproduce:** `cd manuscript-tvst2026 && pip install duckdb && python run_counts.py`

The working mapping in `redCapDataDictionaryWithExtension/value-set-mapping-common/` may change after this release. Use this tag for the version cited in the paper.

See `CHANGELOG.md` for the full list of changes.
Loading