A sensor export can contain both T-004 and T-004. A join treats them as different strings. Trimming solves the mismatch, but it can also turn two rows into the same key. Check for that collision before joining.
This synthetic example uses only the Python standard library:
from collections import defaultdict
raw_ids = ["T-004", "T-004 ", " T-010"]
variants = defaultdict(list)
for row_number, raw in enumerate(raw_ids, start=1):
normalized = raw.strip()
if not normalized:
raise ValueError(f"Row {row_number}: missing experiment ID")
variants[normalized].append((row_number, raw))
for key, rows in variants.items():
if len(rows) > 1:
print(f"{key}: {rows!r}")
The local check prints:
T-004: [(1, 'T-004'), (2, 'T-004 ')]
Keep the raw ID and row number so you can trace both records. Consult the original run log to decide whether they are duplicate exports or distinct runs that should have had different IDs. Only join after making the intended key unique. Avoid changing case or removing leading zeros unless the ID specification requires it.
I work on RigorMed, an AI research workspace for organizing inputs and producing reviewable research outputs. Learn about RigorMed.
Top comments (0)