This subtopic covers the scoping for the design of data instrumentation. Data instrumentation aims to collect the data needed to answer specific questions to inform learning engineering decisions or as feedback loops to directly support learning. Learning engineering teams also aim to understand what data not to collect, i.e. data that are not needed to answer key questions, due to the costs associated with excessive instrumentation.
- Purpose-Driven Data Collection: A key consideration is defining what data is needed to support learning objectives. Instrumentation should focus on collecting data that directly answers specific questions about learning effectiveness, learner engagement, contextual factors, content understanding or skills improvement. Learning engineers need to identify the metrics that matter, ensuring that all collected data has a clear purpose. Metrics may include process measures for the learning experience, measures of the motivation of learners, and measures of the enclosing environment that might help or hinder learning.
- Avoiding Unnecessary Data: Collecting irrelevant data can lead to excessive storage requirements, introduce noise that complicates analysis, and increase privacy risks. It is crucial to distinguish between data that provides actionable insights and data that is redundant or unlikely to contribute meaningfully to decision-making.