Research and evaluation
Research and evaluation of task-delay risk
Lumivox studies task-delay prediction using behavioral data and an interpretable machine-learning classifier. Model metrics, usability outcomes, and platform records are reported separately because they answer different questions.
Problem definition
The model estimates whether an active task is likely to become late within a 14-day prediction horizon. It supports prioritization and reflection; it does not predict grades, personal ability, or academic success.
Behavioral data pipeline
Versioned task snapshots are derived from Lumivox task and activity records. Features are prepared under the native-task-risk-v2 schema, split by unique task into training, validation, and held-out test groups, and evaluated without treating repeated snapshots of one task as independent tasks across splits.
Task-risk model
The selected classifier is Logistic regression with balanced class weights. Version native-task-risk-v2-20260925-021822 is registered as deployment-ready. Logistic regression was selected to preserve a comparatively interpretable decision boundary and class balancing addresses the late-task class distribution.
Dataset
The model artifact records 70,379 snapshots from 6,643 unique tasks and 129 users. The held-out test set contains 14,315 snapshots across 1,329 tasks.
Evaluation methodology
The decision threshold is 0.4. It was chosen on validation data under constraints requiring at least 0.75 recall and 0.50 specificity while limiting the predicted-positive rate. Recall is prioritized because missing a genuinely at-risk task is considered more costly than showing an additional warning.
Explore task-delay risk prediction features
Held-out test metrics
These figures describe classifier behavior on the held-out test set. They are not percentages of students who improved.
| Accuracy | 67.31% |
|---|---|
| Precision | 59.50% |
| Recall | 76.90% |
| Specificity | 59.97% |
| Balanced accuracy | 68.44% |
| F1 | 67.09% |
| ROC-AUC | 0.7512 |
Confusion matrix
- True negative
- 4,865
- False positive
- 3,247
- False negative
- 1,433
- True positive
- 4,770
Deployment validation
The artifact passed its recorded deployment gates, including validation constraints, test-set checks, artifact serialization, feature-schema verification, and required metadata checks. Passing these gates means the artifact met the project's deployment contract; it does not remove the need for monitoring.
User evaluation
A separate usability study collected 15 responses. The standard System Usability Scale calculation produced an average SUS score of 63.5 (range 47.5 to 92.5). Across 180 task checks, 166 were marked successful, giving an aggregate task-completion rate of 92.2%. Participant names and verbatim comments are withheld because public-use consent has not been established.
- Participants
- 15
- Average SUS
- 63.5
- Task completion
- 92.2%
Platform snapshot
As of 27 September 2026, the Supabase project contained the aggregate records below. These are database counts, not efficacy or active-user claims, and they will change as the product is used.
- Profile records
- 168
- Task records
- 6,697
- Goal records
- 462
- Focus-session records
- 46,932
Snapshot: 2026-09-27
Limitations
- • The native dataset contains 129 unique users and may not represent learners outside the current project population.
- • A 59.50% precision and 59.97% specificity mean false-positive warnings remain a material tradeoff.
- • The usability sample is small and the SUS average is descriptive of these 15 participants only.
- • Offline test performance does not establish improved learning outcomes or long-term user benefit.
Privacy and ethical considerations
Only aggregate evaluation values are shown publicly. Predictions should be presented as assistance with uncertainty, not as judgments about a learner. The product should continue monitoring drift, false-positive burden, accessibility, and the effects of recommendations.