PreDyctor

What it does

PreDyctor is a dysgraphia pre-screening system that estimates the chances of Chinese speakers having dysgraphia based on their Chinese handwriting. Powered by both rule-based and AI models, PreDyctor achieves 100% sensitivity and 89.5% specificity on our data.

Your inspiration

Dysgraphia is a learning disorder associated with impaired writing ability. People with dysgraphia have difficulty with the legibility and writing pace. Dysgraphia affects over 300K people in Hong Kong as a main type of dyslexia. An estimated 10% of the global population are suffering from it, including James Dyson. Early intervention is critical for children with dysgraphia to avoid long-term effects, while traditional diagnosis methods involving psychologists normally require long waits and high costs. An automated pre-screening tool can notably reduce the wait times and costs but has yet to be developed for Chinese speakers.

How it works

Taking Chinese handwritten characters as input, PreDyctor adopts two independent models to analyze a user’s handwriting: a rule-based scoring model and a similarity-based comparison model. In the rule-based model, we designed a stroke matching algorithm to match the strokes of the user’s handwritten characters with the strokes of paradigm characters, and crafted a criteria marking algorithm to score the user’s handwriting according to rules provided by dysgraphia specialists. In the similarity-based model, we utilized a Convolutional Neural Network (CNN) as a feature extractor to identify whether a user’s handwriting is more similar to that of healthy people or people having dysgraphia. We aggregated the outputs of the two models to generate a final estimate of the probability that the user has dysgraphia. A web app is developed for users to upload their handwriting to PreDyctor, and the models’ outputs and final results are also visualized.

Design process

First, a dataset containing handwriting images and dysgraphia diagnosis results of primary school students was constructed for model training. We initially attempted to predict whether a student has dysgraphia using deep learning models that are popular in the field of OCR. However, due to the scarcity of training samples and complex radical structures, we later focused on the rule-based algorithms that required fewer data and could decompose the structural information of radicals and strokes in handwritten characters. We first analyze the handwriting correctness and score each handwriting image according to the rules specified in the marking scheme provided by dysgraphia professionals, and then generate prediction results based on the scores. In the process of analyzing the data, we realized that the similarity between handwritten Chinese characters could be well utilized. We designed a model that determines whether a user has dysgraphia by comparing input handwritten characters with characters written by healthy people and with characters written by people having dysgraphia respectively, using a pretrained VGG-16 network as a feature extractor. Finally, we used ensemble methods to aggregate the results and attain the final predictions.

How it is different

While traditional pre-screening methods for dysgraphia require professional experts or special equipment which are very expensive for most families, our product is an automated computer-aided tool that helps diagnose children with dysgraphia and meanwhile easy to reach for most people. Among all similar dysgraphia pre-screening systems, our design is the only one specifically targeted at Chinese handwritten characters while other systems are mostly built for English and Spanish speakers. Moreover, compared with other designs that are still in the research phase or only partially released their analyses and prediction, our product is fully implemented as an easily accessible web application and provides a comprehensive report including the probability that users have dysgraphia.

Future plans

Our team is currently exploring ways to incorporate hierarchical structural information of radicals to further improve the models’ detection efficiency, and interviewing dysgraphia professionals and educational psychologists to learn how PreDyctor could fit into their practice. We also plan to expand the input of our system to similar languages such as Japanese, Korean to benefit a larger population. We aim to develop and test our product to get it medically approved so that physicians can use it as a clinical tool to assist with diagnosis and help children suffering from dysgraphia get diagnosed and treated at an early age at a low cost.

Awards

Best Final Year Project 2021-2022, awarded by Department of Computer Science and Engineering at Hong Kong University of Science and Technology