A Deep Learning Technology based OCR Framework for Recognition Handwritten Expression and Text

Abstract

Recently Optical character recognition (OCR) based on deep learning technology has achieved great advance and broadly applied in various industries. However it still faces many challenging problems in handwritten text recognition and mathematical expression recognition, such as handwritten Chinese recognition, mixture of printed and handwritten Chinese characters, mathematical expression (ME), chemical equations. In traditional OCR, features selection played a vital role for recognition accuracy, while hand-crafted features are costly and time-consuming. In this paper, we introduce a deep learning based framework to detect and recognize handwritten and printed text or math expression. The framework consists of three components. The first component is DCN (Detection & Classification Network), which based on SSD model to detects and classify mathematical expression and text. The second component consists of text recognition and ME recognition models. The final component merges multiple outputs of the second stage into a whole text. Experiment results show that our framework achieves a relative 10% improvement in mixture of texts and MEs which are printed or handwritten in images. The framework has been deployed for recognition paper or homework at one online education platform.