opencv +数字识别

Posted 2021-04-25 dotNET跨平台

现在很多场景需要使用的数字识别，比如银行卡识别，以及车牌识别等，在AI领域有很多图像识别算法，大多是居于opencv 或者谷歌开源的tesseract 识别.

由于公司业务需要，需要开发一个客户端程序，同时需要在xp这种老古董的机子上运行，故研究了如下几个数字识别方案，如果大家有更好的方案可以留言告知我，大家一起学习借鉴，不过需要支持XP系统，万分感谢！

ocr 识别的不同选择方案

•tesseract

•放弃：谷歌的开源tesseract ocr识别目前最新版本不支持xp系统

     •云端ocr 识别接口(不适用)

•费用比较贵：•场景不同，我们的需求是可能毫秒级别就需要调用一次ocr 识别

     •opencv

•概念:OpenCV是一个基于BSD许可（开源）发行的跨平台计算机视觉库，可以运行在Linux、Windows、android和Mac OS操作系统上。它轻量级而且高效——由一系列 C 函数和少量 C++ 类构成，同时提供了Python、Ruby、MATLAB等语言的接口，实现了图像处理和计算机视觉方面的很多通用算法。

以上几种ocr 识别比较，最后选择了opencv 的方式进行ocr 数字识别,下面讲解通过ocr识别的基本流程和算法.

opencv 数字识别流程及算法解析

要通过opencv 进行数字识别离不开训练库的支持，需要对目标图片进行大量的训练，才能做到精准的识别出目标数字;下面我会分别讲解图片训练的过程及识别的过程.

opencv 识别算法原理

1.比如下面一张图片，需要从中识别出正确的数字，需要对图片进行灰度、二值化、腐蚀、膨胀、寻找数字轮廓、切割等一系列操作.

原图

灰度化图

二值化图

寻找轮廓

识别后的结果图

以上就是简单的图片进行灰度化、二值化、寻找数字轮廓得到的识别结果(==这是基于我之前训练过的数字模型下得到的识别结果==) 有些图片比较赋值，比如存在背景斜杠等的图片则需要一定的腐蚀或者膨胀等处理，才能寻找到正确的数字轮廓.

上面的说到我这里使用的是opencv 图像处理库进行的ocr 识别，那我这里简单介绍下C# 怎么使用opencv 图像处理看;

为了在xp上能够运行 我这里通过nuget 包引用了 OpenCvSharp-AnyCPU 第三方库,它使用的是opencv 2410 版本,你们如果不考虑xp系统的情况下开源使用最新的版本，最新版本支持了更多的识别算法.

右击你的个人项目，选择“管理Nuget程序包”。在包管理器页面中，点击“浏览”选项，然后在搜索框中键入“OpenCvSharp-AnyCPU”。选择最顶端的正确项目，并在右侧详情页中点击“安装”，等待安装完成即可。

以上的核心代码如下:

 private void runSimpleOCR(string pathName) { //构造opcvOcr 库，这里的是我单独对opencv 库进行的一次封装,加载训练库模板 var opencvOcr = new OpencvOcr($"{path}Template\\Traindata.xml", opencvOcrConfig: new OCR.Model.OpencvOcrConfig() { ErodeLevel = 2.5, ThresholdType = OpenCvSharp.ThresholdType.Binary, ZoomLevel = 2, });
 var img = new Bitmap(this.txbFilaName.Text);
 var mat = img.ToMat();
 //核心识别方法 var str = opencvOcr.GetText(mat, isDebug: true); this.labContent.Content = str; }

opencvOcr 的核心代码如下

        #region 属性 const double Thresh = 80; const double ThresholdMaxVal = 255; const int _minHeight = 35; bool _isDebug = false; CvKNearest _cvKNearest = null; OpencvOcrConfig _config = new OpencvOcrConfig() { ZoomLevel = 2, ErodeLevel = 3 }; #endregion
 /// <summary> /// 构造函数 /// </summary> /// <param name="path">训练库完整路径</param> /// <param name="opencvOcrConfig">OCR相关配置信息</param> public OpencvOcr(string path, OpencvOcrConfig opencvOcrConfig = null) { if (string.IsNullOrEmpty(path)) throw new ArgumentNullException("path is not null");
 if (opencvOcrConfig != null) _config = opencvOcrConfig;
 this.LoadKnearest(path); }
 /// <summary> /// 加载Knn 训练库模型 /// </summary> /// <param name="dataPathFile"></param> /// <returns></returns> private CvKNearest LoadKnearest(string dataPathFile) { if (_cvKNearest == null) {
 using (var fs = new FileStorage(dataPathFile, FileStorageMode.Read)) { var samples = fs["samples"].ReadMat(); var responses = fs["responses"].ReadMat(); this._cvKNearest = new CvKNearest(); this._cvKNearest.Train(samples, responses); } } return _cvKNearest; }
 /// <summary> /// OCR 识别,仅仅只能识别单行数字  /// </summary> /// <param name="kNearest">训练库</param> /// <param name="path">要识别的图片路径</param> public override string GetText(Mat src, bool isDebug = false) { this._isDebug = isDebug;
 #region 图片处理 var respMat = MatProcessing(src, isDebug); if (respMat == null) return ""; #endregion
 #region 查找轮廓 var sortRect = FindContours(respMat.FindContoursMat); #endregion
 return GetText(sortRect, respMat.ResourcMat, respMat.RoiResultMat); }
 /// <summary> /// 查找轮廓 /// </summary> /// <param name="src"></param> /// <returns></returns> private List<Rect> FindContours(Mat src) { try { #region 查找轮廓 Point[][] contours; HierarchyIndex[] hierarchyIndexes; Cv2.FindContours( src, out contours, out hierarchyIndexes, mode: OpenCvSharp.ContourRetrieval.External, method: OpenCvSharp.ContourChain.ApproxSimple);
 if (contours.Length == 0) throw new NotSupportedException("Couldn't find any object in the image."); #endregion
 #region 单行排序（目前仅仅支持单行文字,多行文字顺序可能不对，按照x坐标进行排序） var sortRect = GetSortRect(contours, hierarchyIndexes); sortRect = sortRect.OrderBy(item => item.X).ToList(); #endregion
 return sortRect; } catch { }
 return null; }
 /// <summary> /// 获得切割后的数量列表 /// </summary> /// <param name="contours"></param> /// <param name="hierarchyIndex"></param> /// <returns></returns> private List<Rect> GetSortRect(Point[][] contours, HierarchyIndex[] hierarchyIndex) { var sortRect = new List<Rect>();
 var _contourIndex = 0; while ((_contourIndex >= 0)) { var contour = contours[_contourIndex]; var boundingRect = Cv2.BoundingRect(contour); //Find bounding rect for each contour
 sortRect.Add(boundingRect); _contourIndex = hierarchyIndex[_contourIndex].Next; } return sortRect; }

 /// <summary> /// 是否放大 /// </summary> /// <param name="src"></param> /// <returns></returns> private bool IsZoom(Mat src) { if (src.Height <= _minHeight) return true;
 return false; }

 private List<EnumMatAlgorithmType> GetAlgoritmList(Mat src) { var result = new List<EnumMatAlgorithmType>(); var algorithm = this._config.Algorithm;
 #region 自定义的算法 try { if (algorithm.Contains("|")) { result = algorithm.Split('|').ToList() .Select(item => (EnumMatAlgorithmType)Convert.ToInt32(item)) .ToList();
 if (!IsZoom(src)) result.Remove(EnumMatAlgorithmType.Zoom);
 return result; } } catch { }
 #endregion
 #region 默认算法 if (IsZoom(src)) { result.Add(EnumMatAlgorithmType.Zoom); } if (this._config.ThresholdType == ThresholdType.Binary) { //result.Add(EnumMatAlgorithmType.Blur);
 result.Add(EnumMatAlgorithmType.Gray); result.Add(EnumMatAlgorithmType.Thresh); if (this._config.DilateLevel > 0) result.Add(EnumMatAlgorithmType.Dilate);
 result.Add(EnumMatAlgorithmType.Erode); return result; } //result.Add(EnumMatAlgorithmType.Blur);
 result.Add(EnumMatAlgorithmType.Gray); result.Add(EnumMatAlgorithmType.Thresh); if (this._config.DilateLevel > 0) result.Add(EnumMatAlgorithmType.Dilate);
 result.Add(EnumMatAlgorithmType.Erode); return result; #endregion }

 /// <summary> /// 对查找的轮廓数据进行训练模型匹配，这里使用的是KNN 匹配算法 /// </summary> private string GetText(List<Rect> sortRect, Mat source, Mat roiSource) { var response = ""; try { if ((sortRect?.Count ?? 0) <= 0) return response;
 var contourIndex = 0; using (var dst = new Mat(source.Rows, source.Cols, MatType.CV_8UC3, Scalar.All(0))) { sortRect.ForEach(boundingRect => { try { #region 绘制矩形 if (this._isDebug) { Cv2.Rectangle(source, new Point(boundingRect.X, boundingRect.Y), new Point(boundingRect.X + boundingRect.Width, boundingRect.Y + boundingRect.Height), new Scalar(0, 0, 255), 1);
 Cv2.Rectangle(roiSource, new Point(boundingRect.X, boundingRect.Y), new Point(boundingRect.X + boundingRect.Width, boundingRect.Y + boundingRect.Height), new Scalar(0, 0, 255), 1); } #endregion
 #region 单个ROI var roi = roiSource.GetROI(boundingRect); //Crop the image roi = roi.Compress(); var result = roi.ConvertFloat(); #endregion
 #region KNN 匹配 var results = new Mat(); var neighborResponses = new Mat(); var dists = new Mat(); var detectedClass = (int)this._cvKNearest.FindNearest(result, 1, results, neighborResponses, dists); var resultText = detectedClass.ToString(CultureInfo.InvariantCulture); #endregion
 #region 匹配 var isDraw = false; if (detectedClass >= 0) { response += detectedClass.ToString(); isDraw = true; } if (detectedClass == -1 && !response.Contains(".")) { response += "."; resultText = "."; isDraw = true; } #endregion
 #region 绘制及输出切割信息库 try { //if (this._isDebug) //{ Write(contourIndex, detectedClass, roi); //} } catch { }
 if (this._isDebug && isDraw) { Cv2.PutText(dst, resultText, new Point(boundingRect.X, boundingRect.Y + boundingRect.Height), 0, 1, new Scalar(0, 255, 0), 2); } #endregion
 result?.Dispose(); results?.Dispose(); neighborResponses?.Dispose(); dists?.Dispose(); contourIndex++; } catch (Exception ex) { TextHelper.Error("GetText ex", ex); } });
 #region 调试模式显示过程 source.IsDebugShow("Segmented Source", this._isDebug); dst.IsDebugShow("Detected", this._isDebug); dst.IsDebugWaitKey(this._isDebug); dst.IsDebugImWrite("dest.jpg", this._isDebug); #endregion } } catch { throw; } finally { source?.Dispose(); roiSource?.Dispose(); } return response; }
 /// <summary> /// 图片处理算法 /// </summary> /// <param name="src"></param> /// <param name="isDebug"></param> /// <returns></returns> public ImageProcessModel MatProcessing(Mat src, bool isDebug = false) { src.IsDebugShow("原图", isDebug);
 var list = GetAlgoritmList(src); var resultMat = new Mat(); src.CopyTo(resultMat); var isZoom = IsZoom(src); list?.ForEach(item => { switch (item) { case EnumMatAlgorithmType.Dilate: resultMat = resultMat.ToDilate(Convert.ToInt32(this._config.DilateLevel)); resultMat.IsDebugShow(EnumMatAlgorithmType.Dilate.GetDescription(), isDebug); break; case EnumMatAlgorithmType.Erode: var eroderLevel = isZoom ? this._config.ErodeLevel * this._config.ZoomLevel : this._config.ErodeLevel; resultMat = resultMat.ToErode(eroderLevel); resultMat.IsDebugShow(EnumMatAlgorithmType.Erode.GetDescription(), isDebug); break; case EnumMatAlgorithmType.Gray: resultMat = resultMat.ToGrey(); resultMat.IsDebugShow(EnumMatAlgorithmType.Gray.GetDescription(), isDebug); break; case EnumMatAlgorithmType.Thresh: var thresholdValue = this._config.ThresholdValue <= 0 ? resultMat.GetMeanThreshold() : this._config.ThresholdValue; resultMat = resultMat.ToThreshold(thresholdValue, thresholdType: this._config.ThresholdType); resultMat.IsDebugShow(EnumMatAlgorithmType.Thresh.GetDescription(), isDebug); break; case EnumMatAlgorithmType.Zoom: resultMat = resultMat.ToZoom(this._config.ZoomLevel); src = resultMat; resultMat.IsDebugShow(EnumMatAlgorithmType.Zoom.GetDescription(), isDebug); break; case EnumMatAlgorithmType.Blur: resultMat = resultMat.ToBlur(); src = resultMat; resultMat.IsDebugShow(EnumMatAlgorithmType.Blur.GetDescription(), isDebug); break; } });
 var oldThreshImage = new Mat(); resultMat.CopyTo(oldThreshImage);
 return new ImageProcessModel() { ResourcMat = src, FindContoursMat = oldThreshImage, RoiResultMat = resultMat }; }

opencv 图片处理开放出去的配置对象实体如下：

 public class OpencvOcrConfig { /// <summary> /// 放大程度级别 默认2 /// </summary> public double ZoomLevel { set; get; }
 /// <summary> /// 腐蚀级别 默认2.5 /// </summary> public double ErodeLevel { set; get; }
 /// <summary> /// 膨胀 /// </summary> public double DilateLevel { set; get; }
 /// <summary> /// 阀值 /// </summary> public double ThresholdValue { set; get; }
 /// <summary> /// 图片处理算法,用逗号隔开 /// </summary> public string Algorithm { set; get; }
 /// <summary> /// 二值化方式 /// </summary> public ThresholdType ThresholdType { set; get; } = ThresholdType.BinaryInv;
 /// <summary> /// 通道模式 /// </summary> public OcrChannelTypeEnums ChannelType { set; get; } = OcrChannelTypeEnums.BlackBox;
 }

opencv 图片处理算法扩展方法如下：

 public static partial class OpenCvExtensions { private const int Thresh = 200; private const int ThresholdMaxVal = 255;
 /// <summary> /// Bitmap Convert Mat /// </summary> /// <param name="bitmap"></param> /// <returns></returns> public static Mat ToMat(this System.Drawing.Bitmap bitmap) { return OpenCvSharp.Extensions.BitmapConverter.ToMat(bitmap); }
 /// <summary> /// Bitmap Convert Mat /// </summary> /// <param name="bitmap"></param> /// <returns></returns> public static System.Drawing.Bitmap ToBitmap(this Mat mat) { return OpenCvSharp.Extensions.BitmapConverter.ToBitmap(mat); }

 public static bool MatIsEqual(this Mat mat1, Mat mat2) { try { if (mat1.Empty() && mat2.Empty()) { return true; } if (mat1.Cols != mat2.Cols || mat1.Rows != mat2.Rows || mat1.Dims() != mat2.Dims() || mat1.Channels() != mat2.Channels()) { return false; } if (mat1.Size() != mat2.Size() || mat1.Type() != mat2.Type()) { return false; } var nrOfElements1 = mat1.Total() * mat1.ElemSize(); if (nrOfElements1 != mat2.Total() * mat2.ElemSize()) return false;
 return MatPixelEqual(mat1, mat2); } catch (Exception ex) { TextHelper.Error("MatIsEqual 异常", ex); return true; } }
 /// <summary> /// 灰度 /// </summary> /// <param name="mat"></param> /// <returns></returns> public static Mat ToGrey(this Mat mat) { try { Mat grey = new Mat(); Cv2.CvtColor(mat, grey, OpenCvSharp.ColorConversion.BgraToGray); return grey; } catch { return mat; } }
 /// <summary> /// 二值化 /// </summary> /// <param name="data"></param> /// <returns></returns> public static Mat ToThreshold(this Mat data, double threshValue = 0, ThresholdType thresholdType = ThresholdType.BinaryInv) { Mat threshold = new Mat();
 if (threshValue == 0) threshValue = Thresh; Cv2.Threshold(data, threshold, threshValue, ThresholdMaxVal, thresholdType); if (threshold.IsBinaryInv()) { Cv2.Threshold(threshold, threshold, threshValue, ThresholdMaxVal, ThresholdType.BinaryInv);            }
 return threshold; }
 /// <summary> /// 是否调试显示 /// </summary> /// <param name="src"></param> /// <param name="name"></param> /// <param name="isDebug"></param> public static void IsDebugShow(this Mat src, string name, bool isDebug = false) { if (!isDebug) return;
 Cv2.ImShow(name, src); }
 public static void IsDebugWaitKey(this Mat src, bool isDebug = false) { if (!isDebug) return;
 Cv2.WaitKey(); }
 public static void IsDebugImWrite(this Mat src, string path, bool isDebug = false) { if (!isDebug) return;
 try { Cv2.ImWrite(path, src); } catch { } }
 /// <summary> /// Mat 转成另外一种存储矩阵方式 /// </summary> /// <param name="roi"></param> /// <returns></returns> public static Mat ConvertFloat(this Mat roi) { var resizedImage = new Mat(); var resizedImageFloat = new Mat(); Cv2.Resize(roi, resizedImage, new Size(10, 10)); //resize to 10X10 resizedImage.ConvertTo(resizedImageFloat, MatType.CV_32FC1); //convert to float var result = resizedImageFloat.Reshape(1, 1); return result; }
 /// <summary> /// 腐蚀 /// </summary> /// <param name="mat"></param> /// <returns></returns> public static Mat ToErode(this Mat mat, double level) {
            #region 自动会判断是否需要腐蚀 if (level < 1) { return mat;            } #endregion
 var erode = new Mat();
 var copyMat = new Mat(); mat.CopyTo(copyMat);
 Cv2.Erode(mat, erode, Cv2.GetStructuringElement(StructuringElementShape.Ellipse, new Size(level, level))); return erode; }
 /// <summary> /// 膨胀 /// </summary> /// <param name="mat"></param> /// <returns></returns> public static Mat ToDilate(this Mat mat, int level) { if (level <= 0) return mat; var dilate = new Mat(); Cv2.Dilate(mat, dilate, Cv2.GetStructuringElement(StructuringElementShape.Ellipse, new Size(level, level))); return dilate; }
 /// <summary> /// mat 转Roi /// </summary> /// <param name="image"></param> /// <param name="boundingRect"></param> /// <returns></returns> public static Mat GetROI(this Mat image, Rect boundingRect) { try { return new Mat(image, boundingRect); //Crop the image } catch{} return null; }
 /// <summary> /// 获取平均阀值 /// </summary> /// <param name="mat"></param> /// <returns></returns> public static int GetMeanThreshold(this Mat mat) { var width = mat.Width; var height = mat.Height;
 var m = mat.Reshape(1, width * height); return (int)m.Sum() / (width * height); }
 /// <summary> /// 获得二值化阀值 /// </summary> /// <param name="bitmap"></param> /// <returns></returns> public static int GetMeanThreshold(this System.Drawing.Bitmap bitmap) { using (var mat = bitmap.ToMat()) using (var grap = mat.ToGrey()) { return grap.GetMeanThreshold(); } }
 public static bool IsErode(this System.Drawing.Bitmap bitmap) { using (var mat = bitmap.ToMat()) using (var grap = mat.ToGrey())            { var thresholdValue = grap.GetMeanThreshold(); using (var threshold = grap.ToThreshold(thresholdValue, ThresholdType.BinaryInv)) { return threshold.IsErode(); } } }
 /// <summary> /// 放大 /// </summary> /// <param name="img"></param> /// <param name="times"></param> /// <returns></returns> public static Mat ToZoom(this Mat img, double times) { if (times <= 0) return img; var width = img.Width * times; var height = img.Height * times;
 img = img.Resize(new Size(width, height), 0, 0, Interpolation.NearestNeighbor); return img; }
 /// <summary> /// 均值滤波 /// </summary> /// <param name="img"></param> /// <returns></returns> public static Mat ToBlur(this Mat img) { return img.Blur(new Size(3, 3)); }
 public static Mat Compress(this Mat img) { var width = 28.0 * img.Width / img.Height;
 var fWidth = width / img.Width;            var fHeight = 28.0 / img.Height;             img = img.Resize(new Size(width, 28), fWidth, fHeight, Interpolation.NearestNeighbor); return img; }
 public static bool MatPixelEqual(this Mat src, Mat are) { var width = src.Width; var height = src.Height; var sum = width * height;
 for (int row = 0; row < height; row++) { for (int col = 0; col < width; col++) { byte p = src.At<byte>(row, col); //获对应矩阵坐标的取像素 byte pAre = are.At<byte>(row, col); if (p != pAre) return false; } } return true; }
 public static int GetSumPixelCount(this Mat threshold) { var width = threshold.Width; var height = threshold.Height; var sum = width * height;
 var value = 0; for (int row = 0; row < height; row++) { for (int col = 0; col < width; col++) { byte p = threshold.At<byte>(row, col); //获对应矩阵坐标的取像素 value++; } } return value; }
 public static int GetPixelCount(this Mat threshold, System.Drawing.Color color) { var width = threshold.Width; var height = threshold.Height; var sum = width * height;
 var value = 0; for (int row = 0; row < height; row++) { for (int col = 0; col < width; col++) { byte p = threshold.At<byte>(row, col); //获对应矩阵坐标的取像素 if (Convert.ToInt32(p) == color.R) { value++; } } } return value; }
 /// <summary> /// 是否需要二值化反转 /// </summary> /// <param name="threshold"></param> /// <returns></returns> public static bool IsBinaryInv(this Mat threshold) { var width = threshold.Width; var height = threshold.Height; var sum = Convert.ToDouble(width * height);
 var black = GetPixelCount(threshold, System.Drawing.Color.Black);
 return (Convert.ToDouble(black) / sum) < 0.5; }
 /// <summary> /// 是否需要腐蚀 /// </summary> /// <param name="mat"></param> /// <returns></returns> public static bool IsErode(this Mat mat) { var percent = mat.GetPercent(); return percent >= 0.20; }
 /// <summary> /// 获得白色像素占比 /// </summary> /// <param name="threshold"></param> /// <returns></returns> public static double GetPercent(this Mat threshold) { var width = threshold.Width; var height = threshold.Height; var sum = Convert.ToDouble(width * height);
 var white = GetPixelCount(threshold, System.Drawing.Color.White); return (Convert.ToDouble(white) / sum); }
 /// <summary> /// 根据模板查找目标图片的在原图标中的开始位置坐标 /// </summary> /// <param name="source"></param> /// <param name="template"></param> /// <param name="matchTemplateMethod"></param> /// <returns></returns> public static Point FindTemplate(this Mat source, Mat template, MatchTemplateMethod matchTemplateMethod = MatchTemplateMethod.SqDiffNormed) { if (source == null) return new OpenCvSharp.CPlusPlus.Point();
 var result = new Mat(); Cv2.MatchTemplate(source, template, result, matchTemplateMethod);
 Cv2.MinMaxLoc(result, out OpenCvSharp.CPlusPlus.Point minVal, out OpenCvSharp.CPlusPlus.Point maxVal);
 var topLeft = new OpenCvSharp.CPlusPlus.Point(); if (matchTemplateMethod == MatchTemplateMethod.SqDiff || matchTemplateMethod == MatchTemplateMethod.SqDiffNormed) { topLeft = minVal; } else { topLeft = maxVal; } return topLeft; } }

以上代码中开源对图片进行轮廓切割，同时会生成切割后的图片代码如下

#region 绘制及输出切割信息库 try {
 Write(contourIndex, detectedClass, roi);
 } catch { }#endregion
private void Write(int contourIndex, int detectedClass, Mat roi){ Task.Factory.StartNew(() => { try { var templatePath = $"{AppDomain.CurrentDomain.BaseDirectory}template"; FileHelper.CreateDirectory(templatePath); var templatePathFile = $"{templatePath}/{contourIndex}_{detectedClass.ToString()}.png"; Cv2.ImWrite(templatePathFile, roi); if (!roi.IsDisposed) { roi.Dispose(); } } catch {} });}

切割后的图片如下：

这里我已经对数字进行切割好了，接下来就是需要对0-9 这些数字进行分类(建立文件夹进行数字归类)，如下：

图中的每一个分类都是我事先切割好的数字图片,图中有-1 和-2 这两个特殊分类，-1 里面我是放的是“.”好的分类，用于训练“.”的图片，这样就可以识别出小数点的数字支持. -2 这个分类主要是其他一些无关紧要的图片，也就是不是数字和点的都归为这一类中.

现在训练库分类已经建立好了，接下来我们需要对这些分类数字进行归一化处理，生成训练模型. 代码如下：

 private void Button_Click_1(object sender, RoutedEventArgs e) { var opencvOcr = new OpencvOcr($"{path}Template\\Traindata.xml", opencvOcrConfig: null); opencvOcr.Save($"{path}Template\\NumberWrite", outputPath: $"{path}Template\\Traindata.xml"); MessageBox.Show("生成训练库成功"); //var img = new Bitmap(this.txbFilaName.Text);
 //var str = opencvOcr.GetText(img.ToMat(), isDebug: true); //this.labContent.Content = str; }
 /// <summary> /// 保存训练模型 /// </summary> /// <param name="dataPath"></param> /// <param name="trainExt"></param> /// <param name="dataPathFile"></param> public void Save(string dataPath, string trainExt = "*.png", string outputPath = "") { if (string.IsNullOrEmpty(outputPath)) throw new ArgumentNullException("save dataPath is not null");
 var trainingImages = this.ReadTrainingImages(dataPath, trainExt); var samples = GetSamples(trainingImages); var response = GetResponse(trainingImages);
 //写入到训练库中 using (var fs = new FileStorage(outputPath, FileStorageMode.WriteText)) { fs.Write("samples", samples); fs.Write("responses", response); } }
 /// <summary> /// 根据目录加载文件 /// </summary> /// <param name="path"></param> /// <param name="ext"></param> /// <returns></returns> private IList<ImageInfo> ReadTrainingImages(string path, string ext) { var images = new List<ImageInfo>(); var imageId = 1; foreach (var dir in new DirectoryInfo(path).GetDirectories()) { var groupId = int.Parse(dir.Name); foreach (var imageFile in dir.GetFiles(ext)) { var srcMat = new Mat(imageFile.FullName, OpenCvSharp.LoadMode.GrayScale); var image = srcMat.ConvertFloat(); if (image == null) { continue; }
 images.Add(new ImageInfo { Image = image, ImageId = imageId++, ImageGroupId = groupId }); } } return images; }
 /// <summary> /// Mat 转成另外一种存储矩阵方式 /// </summary> /// <param name="roi"></param> /// <returns></returns> public static Mat ConvertFloat(this Mat roi) { var resizedImage = new Mat(); var resizedImageFloat = new Mat(); Cv2.Resize(roi, resizedImage, new Size(10, 10)); //resize to 10X10 resizedImage.ConvertTo(resizedImageFloat, MatType.CV_32FC1); //convert to float var result = resizedImageFloat.Reshape(1, 1); return result; }
 /// <summary> /// 获取Samples /// </summary> /// <param name="trainingImages"></param> /// <returns></returns> private Mat GetSamples(IList<ImageInfo> trainingImages) { var samples = new Mat(); foreach (var trainingImage in trainingImages) { samples.PushBack(trainingImage.Image); } return samples; }
 private Mat GetResponse(IList<ImageInfo> trainingImages) { var labels = trainingImages.Select(x => x.ImageGroupId).ToArray(); var responses = new Mat(labels.Length, 1, MatType.CV_32SC1, labels); var tmp = responses.Reshape(1, 1); //make continuous var responseFloat = new Mat(); tmp.ConvertTo(responseFloat, MatType.CV_32FC1); // Convert to float
 return responses; }

到这里ocr 训练模型以及建立好了，会在目录中生成一个Traindata.xml 的训练模型库,我们来打开这个训练模型库文件探索它的神秘的容颜.

如果大家有更好的数字识别方案可以留言告知我，这样可以更好的应用到实际场景中需求场景：

1. 客户端调用，不走api方式

2. 必须要xp这种老爷机的支持