caffe:用自己的数据训练网络mnist
Posted 北海盗
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了caffe:用自己的数据训练网络mnist相关的知识,希望对你有一定的参考价值。
画黑底白字的软件:KolourPaint。
假设所有“1”的图片放到名字为1的文件夹下。(0-9类似)。。获取每个数字的名称文件后,手动表上标签。然后合成train。txt
1、获取文件夹内全部图像的名称:
find ./1 -name ‘*.png‘>1.txt
//此时的1.txt文件中的图像名称包括路劲信息,要把前面的路径信息去掉。
$ sudo sed -i "s/.\/1\///g" 1.txt //(\表示转义,所以这里用双引号而不是单引号)
2、要在1.txt 内的每个名称后面加上标签
1.txt:
1101.png 1
1102.png 1
.....(如此)
3、将图片数据转换为lmdb格式的数据
caffe/examples下建一个文件保存训练用的文件:sd_mnist
3.1 sd_mnist下创建一个sd_create_lmdb.sh用来转换图片格式:
sudo vim sd_create_lmdb.sh ,内容如下:
#!/usr/bin/env sh
# Create the imagenet lmdb inputs
# N.B. set the path to the imagenet train + val data dirs
EXAMPLE=examples/sd_mnist (!注意:这是你在examples下创建的目录)
DATA=data/sd_mnist (!注意:就是你在data文件夹下新建目录,里面有两个图片集(训练和测试训练集)及上面所说的两个txt)
TOOLS=build/tools
TRAIN_DATA_ROOT=data/sd_mnist/train/ (!注意:就是训练图片集路径)
VAL_DATA_ROOT=data/sd_mnist/test/ (!注意:就是测试图片集路径)
# Set RESIZE=true to resize the images to 256x256. Leave as false if images have
# already been resized using another tool.
RESIZE=true
if $RESIZE; then
RESIZE_HEIGHT=28
RESIZE_WIDTH=28
else
RESIZE_HEIGHT=0
RESIZE_WIDTH=0
fi
if [ ! -d "$TRAIN_DATA_ROOT" ]; then
echo "Error: TRAIN_DATA_ROOT is not a path to a directory: $TRAIN_DATA_ROOT"
echo "Set the TRAIN_DATA_ROOT variable in create_imagenet.sh to the path" \
"where the ImageNet training data is stored."
exit 1
fi
if [ ! -d "$VAL_DATA_ROOT" ]; then
echo "Error: VAL_DATA_ROOT is not a path to a directory: $VAL_DATA_ROOT"
echo "Set the VAL_DATA_ROOT variable in create_imagenet.sh to the path" \
"where the ImageNet validation data is stored."
exit 1
fi
echo "Creating train lmdb..."
GLOG_logtostderr=1 $TOOLS/convert_imageset \
--resize_height=$RESIZE_HEIGHT \
--resize_width=$RESIZE_WIDTH \
--shuffle \
$TRAIN_DATA_ROOT \
$DATA/train.txt \ (!注意路劲)
$EXAMPLE/mnist_train_lmdb
echo "Creating test lmdb..."
GLOG_logtostderr=1 $TOOLS/convert_imageset \
--resize_height=$RESIZE_HEIGHT \
--resize_width=$RESIZE_WIDTH \
--shuffle \
$VAL_DATA_ROOT \
$DATA/test.txt \ (!注意路劲)
$EXAMPLE/mnist_test_lmdb
echo "Done."
-----------------------------------------------------------------------
3.2 运行sh example/sd_mnist/sd_create_lmdb.sh
如果成功的话,终端返回的信息中,图片是有大小的而不是0kb。并且在examples/sd_mnist下会有两个文件:mnist_train_lmdb,mnist_test_lmdb它们里面都是data.mdb和lock.mdb。
4、对我们的数据集进行训练:下面的文件都是从caffe\examples\mnist下复制到caffe\examples\sd_mnist下来进行修改的。主要是修改路径信息,整个网络保持不变。
4.1第一个sh文件是train_lenet,sh
#!/usr/bin/env sh
set -e
./build/tools/caffe train --solver=examples/sd_mnist/lenet_solver.prototxt [email protected]
4.2、复制lenet_solver.prototxt文件,并修改:
# The train/test net protocol buffer definition
net: "examples/sd_mnist/lenet_train_test.prototxt"
# test_iter specifies how many forward passes the test should carry out.
# In the case of MNIST, we have test batch size 100 and 100 test iterations,
# covering the full 10,000 testing images.
test_iter: 100
# Carry out testing every 500 training iterations.
test_interval: 500
# The base learning rate, momentum and the weight decay of the network.
base_lr: 0.01
momentum: 0.9
weight_decay: 0.0005
# The learning rate policy
lr_policy: "inv"
gamma: 0.0001
power: 0.75
# Display every 100 iterations
display: 100
# The maximum number of iterations
max_iter: 10000
# snapshot intermediate results
snapshot: 5000
snapshot_prefix: "examples/sd_mnist/lenet"
# solver mode: CPU or GPU
solver_mode: CPU
4.3、lenet_train_test.prototxt复制从mnist文件夹到当前文件夹下
修改路径
name: "LeNet"
layer {
name: "mnist"
type: "Data"
top: "data"
top: "label"
include {
phase: TRAIN
}
transform_param {
scale: 0.00390625
}
data_param {
source: "examples/sd_mnist/mnist_train_lmdb"
batch_size: 64
backend: LMDB
}
}
layer {
name: "mnist"
type: "Data"
top: "data"
top: "label"
include {
phase: TEST
}
transform_param {
scale: 0.00390625
}
data_param {
source: "examples/sd_mnist/mnist_test_lmdb"
batch_size: 100
backend: LMDB
}
}
layer {
name: "conv1"
type: "Convolution"
bottom: "data"
top: "conv1"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 20
kernel_size: 5
stride: 1
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "pool1"
type: "Pooling"
bottom: "conv1"
top: "pool1"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "conv2"
type: "Convolution"
bottom: "pool1"
top: "conv2"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 50
kernel_size: 5
stride: 1
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "pool2"
type: "Pooling"
bottom: "conv2"
top: "pool2"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "ip1"
type: "InnerProduct"
bottom: "pool2"
top: "ip1"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 500
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu1"
type: "ReLU"
bottom: "ip1"
top: "ip1"
}
layer {
name: "ip2"
type: "InnerProduct"
bottom: "ip1"
top: "ip2"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 10
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "accuracy"
type: "Accuracy"
bottom: "ip2"
bottom: "label"
top: "accuracy"
include {
phase: TEST
}
}
layer {
name: "loss"
type: "SoftmaxWithLoss"
bottom: "ip2"
bottom: "label"
top: "loss"
}
4.4 lenet.prototxt复制从mnist文件夹到当前文件夹下,不用修改
4.5 运行 sh example/sd_mnist/train_lenet.sh
没报错,出来accuracy loss这些,说明成功!!
参考:http://blog.csdn.net/xiaoxiao_huitailang/article/details/51361036
以上是关于caffe:用自己的数据训练网络mnist的主要内容,如果未能解决你的问题,请参考以下文章
CAFFE学习笔记Caffe_Example之训练mnist
从零到一:caffe-windows(CPU)配置与利用mnist数据集训练第一个caffemodel