TensorFlow构建卷积神经网络/模型保存与加载/正则化

TensorFlow

官方文档:https://www.tensorflow.org/api_guides/python/math_ops

1# Arithmetic Operators 2import tensorflow as tf 3 4# 用 tf.session.run() 里 feed_dict 参数设置占位 tensor, 如果传入 feed_dict的数据与 tensor 类型不符,就无法被正确处理 5x = tf.placeholder(tf.string) 6y = tf.placeholder(tf.int32) 7z = tf.placeholder(tf.float32) 8 9with tf.Session() as sess: 10 output = sess.run(x, feed_dict={x: 'Test String', y: 123, z: 45.67}) 11 12# 数学运算,类型转换 13tf.subtract(tf.cast(tf.constant(2.0), tf.int32), tf.constant(1)) # 1 14 15# 线性分类函数 16# tf.Variable 类创建一个 tensor,它的初始值可以被改变,就像普通的 Python 变量一样。tensor 把它的状态存在 session里,所以你必须手动初始化它的状态。你用tf.global_variables_initializer() 来初始化所有可变 tensors。 17# tf.global_variables_initializer() 会返回一个操作,它会从graph中初始化所有的 TensorFlow 变量。你可以通过 session 来呼叫这个操作来初始化所有上面的变量 18init = tf.global_variables_initializer() 19with tf.Session() as sess: 20 sess.run(init) 21 22# 从正态分布中选择权重可以避免任意一个权重与其他权重相比有压倒性的特性 23# tf.truncated_normal()函数从一个正态分布中产生随机数, 24# tf.truncated_normal() 返回一个 tensor,它的随机值取自一个正态分布,并且它们的取值会在这个正态分布平均值的两个标准差之内。 25n_features = 120 26n_labels = 5 27weights = tf.Variable(tf.truncated_normal((n_features, n_labels))) 28 29# 因为权重已经被随机化来帮助模型不被卡住,你不需要再把偏差随机化了。让我们简单地把偏差设为 0。 30n_labels = 5 31bias = tf.Variable(tf.zeros(n_labels)) 32# 变量的统一初始化: 33sess.run(tf.global_variables_initializer()) 34 35# 矩阵乘法: 36tf.matmul(input, w) 37 38# TensorFlow Softmax 39# x = tf.nn.softmax([2.0, 1.0, 0.2]) 40output = None 41logit_data = [2.0, 1.0, 0.1] 42logits = tf.placeholder(tf.float32) 43with tf.Session() as sess: 44 output = sess.run(tf.nn.softmax(logits), feed_dict={logits:logit_data}) 45
1# TensorFlow 中的交叉熵(Cross Entropy) 2# tf.reduce_sum() 函数输入一个序列,返回他们的和 3# tf.reduce_mean()计算序列均值 4# tf.log() 返回所输入值的自然对数 5softmax_data = [0.7, 0.2, 0.1] 6one_hot_data = [1.0, 0.0, 0.0] 7 8softmax = tf.placeholder(tf.float32) 9one_hot = tf.placeholder(tf.float32) 10 11cross_entropy = -tf.reduce_sum(tf.multiply(softmax, one_hot)) 12# TODO: Print cross entropy from session 13with tf.Session() as session: 14 output = session.run(cross_entropy, feed_dict={softmax: softmax_data, one_hot: one_hot_data}) 15 print(output) 16
  • TensorFlow Mini-batching

1# 有时候不可能把数据完全分割成相同数量的 batch。例如有 1000 个数据点,你想每个 batch 有 128 个数据。但是1000 无法被 128 整除。你得到的结果是 7 batch,每个128个数据点,一个 batch 有 104个数据点。(7*128 + 1*104 = 1000) 2 3# batch里面的数据点数量会不同的情况下,你需要利用 TensorFlow 的tf.placeholder() 函数来接收这些不同的 batch 4# 如果每个样本有n_input = 784特征,n_classes = 10个可能的标签,features的维度应该是[None, n_input],labels的维度是 [None, n_classes] 5 6# Features and Labels 7# None 维度在这里是一个 batch size 的占位符。在运行时,TensorFlow 会接收任何大于 0 的 batch size 8features = tf.placeholder(tf.float32, [None, n_input]) 9labels = tf.placeholder(tf.float32, [None, n_classes]) 10 11 12
  • 学习率和epochs
  1. 学习率过高,在相同的epochs条件下准确率会过早的停止改进,导致最终准确率会低
  2. 降低学习率需要更多的 epochs,但是可以最终得到更好的准确率
  • 隐藏单元数目的选择
  1. 文章:https://www.quora.com/How-do-I-decide-the-number-of-nodes-in-a-hidden-layer-of-a-neural-network-I-will-be-using-a-three-layer-model
  • The number of hidden nodes you should have is based on a complex relationship between:
  1. Number of input and output nodes
    1. Amount of training data available
    2. Complexity of the function that is trying to be learned
    3. The training algorithm
  2. Too few nodes will lead to high error for your system as the predictive factors might be too complex for a small number of nodes to capture

Too many nodes will overfit to your training data and not generalize well

  1. some general advice: How many hidden units should I use?(详细)

http://www.faqs.org/faqs/ai-faq/neural-nets/part3/section-10.html

  1. The number of hidden nodes in each layer should be somewhere between the size of the input and output layer, potentially the mean.

    1. The number of hidden nodes shouldn't need to exceed twice the number of input nodes, as you are probably grossly overfitting at this point.
  2. 对隐藏层单元个数的选择主要是要满足:能让网络准确预测,使其具有泛化能力,但不会过拟合。

隐藏单元数目一般不超过输入单元的2倍,过高可能引起overfitting, 而且开销很高,但是也不能过小,过小的话会影响到模型的拟合效果

  • 学习率的选择
  1. 对学习速率的选择满足:能使网络成功收敛,且仍然具有较高的时间效率
    1. 学习速率self.lr的经验公式是: self.lr / n_records 趋近于 0.01,可以参考经验公式尝试不同的值,如batch=128, 则选择lr=1左右
    2. 学习速率的优化注意要与其他2个超参数配合调整,特别是迭代次数,单个参数最优不一定最后结果最优,需要3个参数配合调整,以达到最优结果

TensorFlow Examples:
https://github.com/aymericdamien/TensorFlow-Examples

  1. TensorFlow中构建深度神经网络

1# 一个demo 2from tensorflow.examples.tutorials.mnist import input_data 3mnist = input_data.read_data_sets(".", one_hot=True, reshape=False) 4 5import tensorflow as tf 6 7# 参数 Parameters 8learning_rate = 0.001 9training_epochs = 20 10batch_size = 128 # 如果没有足够内存,可以降低 batch size 11display_step = 1 12 13n_input = 784 # MNIST data input (img shape: 28*28) 14n_classes = 10 # MNIST total classes (0-9 digits) 15 16n_hidden_layer = 256 # layer number of features 特征的层数 17 18# Store layers weight & bias 19# 层权重和偏置项的储存 20weights = { 21 'hidden_layer': tf.Variable(tf.random_normal([n_input, n_hidden_layer])), 22 'out': tf.Variable(tf.random_normal([n_hidden_layer, n_classes])) 23} 24biases = { 25 'hidden_layer': tf.Variable(tf.random_normal([n_hidden_layer])), 26 'out': tf.Variable(tf.random_normal([n_classes])) 27} 28 29# tf Graph input 30x = tf.placeholder("float", [None, 28, 28, 1]) 31y = tf.placeholder("float", [None, n_classes]) 32 33x_flat = tf.reshape(x, [-1, n_input]) 34 35# Hidden layer with RELU activation 36# ReLU作为隐藏层激活函数 37layer_1 = tf.add(tf.matmul(x_flat, weights['hidden_layer']),\ 38 biases['hidden_layer']) 39layer_1 = tf.nn.relu(layer_1) 40# Output layer with linear activation 41# 输出层的线性激活函数 42logits = tf.add(tf.matmul(layer_1, weights['out']), biases['out']) 43 44# Define loss and optimizer 45# 定义误差值和优化器 46cost = tf.reduce_mean(\ 47 tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y)) 48optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)\ 49 .minimize(cost) 50 51# Initializing the variables 52# 初始化变量 53init = tf.global_variables_initializer() 54 55# Launch the graph 56# 启动图 57with tf.Session() as sess: 58 sess.run(init) 59 # Training cycle 60 # 训练循环 61 for epoch in range(training_epochs): 62 total_batch = int(mnist.train.num_examples/batch_size) 63 # Loop over all batches 64 # 遍历所有 batch 65 for i in range(total_batch): 66 batch_x, batch_y = mnist.train.next_batch(batch_size) 67 # Run optimization op (backprop) and cost op (to get loss value) 68 # 运行优化器进行反向传导、计算 cost(获取 loss 值) 69 sess.run(optimizer, feed_dict={x: batch_x, y: batch_y}) 70 71# 隐藏层单元数:隐藏层宽度 72# 隐藏层层数:隐藏层深度 73 74

保存和读取 TensorFlow 模型

  • 保存变量

1# 保存变量 2# weights 和 bias Tensors 用 tf.truncated_normal() 函数设定了随机值。用 tf.train.Saver.save() 函数把这些值被保存在save_file 位置,命名为 "model.ckpt",(".ckpt" 扩展名表示"checkpoint")。 3 4import tensorflow as tf 5 6# The file path to save the data 7# 文件保存路径 8save_file = './model.ckpt' 9 10# Two Tensor Variables: weights and bias 11# 两个 Tensor 变量:权重和偏置项 12weights = tf.Variable(tf.truncated_normal([2, 3])) 13bias = tf.Variable(tf.truncated_normal([3])) 14 15# Class used to save and/or restore Tensor Variables 16# 用来存取 Tensor 变量的类 17saver = tf.train.Saver() 18 19with tf.Session() as sess: 20 # Initialize all the Variables 21 # 初始化所有变量 22 sess.run(tf.global_variables_initializer()) 23 24 # Show the values of weights and bias 25 # 显示变量和权重 26 print('Weights:') 27 print(sess.run(weights)) 28 print('Bias:') 29 print(sess.run(bias)) 30 31 # Save the model 32 # 保存模型 33 saver.save(sess, save_file) 34
  • 加载变量

1# 加载变量 2# 注意,你依然需要在 Python 中创建 weights 和 bias Tensors。tf.train.Saver.restore() 函数把之前保存的数据加载到 weights 和 bias 当中。 3# 因为 tf.train.Saver.restore() 设定了 TensorFlow 变量,这里你不需要调用 tf.global_variables_initializer()了。 4# Remove the previous weights and bias 5# 移除之前的权重和偏置项 6tf.reset_default_graph() 7 8# Two Variables: weights and bias 9# 两个变量:权重和偏置项 10weights = tf.Variable(tf.truncated_normal([2, 3])) 11bias = tf.Variable(tf.truncated_normal([3])) 12 13# Class used to save and/or restore Tensor Variables 14# 用来存取 Tensor 变量的类 15saver = tf.train.Saver() 16 17with tf.Session() as sess: 18 # Load the weights and bias 19 # 加载权重和偏置项 20 saver.restore(sess, save_file) 21 22 # Show the values of weights and bias 23 # 显示权重和偏置项 24 print('Weight:') 25 print(sess.run(weights)) 26 print('Bias:') 27 print(sess.run(bias)) 28
  • 保存一个训练好的模型

1# 保存一个训练好的模型 2# Remove previous Tensors and Operations 3# 移除之前的 Tensors 和运算 4tf.reset_default_graph() 5 6from tensorflow.examples.tutorials.mnist import input_data 7import numpy as np 8 9learning_rate = 0.001 10n_input = 784 # MNIST 数据输入 (图片尺寸: 28*28) 11n_classes = 10 # MNIST 总计类别 (数字 0-9) 12 13# Import MNIST data 14# 加载 MNIST 数据 15mnist = input_data.read_data_sets('.', one_hot=True) 16 17# Features and Labels 18# 特征和标签 19features = tf.placeholder(tf.float32, [None, n_input]) 20labels = tf.placeholder(tf.float32, [None, n_classes]) 21 22# Weights & bias 23# 权重和偏置项 24weights = tf.Variable(tf.random_normal([n_input, n_classes])) 25bias = tf.Variable(tf.random_normal([n_classes])) 26 27# Logits - xW + b 28logits = tf.add(tf.matmul(features, weights), bias) 29 30# Define loss and optimizer 31# 定义损失函数和优化器 32cost = tf.reduce_mean(\ 33 tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=labels)) 34optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)\ 35 .minimize(cost) 36 37# Calculate accuracy 38# 计算准确率 39correct_prediction = tf.equal(tf.argmax(logits, 1), tf.argmax(labels, 1)) 40accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32)) 41 42# 训练模型并保存权重: 43save_file = './train_model.ckpt' 44batch_size = 128 45n_epochs = 100 46 47saver = tf.train.Saver() 48 49# Launch the graph 50# 启动图 51with tf.Session() as sess: 52 sess.run(tf.global_variables_initializer()) 53 54 # Training cycle 55 # 训练循环 56 for epoch in range(n_epochs): 57 total_batch = math.ceil(mnist.train.num_examples / batch_size) 58 59 # Loop over all batches 60 # 遍历所有 batch 61 for i in range(total_batch): 62 batch_features, batch_labels = mnist.train.next_batch(batch_size) 63 sess.run( 64 optimizer, 65 feed_dict={features: batch_features, labels: batch_labels}) 66 67 # Print status for every 10 epochs 68 # 每运行10个 epoch 打印一次状态 69 if epoch % 10 == 0: 70 valid_accuracy = sess.run( 71 accuracy, 72 feed_dict={ 73 features: mnist.validation.images, 74 labels: mnist.validation.labels}) 75 print('Epoch {:<3} - Validation Accuracy: {}'.format( 76 epoch, 77 valid_accuracy)) 78 79 # Save the model 80 # 保存模型 81 saver.save(sess, save_file) 82 print('Trained Model Saved.') 83
  • 加载训练好的模型

1# 加载训练好的模型 2saver = tf.train.Saver() 3 4# Launch the graph 5# 加载图 6with tf.Session() as sess: 7 saver.restore(sess, save_file) 8 9 test_accuracy = sess.run( 10 accuracy, 11 feed_dict={features: mnist.test.images, labels: mnist.test.labels}) 12 13print('Test Accuracy: {}'.format(test_accuracy)) 14 15
  • 把权重和偏置项加载到新模型中

1# 很多时候你想调整,或者说“微调”一个你已经训练并保存了的模型。但是,把保存的变量直接加载到已经修改过的模型会产生错误 2# TensorFlow 对 Tensor 和计算使用一个叫 name 的字符串辨识器,如果没有定义 name,TensorFlow 会自动创建一个.容易出现命名错误 3 4# 手动设定name属性: 5import tensorflow as tf 6 7tf.reset_default_graph() 8 9save_file = 'model.ckpt' 10 11# Two Tensor Variables: weights and bias 12# 两个 Tensor 变量:权重和偏置项 13weights = tf.Variable(tf.truncated_normal([2, 3]), name='weights_0') 14bias = tf.Variable(tf.truncated_normal([3]), name='bias_0') 15 16saver = tf.train.Saver() 17 18# Print the name of Weights and Bias 19# 打印权重和偏置项的名称 20print('Save Weights: {}'.format(weights.name)) 21print('Save Bias: {}'.format(bias.name)) 22 23with tf.Session() as sess: 24 sess.run(tf.global_variables_initializer()) 25 saver.save(sess, save_file) 26 27# Remove the previous weights and bias 28# 移除之前的权重和偏置项 29tf.reset_default_graph() 30 31# Two Variables: weights and bias 32# 两个变量:权重和偏置项 33bias = tf.Variable(tf.truncated_normal([3]), name='bias_0') 34weights = tf.Variable(tf.truncated_normal([2, 3]) ,name='weights_0') 35 36saver = tf.train.Saver() 37 38# Print the name of Weights and Bias 39# 打印权重和偏置项的名称 40print('Load Weights: {}'.format(weights.name)) 41print('Load Bias: {}'.format(bias.name)) 42 43with tf.Session() as sess: 44 # Load the weights and bias - No Error 45 # 加载权重和偏置项 - 没有报错 46 saver.restore(sess, save_file) 47 48print('Loaded Weights and Bias successfully.') 49

Save Weights: weights_0:0

Save Bias: bias_0:0

Load Weights: weights_0:0

Load Bias: bias_0:0

Loaded Weights and Bias successfully.

  • TensorFlow Dropout

1# Dropout 是一个降低过拟合的正则化技术。它在网络中暂时的丢弃一些单元(神经元),以及与它们的前后相连的所有节点 2# TensorFlow 提供了一个 tf.nn.dropout() 函数,你可以用来实现 dropout 3 4keep_prob = tf.placeholder(tf.float32) # probability to keep units 5 6hidden_layer = tf.add(tf.matmul(features, weights[0]), biases[0]) 7hidden_layer = tf.nn.relu(hidden_layer) 8hidden_layer = tf.nn.dropout(hidden_layer, keep_prob) 9 10logits = tf.add(tf.matmul(hidden_layer, weights[1]), biases[1]) 11 12# tf.nn.dropout()函数有两个参数: 13# hidden_layer:你要应用 dropout 的 tensor 14# keep_prob:任何一个给定单元的留存率(没有被丢弃的单元) 15 16# keep_prob 可以让你调整丢弃单元的数量。为了补偿被丢弃的单元,tf.nn.dropout() 把所有保留下来的单元(没有被丢弃的单元)* 1/keep_prob 17# 在训练时,一个好的keep_prob初始值是0.5。 18 19# 在测试时,把 keep_prob 值设为1.0 ,这样保留所有的单元,最大化模型的能力 20

一个demo:

1keep_prob = tf.placeholder(tf.float32) # probability to keep units 2 3hidden_layer = tf.add(tf.matmul(features, weights[0]), biases[0]) 4hidden_layer = tf.nn.relu(hidden_layer) 5hidden_layer = tf.nn.dropout(hidden_layer, keep_prob) 6 7logits = tf.add(tf.matmul(hidden_layer, weights[1]), biases[1]) 8 9... 10 11with tf.Session() as sess: 12 sess.run(tf.global_variables_initializer()) 13 14 for epoch_i in range(epochs): 15 for batch_i in range(batches): 16 .... 17 18 sess.run(optimizer, feed_dict={ 19 features: batch_features, 20 labels: batch_labels, 21 keep_prob: 0.5}) 22 23 validation_accuracy = sess.run(accuracy, feed_dict={ 24 features: test_features, 25 labels: test_labels, 26 keep_prob: 1.0}) 27
  • TensorFlow卷积神经网络

设置
H = height, W = width, D = depth

我们有一个输入维度是 32x32x3 (HxWxD)
20个维度为 8x8x3 (HxWxD) 的滤波器
高和宽的stride(步长)都为 2。(S)
padding 大小为1 (P)
计算新的高度和宽度的公式是:

new_height = (input_height - filter_height + 2 * P)/S + 1

new_width = (input_width - filter_width + 2 * P)/S + 1

输出层的大小: 14x14x20

1input = tf.placeholder(tf.float32, (None, 32, 32, 3)) 2filter_weights = tf.Variable(tf.truncated_normal((8, 8, 3, 20))) # (height, width, input_depth, output_depth) 3filter_bias = tf.Variable(tf.zeros(20)) 4strides = [1, 2, 2, 1] # (batch, height, width, depth) 5padding = 'VALID' 6conv = tf.nn.conv2d(input, filter_weights, strides, padding) + filter_bias 7 8 9# TensorFlow 使用如下等式计算 SAME 、PADDING 10 11# SAME Padding, 输出的高和宽,计算如下: 12 13out_height = ceil(float(in_height) / float(strides1)) 14 15out_width = ceil(float(in_width) / float(strides[2])) 16 17# VALID Padding, 输出的高和宽,计算如下: 18 19out_height = ceil(float(in_height - filter_height + 1) / float(strides1)) 20 21out_width = ceil(float(in_width - filter_width + 1) / float(strides[2])) 22 23# ceil:返回大于或者等于指定表达式的最小整数 24
  • TensorFlow 卷积层实现

1# TensorFlow 提供了 tf.nn.conv2d() 和 tf.nn.bias_add() 函数来创建你自己的卷积层 2 3# Output depth 4k_output = 64 5 6# Image Properties 7image_width = 10 8image_height = 10 9color_channels = 3 10 11# Convolution filter 12filter_size_width = 5 13filter_size_height = 5 14 15# Input/Image 16input = tf.placeholder( 17 tf.float32, 18 shape=[None, image_height, image_width, color_channels]) 19 20# Weight and bias 21weight = tf.Variable(tf.truncated_normal( 22 [filter_size_height, filter_size_width, color_channels, k_output])) 23bias = tf.Variable(tf.zeros(k_output)) 24 25# Apply Convolution 26conv_layer = tf.nn.conv2d(input, weight, strides=[1, 2, 2, 1], padding='SAME') 27# Add bias 28conv_layer = tf.nn.bias_add(conv_layer, bias) 29# Apply activation function 30conv_layer = tf.nn.relu(conv_layer) 31 32# 上述代码用了 tf.nn.conv2d() 函数来计算卷积,weights 作为滤波器,[1, 2, 2, 1] 作为 strides。 33# TensorFlow 对每一个 input 维度使用一个单独的 stride 参数,[batch, input_height, input_width, input_channels]。我们通常把 batch 和 input_channels (strides 序列中的第一个第四个)的 stride 设为 1 34# input_height 和 input_width strides 表示滤波器在input 上移动的步长 35 36# tf.nn.bias_add() 函数对矩阵的最后一维加了偏置项 37
  • TensorFlow max pooling 最大池化

Decrease the size of the output and prevent overfitting. Reducing overfitting is a consequence of the reducing the output size, which in turn, reduces the number of parameters in future layers
池化作用:减小输出大小 和 降低过拟合。降低过拟合是减小输出大小的结果,它同样也减少了后续层中的参数的数量。

近期,池化层并不是很受青睐。部分原因是:

现在的数据集又大又复杂,我们更关心欠拟合问题。

Dropout 是一个更好的正则化方法。
池化导致信息损失。想想最大池化的例子,n 个数字中我们只保留最大的,把余下的 n-1 完全舍弃了。

1# TensorFlow 提供了 tf.nn.max_pool() 函数,用于对卷积层实现 最大池化 2 3# tf.nn.max_pool() 函数实现最大池化时, ksize参数是滤波器大小,strides参数是步长。2x2 的滤波器配合 2x2 的步长是常用设定。 4 5# ksize 和 strides 参数也被构建为四个元素的列表,每个元素对应 input tensor 的一个维度 ([batch, height, width, channels]),对 ksize 和 strides 来说,batch 和 channel 通常都设置成 1。 6 7# 注意:池化层的输出深度与输入的深度相同。另外池化操作是分别应用到每一个深度切片层 8 9conv_layer = tf.nn.conv2d(input, weight, strides=[1, 2, 2, 1], padding='SAME') 10conv_layer = tf.nn.bias_add(conv_layer, bias) 11conv_layer = tf.nn.relu(conv_layer) 12# Apply Max Pooling 13conv_layer = tf.nn.max_pool( 14 conv_layer, 15 ksize=[1, 2, 2, 1], 16 strides=[1, 2, 2, 1], 17 padding='SAME') 18 19

一个minist数据集上的三层卷积神经网络demo:

1# 数据集 2from tensorflow.examples.tutorials.mnist import input_data 3mnist = input_data.read_data_sets(".", one_hot=True, reshape=False) 4 5import tensorflow as tf 6 7# Parameters 8# 参数 9learning_rate = 0.00001 10epochs = 10 11batch_size = 128 12 13# Number of samples to calculate validation and accuracy 14# Decrease this if you're running out of memory to calculate accuracy 15# 用来验证和计算准确率的样本数 16# 如果内存不够,可以调小这个数字 17test_valid_size = 256 18 19##################################################### 20# Network Parameters 21# 神经网络参数 22n_classes = 10 # MNIST total classes (0-9 digits) 23dropout = 0.75 # Dropout, probability to keep units 24 25# weights and biases 26# Store layers weight & bias 27weights = { 28 'wc1': tf.Variable(tf.random_normal([5, 5, 1, 32])), 29 'wc2': tf.Variable(tf.random_normal([5, 5, 32, 64])), 30 'wd1': tf.Variable(tf.random_normal([7*7*64, 1024])), 31 'out': tf.Variable(tf.random_normal([1024, n_classes]))} 32 33biases = { 34 'bc1': tf.Variable(tf.random_normal([32])), 35 'bc2': tf.Variable(tf.random_normal([64])), 36 'bd1': tf.Variable(tf.random_normal([1024])), 37 'out': tf.Variable(tf.random_normal([n_classes]))} 38 39##################################################### 40# 卷积 41def conv2d(x, W, b, strides=1): 42 x = tf.nn.conv2d(x, W, strides=[1, strides, strides, 1], padding='SAME') 43 x = tf.nn.bias_add(x, b) 44 return tf.nn.relu(x) 45# 最大池化 46def maxpool2d(x, k=2): 47 return tf.nn.max_pool( 48 x, 49 ksize=[1, k, k, 1], 50 strides=[1, k, k, 1], 51 padding='SAME') 52 53##################################################### 54# 模型 55# 创建了 3 层来实现卷积,最大池化以及全链接层和输出层 56def conv_net(x, weights, biases, dropout): 57 # Layer 1 - 28*28*1 to 14*14*32 58 conv1 = conv2d(x, weights['wc1'], biases['bc1']) 59 conv1 = maxpool2d(conv1, k=2) 60 61 # Layer 2 - 14*14*32 to 7*7*64 62 conv2 = conv2d(conv1, weights['wc2'], biases['bc2']) 63 conv2 = maxpool2d(conv2, k=2) 64 65 # Fully connected layer - 7*7*64 to 1024 66 fc1 = tf.reshape(conv2, [-1, weights['wd1'].get_shape().as_list()[0]]) 67 fc1 = tf.add(tf.matmul(fc1, weights['wd1']), biases['bd1']) 68 fc1 = tf.nn.relu(fc1) 69 fc1 = tf.nn.dropout(fc1, dropout) 70 71 # Output Layer - class prediction - 1024 to 10 72 out = tf.add(tf.matmul(fc1, weights['out']), biases['out']) 73 return out 74##################################################### 75# Session 76# tf Graph input 77x = tf.placeholder(tf.float32, [None, 28, 28, 1]) 78y = tf.placeholder(tf.float32, [None, n_classes]) 79keep_prob = tf.placeholder(tf.float32) 80 81# Model 82logits = conv_net(x, weights, biases, keep_prob) 83 84# Define loss and optimizer 85cost = tf.reduce_mean(\ 86 tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y)) 87optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)\ 88 .minimize(cost) 89 90# Accuracy 91correct_pred = tf.equal(tf.argmax(logits, 1), tf.argmax(y, 1)) 92accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32)) 93 94# Initializing the variables 95init = tf. global_variables_initializer() 96 97# Launch the graph 98with tf.Session() as sess: 99 sess.run(init) 100 101 for epoch in range(epochs): 102 for batch in range(mnist.train.num_examples//batch_size): 103 batch_x, batch_y = mnist.train.next_batch(batch_size) 104 sess.run(optimizer, feed_dict={ 105 x: batch_x, 106 y: batch_y, 107 keep_prob: dropout}) 108 109 # Calculate batch loss and accuracy 110 loss = sess.run(cost, feed_dict={ 111 x: batch_x, 112 y: batch_y, 113 keep_prob: 1.}) 114 valid_acc = sess.run(accuracy, feed_dict={ 115 x: mnist.validation.images[:test_valid_size], 116 y: mnist.validation.labels[:test_valid_size], 117 keep_prob: 1.}) 118 119 print('Epoch {:>2}, Batch {:>3} -' 120 'Loss: {:>10.4f} Validation Accuracy: {:.6f}'.format( 121 epoch + 1, 122 batch + 1, 123 loss, 124 valid_acc)) 125 126 # Calculate Test Accuracy 127 test_acc = sess.run(accuracy, feed_dict={ 128 x: mnist.test.images[:test_valid_size], 129 y: mnist.test.labels[:test_valid_size], 130 keep_prob: 1.}) 131 print('Testing Accuracy: {}'.format(test_acc)) 132 133 134
  • tensorflow中关于CNN的使用和介绍

https://www.tensorflow.org/api_guides/python/nn\#Convolution

转载于:https://www.cnblogs.com/songdanzju/p/7441732.html

代码交流 2021