TensorFlow 2.0 中的神经网络问题
Posted
技术标签:
【中文标题】TensorFlow 2.0 中的神经网络问题【英文标题】:Problem with neural network in TensorFlow 2.0 【发布时间】:2020-06-01 10:10:18 【问题描述】:import tensorflow as tf
import pandas as pd
import numpy as np
import matplotlib as plt
from sklearn.model_selection import train_test_split
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from sklearn.preprocessing import StandardScaler
import functools
LABEL_COLUMN = 'Endstage'
LABELS = [1, 2, 3, 4]
x = pd.read_csv('HCVnew.csv', index_col=False)
def get_dataset(file_path, **kwargs):
dataset = tf.data.experimental.make_csv_dataset(
file_path,
batch_size=35, # Artificially small to make examples easier to show.
label_name=LABEL_COLUMN,
na_value="?",
num_epochs=1,
ignore_errors=True,
**kwargs)
return dataset
SELECT_COLUMNS = ["Alter", "Gender", "BMI", "Fever", "Nausea", "Fatigue",
"WBC", "RBC", "HGB", "Plat", "AST1", "ALT1", "ALT4", "ALT12", "ALT24", "ALT36", "ALT48", "ALT24w",
"RNABase", "RNA4", "Baseline", "Endstage"]
DEFAULTS = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
temp_dataset = get_dataset("HCVnew.csv",
select_columns=SELECT_COLUMNS,
column_defaults=DEFAULTS)
def pack(features, label):
return tf.stack(list(features.values()), axis=-1), label
packed_dataset = temp_dataset.map(pack)
"""
for features, labels in packed_dataset.take(1):
print(features.numpy())
print()
print(labels.numpy())
"""
NUMERIC_FEATURES = ["Alter", "Gender","BMI", "Fever", "Nausea", "Fatigue",
"WBC", "RBC", "HGB", "Plat", "AST1", "ALT1", "ALT4", "ALT12", "ALT24", "ALT36", "ALT48", "ALT24w",
"RNABase", "RNA4", "Baseline", "Endstage"]
desc = pd.read_csv("HCVnew.csv")[NUMERIC_FEATURES].describe()
MEAN = np.array(desc.T['mean'])
STD = np.array(desc.T['std'])
def normalize_numeric_data(data, mean, std):
# Center the data
return (data-mean)/std
# See what you just created.
raw_train_data = get_dataset("HCVnew.csv")
raw_test_data = get_dataset("HCVnew.csv")
class PackNumericFeatures(object):
def __init__(self, names):
self.names = names
def __call__(self, features, labels):
numeric_freatures = [features.pop(name) for name in self.names]
numeric_features = [tf.cast(feat, tf.float32) for feat in numeric_freatures]
numeric_features = tf.stack(numeric_features, axis=-1)
features['numeric'] = numeric_features
return features, labels
NUMERIC_FEATURES = ["Alter", "Gender","BMI", "Fever", "Nausea", "Fatigue",
"WBC", "RBC", "HGB", "Plat", "AST1", "ALT1", "ALT4", "ALT12", "ALT24", "ALT36", "ALT48", "ALT24w",
"RNABase", "RNA4", "Baseline", "Endstage"]
packed_train_data = raw_train_data.map(
PackNumericFeatures(NUMERIC_FEATURES))
packed_test_data = raw_test_data.map(
PackNumericFeatures(NUMERIC_FEATURES))
normalizer = functools.partial(normalize_numeric_data, mean=MEAN, std=STD)
numeric_column = tf.feature_column.numeric_column('numeric', normalizer_fn=normalizer, shape=[len(NUMERIC_FEATURES)])
numeric_columns = [numeric_column]
numeric_layer = tf.keras.layers.DenseFeatures(numeric_columns)
preprocessing_layer = tf.keras.layers.DenseFeatures(numeric_columns)
#———————————————————————MODEL———————————————————————————————————————————————————————————————————————————————————————————
model = tf.keras.Sequential([
preprocessing_layer,
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dense(1, activation='sigmoid'),
])
model.compile(
loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
data_x = get_dataset("HCVnew.csv")
train_data = data_x.shuffle(500)
model.fit(train_data, epochs=20)
您好,我正在尝试构建一个可以根据包含患者信息的 csv 文件预测丙型肝炎的神经网络,但我无法修复错误... 我收到错误:KeyError 'Endstage',而 Endstage 是包含相应值(1 到 4 之间)并用作标签列的 csv 列。 如果有人有可以解决我问题的想法,请告诉我。 非常感谢您的帮助!
【问题讨论】:
检查Endstage
的列名是否正确
确实是正确的列名
【参考方案1】:
那是因为Endstage
是您的标签列,并且框架通过将其从数据集中删除(弹出)来帮助您。否则,您的训练数据集也将包含目标类,使其无用。
将其从 NUMERIC_FEATURES
和任何其他将其纳入您的训练集特征的地方删除。
[编辑]
OP 在后续问题(在 cmets 中)中问为什么,在解决了最初的问题后,他得到了一个错误:
ValueError: 特征数字不在特征字典中
看起来,名为numeric
的功能是通过调用PackNumericFeatures
生成的。后者用于创建packed_train_data
和packed_test_data
,但这些从未使用过。然而这一行:
numeric_column = tf.feature_column.numeric_column('numeric', normalizer_fn=normalizer, shape=[len(NUMERIC_FEATURES)])
假设数据在那里 - 因此错误。
【讨论】:
非常感谢!但现在我收到错误消息:ValueError: Feature numeric is not in features dictionary。抱歉问了这么多…… @MountGreen 我很高兴它成功了。我自然明白你为什么要问后续问题(你想让它运行),但这是一个不同的问题。我将用它来编辑我的答案,但我们不要过多地陷入兔子洞。否则任何一个问题都可以变成一个整体解决方案和持续发展:)。我建议接受答案(假设我回答了最初的问题),然后发布后续问题。 是的,我完全理解您的感叹,但正如您已经说过的,我只想让它工作:D ...非常感谢您的大力帮助和快速响应!虽然还不行,但我会努力的…… @MountGreen 这是一段旅程 :)。当您最终让模型进行训练时,这并不意味着它确实有效(例如:是一个有用的模型,可以推广到现实世界)。祝你好运!以上是关于TensorFlow 2.0 中的神经网络问题的主要内容,如果未能解决你的问题,请参考以下文章
Tensorflow 2.0|网络优化与参数选择及Dropout抑制过拟合原则
深度学习与TensorFlow 2.0卷积神经网络(CNN)
TensorFlow2.0TensorFlow 2.0高阶API: Keras—使用Keras基于Squential的序列编排模式创建神经网络过程(附带源码)