X 和 Y 的 SKlearn 重塑警告

Posted 2023-03-12

【中文标题】X 和 Y 的 SKlearn 重塑警告

【英文标题】：SKlearn reshape warning for X and Y

【发布时间】：2018-09-12 10:09:16

【问题描述】：

我是机器学习的新手，正在使用 python(3.6)、pandas、Numpy 和 SKLearn 进行项目。

我的数据框是：

discount   tax   total   subtotal   productid
  3         0     20       13        002
  10        3     106      94        003
  46.49     6     21       20        004

这是我执行分类的方式：

df_full = pd.read_excel('input/Potential_Learning_Patterns.xlsx', sheet_name=0)
df_full.head()
#for convert to numeric
df_full['discount'] = pd.to_numeric(df_full['discount'], errors='coerce')
df_full['productdiscount'] = pd.to_numeric(df_full['discount'], errors='coerce')
df_full['Class'] = ((df_full['discount'] > 20) & 
                (df_full['tax'] == 0) &
                (df_full['productdiscount'] > 20) &
                (df_full['total'] > 100)).astype(int)
print (df_full)

# Get some sample data from entire dataset
data = df_full.sample(frac = 0.1, random_state = 1)
print(data.shape)
data.isnull().sum()
# Convert excel data into matrix
columns = "invoiceid locationid timestamp customerid discount tax total subtotal productid quantity productprice productdiscount invoice_products_id producttax invoice_payments_id paymentmethod paymentdetails amount Class(0/1) Class".split()
X = pd.DataFrame.as_matrix(data, columns=columns)
Y = data.Class
# temp = np.array(temp).reshape((len(temp), 1)
Y = Y.values.reshape(Y.shape[0], 1)
X.shape
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.06)
X_test, X_dev, Y_test, Y_dev = train_test_split(X_test, Y_test, test_size = .5)

# Check if there is Classification Values - 0/1 in training set and other set 
np.where(Y_train == 1)
np.where(Y_test == 1)
np.where(Y_dev == 1)

# Determine no of fraud cases in dataset
Fraud = data[data['Class'] == 1]
Valid = data[data['Class'] == 0]

# calculate percentages for Fraud & Valid 
outlier_fraction = len(Fraud) / float(len(Valid))
print(outlier_fraction)

print('Fraud Cases : '.format(len(Fraud)))
print('Valid Cases : '.format(len(Valid)))

# Correlation matrix
corrmat = data.corr()
fig = plt.figure( figsize = (12, 9))

sns.heatmap(corrmat, vmax = .8, square = True)
plt.show()

以下是我应用重塑的方式：

# Get all the columns from dataframe
columns = data.columns.tolist()

# Filter the columns to remove data we don't want
columns = [c for c in columns if c not in ["Class"] ]

# store the variables we want to predicting on
target = "Class"
for column in data.columns:
    if data[column].dtype == type(object):
        le = LabelEncoder()
        data[column] = le.fit_transform(data[column])
        X = data[column]
    X = data[column]        
    Y = data[target]

    # Print the shapes of X & Y
    print(X.shape)
    print(Y.shape)
    # define a random state
state = 1

# define the outlier detection method
classifiers = 
    "Isolation Forest": IsolationForest(max_samples=len(X),
                                       contamination=outlier_fraction,
                                       random_state=state),
    "Local Outlier Factor": LocalOutlierFactor(
    n_neighbors = 20,
    contamination = outlier_fraction)




# fit the model
n_outliers = len(Fraud)

for i, (clf_name, clf) in enumerate(classifiers.items()):

    # fit te data and tag outliers
    if clf_name == "Local Outlier Factor":
        y_pred = clf.fit_predict(X)
        scores_pred = clf.negative_outlier_factor_
    else:
        clf.fit(X)
        scores_pred = clf.decision_function(X)
        y_pred = clf.predict(X)

    # Reshape the prediction values to 0 for valid and 1 for fraudulent
    y_pred[y_pred == 1] = 0
    y_pred[y_pred == -1] = 1

    n_errors = (y_pred != Y).sum()

    # run classification metrics 
    print(':').format(clf_name, n_errors)
    print(accuracy_score(Y, y_pred ))
    print(classification_report(Y, y_pred ))

在重塑样本和目标之前，代码可以正常工作。但是当我为我的分类器尝试 fit 方法时，它会返回如下错误：

ValueError: Expected 2D array, got 1D array instead: 数组=[1 0]。 如果您的数据具有单个特征，则使用 array.reshape(-1, 1) 重塑您的数据，如果数据包含单个样本，则使用 array.reshape(1, -1)。

我是机器学习的新手，我在这里做错了什么？我有多种功能如何正确重塑我的示例数组？

请帮帮我！ 提前致谢！

【问题讨论】：

print(X.shape) 的输出是什么？

X.shape 的输出是 (2,)

【参考方案1】：

在以下循环中，您将在每次循环迭代中用单列（系列）覆盖变量 X：

for column in data.columns:
    if data[column].dtype == type(object):
        le = LabelEncoder()
        data[column] = le.fit_transform(data[column])
        X = data[column]
    X = data[column]        #  <------- NOTE: 
    Y = data[target]

其实你可以在循环之后定义X和Y如下：

X = data.drop(target, 1)
Y = data[target]

绝大多数sklrean 方法都接受pandas DataFrames 和Series 作为输入数据集...

【讨论】：

现在它在底部第三行返回另一个错误AttributeError: 'NoneType' object has no attribute 'format'。

@AbdulRehman，首先这是另一个错误，与您提出的问题没有任何关系。替换：print(':').format(clf_name, n_errors) --> print(':'.format(clf_name, n_errors))

嗨@MaxU，最后一件事，如果你能帮忙的话，拜托！它在最后一行引发另一个错误，警告为/Users/abdul/anaconda3/lib/python3.6/site-packages/sklearn/metrics/classification.py:1135: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. 'precision', 'predicted', average, warn_for) 和错误ValueError: contamination must be in (0, 0.5]

@AbdulRehman，您可能想查看this question

嗨@MaxU，你能看看这个问题：***.com/q/49839270/7644562好吗？