运行模型后,如何将隔离林和局部离群因子保存为两个不同的模型?

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了运行模型后,如何将隔离林和局部离群因子保存为两个不同的模型?相关的知识,希望对你有一定的参考价值。

我一直在尝试编写一种机器学习程序,以使用sklearnpandas中的隔离林和局部异常值方法检测信用卡欺诈。

我有运行代码并做出预测的代码,但我不知道如何将它们保存为不同的模型。我一直在跟踪一些示例,但不知道在哪里以及如何保存它。我认为它类似于.save('Isolation.h5').save('Outlier.h5'),但我不确定要在.save前面加上什么。

如果有人可以帮助我了解如何保存每个模型,将不胜感激。

我当前的代码:

import numpy
import pandas
import matplotlib
import seaborn
import scipy

# import the necessary packages
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

# Load the dataset from the csv file using pandas
data = pd.read_csv('C:/Users/super/OneDrive/Documents/School/Spring 2020/CS 657/Final Project/creditcard.csv')

# Start exploring the dataset
print(data.columns)

data = data.sample(frac=0.1, random_state = 1)
print(data.shape)
print(data.describe())

# V1 - V28 are the results of a PCA Dimensionality reduction to protect user identities and sensitive features

# Plot histograms of each parameter 
data.hist(figsize = (20, 20))
plt.show()

# Determine number of fraud cases in dataset

Fraud = data[data['Class'] == 1]
Valid = data[data['Class'] == 0]

outlier_fraction = len(Fraud)/float(len(Valid))
print(outlier_fraction)

print('Fraud Cases: '.format(len(data[data['Class'] == 1])))
print('Valid Transactions: '.format(len(data[data['Class'] == 0])))

# Correlation matrix
corrmat = data.corr()
fig = plt.figure(figsize = (12, 9))

sns.heatmap(corrmat, vmax = .8, square = True)
plt.show()

# Get all the columns from the dataFrame
columns = data.columns.tolist()

# Filter the columns to remove data we do not want
columns = [c for c in columns if c not in ["Class"]]

# Store the variable we'll be predicting on
target = "Class"

X = data[columns]
Y = data[target]

# Print shapes
print(X.shape)
print(Y.shape)

from sklearn.metrics import classification_report, accuracy_score
from sklearn.ensemble import IsolationForest
from sklearn.neighbors import LocalOutlierFactor

# define random states
state = 1

# define outlier detection tools to be compared
classifiers = 
    "Isolation Forest": IsolationForest(max_samples=len(X),
                                        contamination=outlier_fraction,
                                        random_state=state),
    "Local Outlier Factor": LocalOutlierFactor(
        n_neighbors=20,
        contamination=outlier_fraction)

# Fit the model
plt.figure(figsize=(9, 7))
n_outliers = len(Fraud)


for i, (clf_name, clf) in enumerate(classifiers.items()):

    # fit the data and tag outliers
    if clf_name == "Local Outlier Factor":
        y_pred = clf.fit_predict(X)
        scores_pred = clf.negative_outlier_factor_
    else:
        clf.fit(X)
        scores_pred = clf.decision_function(X)
        y_pred = clf.predict(X)

    # Reshape the prediction values to 0 for valid, 1 for fraud. 
    y_pred[y_pred == 1] = 0
    y_pred[y_pred == -1] = 1

    n_errors = (y_pred != Y).sum()

    # Run classification metrics
    print(': '.format(clf_name, n_errors))
    print(accuracy_score(Y, y_pred))
    print(classification_report(Y, y_pred))



答案

由于您遍历了所有分类器并对其进行了训练/做出了预测,因此您可以简单地同时保存模型。

例如,使用pickle

import pickle

def save_model(clf, filename):
    with open(filename, 'wb') as f:
        pickle.dump(clf, f)

for i, (clf_name, clf) in enumerate(classifiers.items()):

    # fit the data and tag outliers
    if clf_name == "Local Outlier Factor":
        y_pred = clf.fit_predict(X)
        scores_pred = clf.negative_outlier_factor_
        save_model(clf, 'Outlier.pkl')  # Saving the LOF
    else:
        clf.fit(X)
        scores_pred = clf.decision_function(X)
        y_pred = clf.predict(X)
        save_model(clf, 'Isolation.pkl')  # Saving the isolation forest

    ...

然后您可以使用以下方式加载模型:

def load_model(filename):
    with open(filename, 'rb') as f:
        clf = pickle.load(f)
    return clf

您也可以保存为其他格式,该想法与所使用的软件包完全相同。

以上是关于运行模型后,如何将隔离林和局部离群因子保存为两个不同的模型?的主要内容,如果未能解决你的问题,请参考以下文章

如何修复局部视图的视图模型

OpenGL:加载 3d 模型(比如 obj 文件)后如何找到正确的缩放因子? [关闭]

离群值检测中的隔离森林与鲁棒随机森林砍伐森林

如何加载保存的 KMeans 模型(在 ML Pipeline 中)?

异常检测之基于聚类的局部异常因子检测算法(CBLOF)详细解释且配上代码运行实例

更改核心数据关系验证规则后保存缓慢