带有熊猫的剪影分数的正确数据格式

Posted 2023-03-12

【中文标题】带有熊猫的剪影分数的正确数据格式

【英文标题】：The right data format for silhouette_score with pandas

【发布时间】：2019-03-10 22:16:59

【问题描述】：

我想使用 silhouette_score 来估计最佳聚类数。我正在使用 sklearn 的官方示例，但它给了我这个错误：TypeError: silhouette_score() takes 1 positional argument but 2 were given。

我的数据 (X) 是具有 20 个特征（所有非空 float64）和索引为唯一 ID 字符串的 pandas 数据帧（这可能是个问题吗？）。

    f1   f2   f3    …   f20
ID                  
AA2 0.33 0   0.31   …   0.16
BS4 0    0   0      …   0.41
VK9 0    0   0      …   0.48

我正在使用 data.values 将其转换为矩阵（参见下面的代码）。感谢您的帮助！

X = data.values
for n_clusters in range_n_clusters:
    # Create a subplot with 1 row and 2 columns
    fig, (ax1, ax2) = plt.subplots(1, 2)
    fig.set_size_inches(18, 7)
# The 1st subplot is the silhouette plot
# The silhouette coefficient can range from -1, 1 but in this example all
# lie within [-0.1, 1]
ax1.set_xlim([-0.1, 1])
# The (n_clusters+1)*10 is for inserting blank space between silhouette
# plots of individual clusters, to demarcate them clearly.
ax1.set_ylim([0, len(X) + (n_clusters + 1) * 10])

# Initialize the clusterer with n_clusters value and a random generator
# seed of 10 for reproducibility.
clusterer = KMeans(n_clusters=n_clusters, random_state=10)
cluster_labels = clusterer.fit_predict(X)

# The silhouette_score gives the average value for all the samples.
# This gives a perspective into the density and separation of the formed
# clusters
silhouette_avg = silhouette_score(X, cluster_labels)
print("For n_clusters =", n_clusters,
      "The average silhouette_score is :", silhouette_avg)

# Compute the silhouette scores for each sample
sample_silhouette_values = silhouette_samples(X, cluster_labels)

y_lower = 10
for i in range(n_clusters):
    # Aggregate the silhouette scores for samples belonging to
    # cluster i, and sort them
    ith_cluster_silhouette_values = \
        sample_silhouette_values[cluster_labels == i]

    ith_cluster_silhouette_values.sort()

    size_cluster_i = ith_cluster_silhouette_values.shape[0]
    y_upper = y_lower + size_cluster_i

    color = cm.nipy_spectral(float(i) / n_clusters)
    ax1.fill_betweenx(np.arange(y_lower, y_upper),
                      0, ith_cluster_silhouette_values,
                      facecolor=color, edgecolor=color, alpha=0.7)

    # Label the silhouette plots with their cluster numbers at the middle
    ax1.text(-0.05, y_lower + 0.5 * size_cluster_i, str(i))

    # Compute the new y_lower for next plot
    y_lower = y_upper + 10  # 10 for the 0 samples

ax1.set_title("The silhouette plot for the various clusters.")
ax1.set_xlabel("The silhouette coefficient values")
ax1.set_ylabel("Cluster label")

# The vertical line for average silhouette score of all the values
ax1.axvline(x=silhouette_avg, color="red", linestyle="--")

ax1.set_yticks([])  # Clear the yaxis labels / ticks
ax1.set_xticks([-0.1, 0, 0.2, 0.4, 0.6, 0.8, 1])

# 2nd Plot showing the actual clusters formed
colors = cm.nipy_spectral(cluster_labels.astype(float) / n_clusters)
ax2.scatter(X[:, 0], X[:, 1], marker='.', s=30, lw=0, alpha=0.7,
            c=colors, edgecolor='k')

# Labeling the clusters
centers = clusterer.cluster_centers_
# Draw white circles at cluster centers
ax2.scatter(centers[:, 0], centers[:, 1], marker='o',
            c="white", alpha=1, s=200, edgecolor='k')

for i, c in enumerate(centers):
    ax2.scatter(c[0], c[1], marker='$%d$' % i, alpha=1,
                s=50, edgecolor='k')

ax2.set_title("The visualization of the clustered data.")
ax2.set_xlabel("Feature space for the 1st feature")
ax2.set_ylabel("Feature space for the 2nd feature")

plt.suptitle(("Silhouette analysis for KMeans clustering on sample data "
              "with n_clusters = %d" % n_clusters),
             fontsize=14, fontweight='bold')

plt.show()

错误：

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-129-1d93fb88b278> in <module>()
----> 1 sil_(var_th.values,[2, 3, 4, 5, 6])

<ipython-input-127-0e092cfcc4be> in sil_(X, range_n_clusters)
     21         # This gives a perspective into the density and separation of the formed
     22         # clusters
---> 23         silhouette_avg = silhouette_score(X, cluster_labels)
     24         print("For n_clusters =", n_clusters,
     25               "The average silhouette_score is :", silhouette_avg)

TypeError: silhouette_score() takes 1 positional argument but 2 were given

【问题讨论】：

如何导入silhouette_score？来自 scikit-learn？如果有，scikit-learn 的版本是多少？

这些是我的导入from sklearn.cluster import KMeans from sklearn.metrics import silhouette_samples, silhouette_score 版本是scikit-learn==0.19.2

那么在这种情况下，silhouette_score 确实需要 2 个位置参数。所以你的用法似乎是正确的。但是为了查明错误，我们需要您的完整脚本和一些示例数据。

脚本完成，我描述了数据。我可以在 pandas 中生成样本随机数据并包含代码。这会有帮助吗？

【参考方案1】：

我想通了...问题是由具有多个级别的索引引起的。 data.reset_index(inplace=True) 然后对数据进行切片 X = data[data.columns[1:]].values 以删除 ID 列就成功了……但是感谢 cmets，因为他们迫使我更仔细地查看数据。