python 通过执行LOWESS拟合,然后仅保留特定中位西格玛内的斑点,从EELS光谱中移除异常数据点。

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了python 通过执行LOWESS拟合,然后仅保留特定中位西格玛内的斑点,从EELS光谱中移除异常数据点。相关的知识,希望对你有一定的参考价值。

%pylab inline
import hyperspy.api as hs
from statsmodels.nonparametric.smoothers_lowess import lowess

def DespikeEELS(spec, m=3., interp_outliers=True):
    # Make a copy of the data
    E = spec.axes_manager[0].axis.copy()
    I = spec.data.copy()
    
    # Lowess smooth.
    smooth = lowess(I, E, frac=0.02)
    
    # Using median, find inliers and outliers as # sigma away.
    dist = np.abs(I-smooth[:,1])
    d = np.abs(dist - np.median(dist))
    mdev = np.median(d)
    s = d/mdev if mdev else 0.
    
    # Which points are inliers and which are out?
    inliers = np.where(s<=m)[0]
    outliers = np.where(s>m)[0]
    
    if interp_outliers == False:
        # Keep the inliers only (drop outliers).
        E = E[inliers]
        I = I[inliers]
        spec2 = spec.deepcopy()
        spec2.axes_manager[0].axis = E
        spec2.data = I
        return spec2
    else:
        # Interpolate the outliers to the lowess curve (retains same spectrum dimensions).
        E[outliers] = smooth[outliers,0]
        I[outliers] = smooth[outliers,1]
        spec2 = spec.deepcopy()
        spec2.axes_manager[0].axis = E
        spec2.data = I
        return spec2

def ProcessEELS(CoreLoss=None, LowLoss=None, BkgRange=None, DespikeThreshhold=None, NormRange=None):
    # Load the core and low loss spectra.
    cl = hs.load(CoreLoss)
    if len(cl) > 1:
        cl = cl[0]
        
    if LowLoss is not None:
        ll = hs.load(LowLoss)
        # Align to ZLP.
        ll.align_zero_loss_peak(also_align=[cl], subpixel=True)
        # Estimate the thickness.
        ll.estimate_thickness(threshold=5.).data
    
    if DespikeThreshhold is not None:
        cl = DespikeEELS(cl, m=DespikeThreshhold)
    
    if BkgRange is not None:
        x = cl.remove_background(background_type='PowerLaw',signal_range=BkgRange, fast=False)
        cl.data = x
        
    if LowLoss is not None:
        deconv = cl.fourier_ratio_deconvolution(ll)

    if NormRange is not None:
        cl.data -= np.min(cl.data)
        cl.data /= np.mean(cl.data[(cl.axes_manager[0].axis > NormRange[0]) & (cl.axes_manager[0].axis < NormRange[1])])
        deconv.data -= np.min(deconv.data)
        deconv.data /= np.mean(deconv.data[(deconv.axes_manager[0].axis > NormRange[0]) & (deconv.axes_manager[0].axis < NormRange[1])])


    return ll, cl, deconv

以上是关于python 通过执行LOWESS拟合,然后仅保留特定中位西格玛内的斑点,从EELS光谱中移除异常数据点。的主要内容,如果未能解决你的问题,请参考以下文章

R语言可视化散点图(scatter plot)并在散点图中叠加回归曲线叠加lowess拟合曲线(linear and lowess fit lines)使用plotlineabline函数

R语言散点图可视化:自定义标题和标签拟合回归线lowess为散点图添加平滑拟合线修改散点图中点颜色和点符号分组散点图添加图例pairs可视化散点图矩阵ggplt2可视化lattice

局部加权回归法是啥

Python中LOWESS的置信区间

如何克隆包括其数据的 scikit-learn 估计器?

最佳拟合线与R中的阈值