django+xadmin+echarts实现数据可视化

Posted nolali

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了django+xadmin+echarts实现数据可视化相关的知识,希望对你有一定的参考价值。

使用xadmin后功能比较强大,在后台展示统计图表,这个需求真的有点烫手,最终实现效果如下图:

技术分享图片技术分享图片技术分享图片技术分享图片

xadmin后台与echarts完全融合遇到以下问题:

  1.没有现成的数据model

  2.获得指定时间段的数据

  3.添加自定义菜单

  4.图表不能在当前页展示(后台点击每个model都是内嵌在当前页)

  5.echarts动态展示数据


下面解决第一个问题:

  目前现状是得从一个千万级的大表里提取近12个月,近30天,近24小时3个时间维度的数据,同事建议使用中间表,于是乎建了3个。

  model如下:

# 定义发送短信按小时统计模型类
class Count24(models.Model):
    alia_day_time = models.CharField(
        max_length=20,
        verbose_name=年月
    )
    total_nums = models.IntegerField(
        default=0,
        verbose_name=发送总数
    )
    error_nums = models.IntegerField(
        default=0,
        verbose_name=失败总数
    )

    class Meta:
        verbose_name = verbose_name_plural = 短信发送按小时统计

    def __str__(self):
        return {0}: {1} {2}.format(self.alia_day_time, self.total_nums,self.error_nums)


# 定义机构发送短信统计模型类
class OrganizationCount(models.Model):
    alia_month_time = models.CharField(
        max_length=20,
        verbose_name=年月
    )
    alia_date_time = models.CharField(
        max_length=20,
        verbose_name=年月日
    )
    total_nums = models.IntegerField(
        default=0,
        verbose_name=发送总数
    )
    error_nums = models.IntegerField(
        default=0,
        verbose_name=失败总数
    )
    name = models.CharField(
        max_length=50,
        verbose_name=机构,
    )

    class Meta:
        verbose_name = verbose_name_plural = 机构发送短信统计

    def __str__(self):
        return {0}: {1} {2} {3} {4} {5}.format(self.id, self.alia_month_time,self.alia_date_time,self.total_nums, self.error_nums, self.name)


# 定义通道发送短信统计模型类
class ChannelCount(models.Model):
    alia_month_time = models.CharField(
        max_length=20,
        verbose_name=年月
    )
    alia_date_time = models.CharField(
        max_length=20,
        verbose_name=年月日
    )
    total_nums = models.IntegerField(
        default=0,
        verbose_name=发送总数
    )
    error_nums = models.IntegerField(
        default=0,
        verbose_name=失败总数
    )
    name = models.CharField(
        max_length=50,
        verbose_name=通道,
    )

    class Meta:
        verbose_name = verbose_name_plural = 通道发送短信统计

    def __str__(self):
        return {0}: {1} {2} {3} {4} {5}.format(self.id, self.alia_month_time,self.alia_date_time, self.total_nums, self.error_nums, self.name)

  sql查询如下(当然这样sql我是憋不出来的。。。):

# 查询得到新表数据
SELECT req_time, alia_time, count(*) as total_nums, count(t.`status`=2 or null) as error_nums, name FROM
(select *, DATE_FORMAT(req_time,%Y-%m) as alia_time, LEFT(body,LOCATE(,body)) as name from sms_smslog
where LOCATE(,body) >0
and LEFT(body,1)= ) as t GROUP BY alia_time , name;

# 插入新表【机构】
INSERT into sms_organizationcount (alia_month_time, alia_date_time, total_nums, error_nums, `name`) 
SELECT alia_month_time, alia_date_time, count(*) as total_nums, count(t.`status`=2 or null) as error_nums, name FROM


(select *, DATE_FORMAT(req_time,%Y-%m) as alia_month_time, DATE_FORMAT(req_time,%Y-%m-%d) as alia_date_time,
 LEFT(body,LOCATE(,body)) as name from sms_smslog
 where LOCATE(,body) >0
and LEFT(body,1)= ) as t GROUP BY alia_date_time , name;

# 插入新表【通道】
INSERT into sms_channelcount (alia_month_time, alia_date_time, total_nums, error_nums, `name`) 
SELECT alia_month_time, alia_date_time, count(*) as total_nums, count(t.`status`=2 or null) as error_nums, name FROM


(select *, DATE_FORMAT(req_time,%Y-%m) as alia_month_time, DATE_FORMAT(req_time,%Y-%m-%d) as alia_date_time, channel as name from sms_smslog ) as t GROUP BY alia_date_time , channel;

# 插入新表【24小时】
INSERT into sms_count24 (alia_day_time, total_nums, error_nums) 
SELECT alia_day_time, count(*) as total_nums, count(t.`status`=2 or null) as error_nums FROM


(select *, DATE_FORMAT(req_time,%Y-%m-%d %H) as alia_day_time from sms_smslog) as t GROUP BY alia_day_time;

  这样数据雏形就出来了,数据肯定不能这样插入,dba也不会答应的,下一篇会详细解决这个问题。


下面解决第二个问题:

  近12个月,近30天,近24小时,显然都是动态的。下面几个方法值得收藏下:

import time
from datetime import datetime, date, timedelta

# 生成近期多少天的日期
def gen_dates(end_date, days):
    day = timedelta(days=1)
    for i in range(days):
        yield (end_date - day * i)


# 生成近期多少小时
def gen_hour(end_date, hours):
    hour = timedelta(hours=1)
    for i in range(hours):
        yield (end_date - hour * i).strftime(%Y-%m-%d %H)


# 解决datetime类型不能序列化问题
class CJsonEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime):
            return obj.strftime(%Y/%m/%d %H:%M:%S)
        elif isinstance(obj, date):
            return obj.strftime(%Y/%m/%d)
        else:
            return json.JSONEncoder.default(self, obj)


# 生成近期12个月
def gen_months():
    now = datetime.now()
    today_year = now.year
    last_year = int(now.year) - 1
    today_year_months = range(1, now.month + 1)
    last_year_months = range(now.month + 1, 13)
    data_list_lasts = []
    for last_year_month in last_year_months:
        date_list = %s-%s % (last_year, last_year_month)
        data_list_lasts.append(date_list)
    data_list_todays = []
    for today_year_month in today_year_months:
        data_list = %s-%s % (today_year, today_year_month)
        data_list_todays.append(data_list)
    data_year_month = data_list_lasts + data_list_todays
    # data_year_month.reverse()
    return data_year_month

  循环得到的时间list,拼接sql就可以查询到所需要的数据了。


下面解决第三个问题:

  使用django来展示数据,主线一般是写视图,配路由,模板渲染。

  a.写视图,获取数据逻辑上面写的差不多了,最终得返回echarts什么格式的数据

def msgsend_recent_30days_failed(request):
    # 获取近30天短信失败量
    con = Cache_data_to_redis().connection
    cursor = con.cursor()
    date_li, mon_li, data = [], [], {}
    end_date = datetime.now().date()
    # 获取近30天
    for i in gen_dates(end_date, 30):
        date_li.append(i)
    date_li = date_li[::-1]
    data[date_li] = json.dumps(date_li, cls=CJsonEncoder)
    for i in date_li:
        sql = "SELECT error_nums as error from sms_organizationcount  where alia_date_time=‘{alia_date_time}‘;".format(alia_date_time=i)
        cursor.execute(sql)
        num = cursor.fetchone()
        if num == None:
            num = {}
            num[error] = 0
        mon_li.append(num[error])
    data[mon] = mon_li
    return render(request, data_analysis/msgsend_recent_30days_failed.html, context=data)

#得到的数据如下:
{
    date_li: ["2018/07/15", "2018/07/16", "2018/07/17", "2018/07/18", "2018/07/19", "2018/07/20", "2018/07/21", "2018/07/22", "2018/07/23", "2018/07/24", "2018/07/25", "2018/07/26", "2018/07/27", "2018/07/28", "2018/07/29", "2018/07/30", "2018/07/31", "2018/08/01", "2018/08/02", "2018/08/03", "2018/08/04", "2018/08/05", "2018/08/06", "2018/08/07", "2018/08/08", "2018/08/09", "2018/08/10", "2018/08/11", "2018/08/12", "2018/08/13"], 
    mon: [0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
}

  b.配路由,首先在xadmin后台添加自定义菜单

# app下的urls.py添加路由
url(r^data_analysis/msgsend_recent_30days_failed/$, views.msgsend_recent_30days_failed, name=msgsend_recent_30days_failed),

# adminx里面添加自定义菜单和url
class GlobalSetting(object):
    site_title = "短信后台管理系统"
    site_footer = "http://smsweb.corp.ncfgroup.com/xadmin"
    menu_style = "accordion"

    # 菜单
    def get_site_menu(self):
        return [
            {
                title: 近期数据统计和分析,
                perm: self.get_model_perm(SMSLog, view),
                icon: fa fa-bar-chart-o,
                menus: (
                    {
                        title: 短信整体情况,
                        # 写死的url进行替换
                        url: self.get_model_url(SMSLog,changelist).replace(xadmin/sms/smslog/,sms/data_analysis/msgsend_recent_24hours/),
                        # ‘url‘: ‘http://10.17.20.86:8004/sms/data_analysis/msgsend_recent_24hours/‘,
                        perm: self.get_model_perm(SMSLog, view),
                        icon: fa fa-smile-o
                    },
                )
            }
        ]
 
xadmin.site.register(views.CommAdminView, GlobalSetting)

下面解决第四个问题:

  这个问题重新描述下:点数据统计图表的时候会在当前页展示出来,要是想回到主页需要点后退,这样操作就很不舒服了。找到xadmin源码里对应菜单处,添加上a标签就可以解决这个比较尴尬的问题。

xadmin/templates/xadmin/includes/sitemenu_accordion.html最后一行上面加上:

<script>
    var anchors = document.getElementById("nav-panel-1").getElementsByTagName("a");
    for(i=0;i<anchors.length;i++){
        var anchor_item = anchors[i];
        anchor_item.setAttribute("target","_blank");
    }
</script>

下面解决最后一个问题:

使用echarms模板需要改动以下地方,

(1)使用sublink时,自动获取服务器host;

(2)x轴上的坐标原点可能不是紧挨着y轴,错位了一点,xAxis下面加上boundaryGap : false,

(3)title改成自己的

(4)x轴与y轴数据对应见下面完整代码

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>近30天通道短信情况</title>
</head>
<body>
<!-- 为ECharts准备一个具备大小(宽高)的Dom -->
<div id="lineMain" style="height:400px"></div>
<!-- ECharts单文件引入 -->
<script src="http://echarts.baidu.com/build/dist/echarts.js"></script>
<script type="text/javascript">
    var target = {{ target|safe }}
// 路径配置
require.config({
  paths: {
    echarts: http://echarts.baidu.com/build/dist
  }
});
// 使用
require(
      [
        echarts,
        echarts/chart/bar,
        echarts/chart/line
      ],
      drawEcharts
);

function drawEcharts(ec){
  drawLine(ec);
}

function drawLine(ec){
  var myLineChart = ec.init(document.getElementById(lineMain));
    var date_li = {{ date_li|safe }}
    var num = {{ num|safe }}
    var sub = window.location.href.match((.*)/(.*)/)[1];
    var sublink = sub + /msgsend_recent_12months_channel/;
    // 动态push数据到series
    var series = [];
    for (var k = 0; k< target.length;k++){
        var item = {
            name:target[k],
            type:line,
            data:num[k],
        };
        series.push(item);
    };
  var option2 = {
    title : {
    text: 近30天渠道短信情况,
        subtext: 近12个月渠道短信情况,
        sublink: sublink,
  },
  tooltip : {
    trigger: axis
  },
    grid:{
        y2: 80
    },
  legend: {
        orient: horizontal,
        y:  bottom,
        data:target,
  },
  toolbox: {
    show : true,
    feature : {
      mark : {show: true},
      dataView : {show: true, readOnly: false},
      magicType : {show: true, type: [line, bar]},
      restore : {show: true},
      saveAsImage : {show: true}
    }
  },
  calculable : true,
  xAxis : [
    {
      type : category,
      boundaryGap : false,
      data : date_li
    }
  ],
  yAxis : [
    {
      type : value,
    }
  ],
  series : series, 
};
myLineChart.setOption(option2,true);
}
</script>
</body>
</html>

 

以上是关于django+xadmin+echarts实现数据可视化的主要内容,如果未能解决你的问题,请参考以下文章

Django 后台管理xadmin

Django-xadmin+rule对象级权限的实现

Django xadmin的使用

Django1.9开发博客(14)- 集成Xadmin

第三百八十节,Django+Xadmin打造上线标准的在线教育平台—将所有app下的models数据库表注册到xadmin后台管理

Django+xadmin打造在线教育平台