面试题：20230217

Posted 2023-03-04 宝哥大数据

1、hdfs 删除 回收站中1T的文件
2、hbase 连接池高可用，性能，包含连接的创建与销毁
3、次日留存率
今日访问过，次日也访问过。

device_id	login_date
2138	2021-05-03
2315	2021-08-13
2315	2021-08-14
2315	2021-08-15
3214	2021-05-09
3214	2021-06-15
3214	2021-08-15
3214	2021-08-16
3214	2021-08-18
6543	2021-08-13

通过窗口LEAD函数，判断次日是否访问

select device_id, 
login_date, 
lead(login_date, 1, login_date) over(partition by device_id order by login_date) as login_date2 -- 次日
from log

通过datediff计算两个日期的差值，=1 则 次日留存。

SELECT 
    device_id,
    login_date,
    login_date2,
    CASE WHEN datediff(login_date2, login_date)==1 THEN 1 ELSE 0 END AS isLC
FROM 
    (SELECT device_id,
         login_date,
         lead(login_date, 1, login_date) over(partition by device_id  ORDER BY  login_date) AS login_date2
    FROM log 
    ) a;
device_id	login_date	login_date2		isLC
2138    2021-05-03      2021-05-03      0
2315    2021-08-13      2021-08-14      1  --- 13号访问过，14号也访问过，即为13的次日留存
2315    2021-08-14      2021-08-15      1
2315    2021-08-15      2021-08-15      0
3214    2021-05-09      2021-06-15      0
3214    2021-06-15      2021-08-15      0
3214    2021-08-15      2021-08-16      1
3214    2021-08-16      2021-08-18      0
3214    2021-08-18      2021-08-18      0
6543    2021-08-13      2021-08-13      0

统计留存率，按照日期分组

SELECT login_date, sum(isLC)/count(device_id)
FROM (
SELECT 
    device_id,
    login_date,
    CASE WHEN datediff(login_date2, login_date)==1 THEN 1 ELSE 0 END AS isLC
FROM 
    (SELECT device_id,
         login_date,
         lead(login_date, 1, login_date) over(partition by device_id  ORDER BY  login_date) AS login_date2
    FROM log 
    ) a
) b GROUP BY login_date;


2021-05-03      0.0
2021-05-09      0.0
2021-06-15      0.0
2021-08-13      0.5
2021-08-14      1.0
2021-08-15      0.5
2021-08-16      0.0
2021-08-18      0.0

4、数据仓库是否可以使用范式建模？