sql去除重复数据

Posted 2020-07-04

1，采用存储过程添加测试数据

CREATE PROCEDURE NewProcedure()
BEGIN
DECLARE i INT;
SET i=1;
WHILE i<50000 DO
INSERT INTO user_new (id,name,phone,sex) VALUES (i,i,i,i);
SET i = i + 1;
END WHILE;
END;

两次插入，修改后一次存储过程的id取值，于是得到了name,phone,sex重复的5W条数据（时间有限，未操作更大数量）

2，数据去重

常见方式：

一、

update user_new  set data_status=0 where id not in

(select max(id) as id from t_testr_res_indicate group by name,phone,sex
);

 

二、重复数据量少使用

update user_new  set data_status=0 where id not in

(select max(id) as id from user_new  group by name,phone,sex having count(*)>1
) and

id in

(select  id from user_new  group by name,phone,sex having count(*)>1
) ;