FT实现逆地理编码&报错处理
Posted 拼命十三郎
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了FT实现逆地理编码&报错处理相关的知识，希望对你有一定的参考价值。
最近遇到一个逆地理编码的需求（有经纬度数据需要转化成省市县数据），确定采用高德的坐标系（还有百度坐标系、大地坐标系等其他坐标系），因此借用高德的逆地理编码接口进行转换（个人日限制30万次），因采用FT来做，遇到一点问题特此记录。
一、编码任务
为求稳定，这次采用逐次调用，参考文档地理/逆地理编码。完成编码需申请成为高德个人开发者(免费），在高德控制台管理Key页面创建新应用并生成专属key。
1. FT流程
表输入----HTTP----JSON----表输出
表输出：
SELECT * FROM
(
select
re_id,
ROW_NUMBER() over(ORDER BY re_id) AS NUM,
re_time,
vin_id,
substring (longitude,1,10) longitude,
substring (latitude,1,10) latitude,
concat(\'https://restapi.amap.com/v3/geocode/regeo?output=json&location=\',substring (longitude,1,10),\',\',substring (latitude,1,10),\'&key=************************&radius=1000&extensions=base\') as url,
upload_time
from dw.car_gps
WHERE province is null
) car
WHERE NUM > 70000
**********************替换成自己的key
HTTP：
JSON:
字段值路径：
$.regeocode.addressComponent.province
$.regeocode.addressComponent.city
$.regeocode.addressComponent.district
2. 加速调取
利用表输入的sql：sql ROW_NUMBER() over(ORDER BY re_id) AS NUM把数据分组，在多个任务中并行调取。
如需并行，FT需设置：
二、问题解决
1. 后台报错连接超时
java.net.ConnectException: 连接超时 (Connection timed out)
1.1 错误详情
错误详情：com.fr.finetube.base.exp.work.step.EventHandleFailureException: Step HTTP Client request url https://restapi.amap.com/v3/geocode/regeo?output=json&location=121.210542,31.2670332&key=***********************&radius=1000&extensions=base failed.
at com.fr.finetube.core.work.step.component.query.HttpClientStep.doWork(HttpClientStep.java:209)
at com.fr.finetube.core.work.step.component.AbstractNamedStep.onEvent(AbstractNamedStep.java:176)
at com.fr.finetube.core.work.task.thread.runner.impl.SimpleProcessRunner.processStep(SimpleProcessRunner.java:132)
at com.fr.finetube.core.work.task.thread.runner.impl.SimpleProcessRunner.processStep(SimpleProcessRunner.java:136)
at com.fr.finetube.core.work.task.thread.runner.impl.SimpleProcessRunner.process(SimpleProcessRunner.java:68)
at com.fr.finetube.core.work.task.thread.CronRepeatThread.run(CronRepeatThread.java:50)
Caused by: org.apache.http.conn.HttpHostConnectException: Connect to restapi.amap.com:443 [restapi.amap.com/59.82.14.114] failed: 连接超时 (Connection timed out)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:373)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:394)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at com.fr.finetube.base.utils.HttpClientUtils.httpResponseGet(HttpClientUtils.java:208)
at com.fr.finetube.core.work.step.component.query.HttpClientStep.lambda$getHttpClientMethod$4(HttpClientStep.java:374)
at com.fr.finetube.core.work.step.component.query.HttpClientStep.doWork(HttpClientStep.java:173)
... 5 more
Caused by: java.net.ConnectException: 连接超时 (Connection timed out)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
... 16 more
1.2 解决
1.3 解决后现象
采用异常处理后，每次任务失败会显示为暂停，并间隔重启任务进行编码（曲线救国了属于是）。
2. 后台报错SSL信号中断
javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
2.1 错误详情
错误详情：com.fr.finetube.base.exp.work.step.EventHandleFailureException: Step HTTP Client request url https://restapi.amap.com/v3/geocode/regeo?output=json&location=120.636936,33.0491199&key=******************************&radius=1000&extensions=base failed.
at com.fr.finetube.core.work.step.component.query.HttpClientStep.doWork(HttpClientStep.java:209)
at com.fr.finetube.core.work.step.component.AbstractNamedStep.onEvent(AbstractNamedStep.java:176)
at com.fr.finetube.core.work.task.thread.runner.impl.SimpleProcessRunner.processStep(SimpleProcessRunner.java:132)
at com.fr.finetube.core.work.task.thread.runner.impl.SimpleProcessRunner.processStep(SimpleProcessRunner.java:136)
at com.fr.finetube.core.work.task.thread.runner.impl.SimpleProcessRunner.process(SimpleProcessRunner.java:68)
at com.fr.finetube.core.work.task.thread.SelfRepeatThread.run(SelfRepeatThread.java:88)
Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1002)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:373)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:394)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at com.fr.finetube.base.utils.HttpClientUtils.httpResponseGet(HttpClientUtils.java:208)
at com.fr.finetube.core.work.step.component.query.HttpClientStep.lambda$getHttpClientMethod$4(HttpClientStep.java:374)
at com.fr.finetube.core.work.step.component.query.HttpClientStep.doWork(HttpClientStep.java:173)
... 5 more
Caused by: java.io.EOFException: SSL peer shut down incorrectly
at sun.security.ssl.InputRecord.read(InputRecord.java:505)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
... 22 more
2.2 现象
即便是使用异常处理，该报错仍然不可避免：消息中心显示三次暂停&一次中断。目前只能手动再次开启任务。
2.3 解决
目前只能重启。
本文来自博客园，作者：拼命十三郎，转载请注明原文链接：https://www.cnblogs.com/shixiu/p/15502654.html
高德地图：地理/逆地理编码
高德API：http://lbs.amap.com/api/webservice/guide/api/georegeo/
http://restapi.amap.com/v3/geocode/geo?key=7de8697669288fc848e12a08f58d995e&s=rsv3&city=杭州市&address=杭州市天成路87号现代大厦
这里边的key是别人开放的key，用起来感觉还可以。基本上没有太大的限制，使用上边的接口获取经纬度是基本上100%成功，但是这个坐标系是GCJ02坐标。
按照上边的接口可以返回的结果信息为：
{"status":"1","info":"OK","infocode":"10000","count":"1","geocodes":[{"formatted_address":"浙江省杭州市江干区现代大厦","province":"浙江省","citycode":"0571","city":"杭州市","district":"江干区","township":[],"neighborhood":{"name":[],"type":[]},"building":{"name":[],"type":[]},"adcode":"330104","street":[],"number":[],"location":"120.204319,30.287145","level":"兴趣点"}]}
根据GPS格式的经纬度，调用高德API返回具体地址信息：
using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Net; using System.Text; namespace ConsoleApp1 { class Program { static void Main(string[] args) { string address = GetAddressByGcjLngLat("120.204319", "30.287145"); Console.WriteLine(address); Console.ReadKey(); } /// <summary> /// 根据GPS经纬度，返回对应位置的地址信息。 /// 请求API:http://restapi.amap.com/v3/geocode/regeo?output=json&location=120.204319,30.287145&key=7de8697669288fc848e12a08f58d995e&radius=100&extensions=base /// </summary> /// <param name="longitude"></param> /// <param name="latitude"></param> /// <returns></returns> public static string GetAddressByGcjLngLat(string longitude, string latitude) { Tuple<bool, string, string> gcjLngLat = TransformGPS2GCJ(longitude, latitude); // Console.WriteLine("{0}:{1}:{2}", gcjLngLat.Item1, gcjLngLat.Item2, gcjLngLat.Item3); if (false == gcjLngLat.Item1) return string.Empty; string address = string.Empty; try { // 格式：http://restapi.amap.com/v3/geocode/regeo?output=json&location=120.204319,30.287145&key=7de8697669288fc848e12a08f58d995e&radius=100&extensions=base string content = HttpGet("http://restapi.amap.com/v3/geocode/regeo?output=json&location=" + gcjLngLat.Item2 + "," + gcjLngLat.Item3 + "&key=7de8697669288fc848e12a08f58d995e&radius=200&extensions=base"); // 返回值格式：{"status":"1","info":"OK","infocode":"10000","regeocode":{"formatted_address":"浙江省杭州市江干区闸弄口街道万家花园(天城路)","addressComponent":{"country":"中国","province":"浙江省","city":"杭州市","citycode":"0571","district":"江干区","adcode":"330104","township":"闸弄口街道","towncode":"330104007000","neighborhood":{"name":"万家花园(天城路)","type":"商务住宅;住宅区;住宅小区"},"building":{"name":[],"type":[]},"streetNumber":{"street":"天城路","number":"87号","location":"120.204319,30.2871444","direction":"南","distance":"0.0617778"},"businessAreas":[{"location":"120.20604745161295,30.28125106451613","name":"火车东站","id":"330104"}]}}} string[] adddressItems = content.Replace("\\"", string.Empty).Split(new char[] { \'{\', \',\', \'}\', \'[\', \']\' }); if (adddressItems != null && adddressItems.Length > 0) { foreach (string addressItem in adddressItems) { if (addressItem.StartsWith("formatted_address:", StringComparison.OrdinalIgnoreCase)) { address = addressItem.Replace("formatted_address:", string.Empty); break; } } } } catch (Exception ex) { // 出现异常，返回转化失败且不扑捉异常信息。 } // Console.WriteLine(address); return address; } /// <summary> /// 接口地址 /// http://api.zdoz.net/transmore.ashx /// 接口说明 /// 批量纠偏，一次最大可纠偏1000个坐标点 /// 参数 /// lats:维度，多个维度用“;”隔开 /// lngs:经度，多个经度用“;”隔开（要注意经纬度个数相等） /// type:转换类型 【1.WGS -> GCJ】 【2.GCJ -> WGS】 【3.GCJ -> BD】 【4.BD -> GCJ】 【5.WGS -> BD】 【6.BD -> WGS】 /// 返回值JSON /// 根据次序返回一个json格式的数组 /// 演示 /// 参数：lats=34.123;34.332;55.231&lngs=113.123;112.213;115.321&type=1 /// 返回：[{"Lng":113.12942937312582,"Lat":34.121761850760855},{"Lng":112.21911710957568,"Lat":34.3306763095054}, {"Lng":115.33036232125529,"Lat":55.232930158541052}] /// </summary> /// <param name="lngGPS">gps格式的经度，将会被转化为GCJ02格式的经度</param> /// <param name="latGPS">gps格式的纬度，将会被转化为GCJ02格式的纬度</param> /// <returns>(bool:是否转化成功,string:gcj02格式的经度,string:gcj02格式的纬度)</returns> public static Tuple<bool, string, string> TransformGPS2GCJ(string lngGPS, string latGPS) { Tuple<bool, string, string> result = new Tuple<bool, string, string>(false, string.Empty, string.Empty); try { // 格式：http://api.zdoz.net/transmore.ashx?lats=34.123&lngs=113.123&type=1 string content = HttpGet("http://api.zdoz.net/transmore.ashx?lats=" + latGPS + "&lngs=" + lngGPS + "&type=1"); // 返回值格式：[{"Lng":113.12942937312582,"Lat":34.121761850760855}] content = content.Replace("[{", string.Empty).Replace("}]", string.Empty).Replace("\\"", string.Empty); string[] lngLatItems = content.Split(new char[] { \',\' }); if (lngLatItems != null && lngLatItems.Length == 2 && lngLatItems[0].StartsWith("lng", StringComparison.OrdinalIgnoreCase) && lngLatItems[1].StartsWith("lat", StringComparison.OrdinalIgnoreCase)) { result = new Tuple<bool, string, string>(true, lngLatItems[0].ToLower().Replace("lng:", string.Empty), lngLatItems[1].ToLower().Replace("lat:", string.Empty)); } } catch (Exception ex) { // 出现异常，返回转化失败且不扑捉异常信息。 } return result; } /// <summary> /// Get访问uri并反回请求响应内容。 /// </summary> /// <param name="uri"></param> /// <param name="timeout">请求超时时间，默认：30s超时</param> /// <returns></returns> public static string HttpGet(string uri, int timeout = 30 * 1000) { string result = string.Empty; HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri); request.Method = "GET"; request.ContentType = "text/html;charset=UTF-8"; request.Timeout = timeout; using (HttpWebResponse response = (HttpWebResponse)request.GetResponse()) { using (Stream myResponseStream = response.GetResponseStream()) { using (StreamReader myStreamReader = new StreamReader(myResponseStream, Encoding.UTF8)) { result = myStreamReader.ReadToEnd(); } } } return result; } } }
以上是关于FT实现逆地理编码&报错处理的主要内容，如果未能解决你的问题，请参考以下文章
ArcGIS中Python逆地理编码，根据坐标获取实际的地址