com.google.cloud.bigquery.BigQueryException：读取超时

Posted 2023-03-24

【中文标题】com.google.cloud.bigquery.BigQueryException：读取超时

【英文标题】：com.google.cloud.bigquery.BigQueryException: Read timed out

【发布时间】：2017-01-25 09:19:27

【问题描述】：

我正在从 BigQuery 查询数据

这是我的代码：

import com.google.cloud.bigquery.*;

 public static JSONArray query(String tableId, String field, String val) throws Exception

    String queryString = "SELECT * FROM `" + tableId +"` where " + field + "=\"" + val + "\";";
    BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
    // Create a query request
    QueryRequest queryRequest =
            QueryRequest.newBuilder(queryString)
                    .setMaxWaitTime(100000L)
                    .setPageSize(1000L)
                    .setUseLegacySql(false)
                    .build();
    // Request query to be executed and wait for results
    QueryResponse queryResponse = bigquery.query(queryRequest);
    while (!queryResponse.jobCompleted()) 
        Thread.sleep(1000L);
        queryResponse = bigquery.getQueryResults(queryResponse.getJobId());
    
    // Read rows
    Iterator<List<FieldValue>> rowIterator = queryResponse.getResult().iterateAll();
    while (rowIterator.hasNext()) 
        System.out.println(rowIterator.next());
    

这是我要获取的数据，位于 tableA 和 tableB 中：

id: "111", data: a: "222", b: "333"

当我在第一个表上运行查询时

query("dataset.tableA", "data.a",  "222")

我得到了很好的结果

但是当我在第二张表上运行查询时

query("dataset.tableB", "data.a",  "222")

我得到一个错误 - 即使当我查询抛出 webUI 时我得到的结果很好：

这是错误：

Exception in thread "main" com.google.cloud.bigquery.BigQueryException: Read timed out
    at com.google.cloud.bigquery.spi.DefaultBigQueryRpc.translate(DefaultBigQueryRpc.java:93)
    at com.google.cloud.bigquery.spi.DefaultBigQueryRpc.query(DefaultBigQueryRpc.java:408)
    at com.google.cloud.bigquery.BigQueryImpl$21.call(BigQueryImpl.java:584)
    at com.google.cloud.bigquery.BigQueryImpl$21.call(BigQueryImpl.java:581)
    at com.google.cloud.RetryHelper.doRetry(RetryHelper.java:179)
    at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:244)
    at com.google.cloud.bigquery.BigQueryImpl.query(BigQueryImpl.java:580)
    at com.example.helloworld.integration.helpers.BigQueryDownload.query(BigQueryDownload.java:59)
    at com.example.helloworld.integration.helpers.BigQueryDownload.main(BigQueryDownload.java:36)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
Caused by: java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
    at java.net.SocketInputStream.read(SocketInputStream.java:170)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
    at sun.security.ssl.InputRecord.read(InputRecord.java:503)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
    at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
    at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
    at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
    at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
    at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338)
    at com.google.api.client.http.javanet.NetHttpResponse.<init>(NetHttpResponse.java:37)
    at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:94)
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
    at com.google.cloud.bigquery.spi.DefaultBigQueryRpc.query(DefaultBigQueryRpc.java:406)

【问题讨论】：

不 - 这完全是一个不同的问题 - 我的回复不是太大 - 只包含一行 我收到 Read timed out 错误，而不是 Response is too large

您的查询在 UI 中运行需要多长时间？它可能会超时（尝试设置 timeoutMs 字段 - ***.com/questions/41494059/…

1 毫秒。没多久，我的表没有很多记录

"1ms" - 这是不可能的！你是说1s还是1m？

Query complete (1.9s elapsed, 4.01 MB processed) 我也不明白为什么我在一个表中得到超时而在另一个我没有，我尝试将timeoutMs 字段设置为更长，但没有帮助

【参考方案1】：

您可以在构建 BigQuery 客户端时配置超时：

BigQuery bigquery = BigQueryOptions.getDefaultInstance()
     .setRetrySettings(RetrySettings.newBuilder()
     .setMaxAttempts(10)
     .setRetryDelayMultiplier(1.5)
     .setTotalTimeout(Duration.ofMinutes(5))
.build()).getService();