通过 BigQuery Java api 创建数据存储备份表

Posted 2023-03-25

【中文标题】通过 BigQuery Java api 创建数据存储备份表

【英文标题】：Create table of datastore backup via BigQuery Java api

【发布时间】：2017-06-12 11:09:06

【问题描述】：

我在 Google Cloud Storage 上有一个 Google Cloud Datastore 备份，我想将其自动导入 BigQuery。

TableDefinition tableDefinition = ExternalTableDefinition
        .newBuilder(tableUri, Schema.of(), FormatOptions.datastoreBackup())
        .setAutodetect(true)
        .build();
return bigQuery.create(TableInfo.of(TableId.of("ds", tableName), tableDefinition));

这会引发以下异常；

com.google.cloud.bigquery.BigQueryException：指定架构是 STORAGE_FORMAT_DATASTORE_BACKUP 不允许

如果我将 Schema.of() 更改为 null，它会抛出一个空指针。并且所有工厂方法都有一个需要方案的方法签名。如何通过 Java API 将此表创建为外部表？

【参考方案1】：

试试这个

public class DatastoreBackupImport 

        private String datasetName = "...";
        private String uri = "gs://xxx/xxx/.xxx.backup_info";
        private String tableName = "xxx";

        private void importBackup()
            BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
            TableId tableId = TableId.of(datasetName, tableName);
            TableDefinition tableDefinition = StandardTableDefinition.newBuilder().build();
            TableInfo tableInfo = TableInfo.newBuilder(tableId, tableDefinition).build();
            Table table = bigquery.create(tableInfo);
            Job job = table.load(FormatOptions.datastoreBackup(), uri);
    // Wait for the job to complete
            try 
                Job completedJob = job.waitFor(WaitForOption.checkEvery(1, TimeUnit.SECONDS),
                    WaitForOption.timeout(3, TimeUnit.MINUTES));
                if (completedJob != null && completedJob.getStatus().getError() == null) 
                    // Job completed successfully
                    System.out.println("Job completed successfully");
                 else 
                    // Handle error case
                    System.out.printf("There was an error");
                    System.out.println(completedJob.getStatus().getError().getMessage());
                
             catch (InterruptedException | TimeoutException e) 
                // Handle interrupted wait
                System.out.println("There was an interruption");