如何对推特数据进行分页以显示 100 条或更多最新推文？

Posted 2023-03-28

【中文标题】如何对推特数据进行分页以显示 100 条或更多最新推文？

【英文标题】：How to paginate twitter data to show 100 or more latest tweet?

【发布时间】：2021-07-16 10:52:10

【问题描述】：

我是 PowerBI 环境的新手。并从一些来源获取此源代码以创建搜索参数并显示有关多个关键字的最新推文。问题是，PowerBI 中显示的最新数据仅累积到最近 7 天的推文。如何生成过去 1 个月或几年的最新数据？谢谢。

这里是代码 /* 此 M 脚本获取不记名令牌并从 Twitter REST API 执行推文搜索 https://dev.twitter.com/oauth/application-only 需要建立 Twitter 应用程序才能获得消费者密钥和消费者秘密 https://apps.twitter.com/

IMPORTANT - The Consumer Key and Consumer secret should be treated as passwords and not distributed
*/

let
     // Concatenates the Consumer Key & Consumer Secret and converts to base64
     authKey = "Basic " & Binary.ToText(Text.ToBinary("XXXAPITOKENXXX"),0),
     url = "https://api.twitter.com/oauth2/token",
     // Uses the Twitter POST oauth2/token method to obtain a bearer token
     GetJson = Web.Contents(url,
         [
         Headers = [#"Authorization"=authKey,
                    #"Content-Type"="application/x-www-form-urlencoded;charset=UTF-8"],
         Content = Text.ToBinary("grant_type=client_credentials") 
     ]
 ),
 FormatAsJson = Json.Document(GetJson),
 // Gets token from the Json response
 AccessToken = FormatAsJson[access_token],
 AccessTokenHeader = "bearer " & AccessToken,
 // Uses the Twitter GET search/tweets method using the bearer token from the previous POST oauth2/token method
 GetJsonQuery = Web.Contents("https://api.twitter.com/1.1/search/tweets.json?q="&SearchParameters&"&count=100",
     [
         Headers = [#"Authorization"=AccessTokenHeader]
     ]
 ),
 FormatAsJsonQuery = Json.Document(GetJsonQuery),
 NavigateToStatuses = FormatAsJsonQuery[statuses],
 TableFromList = Table.FromList(NavigateToStatuses, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
 ExpandColumn = Table.ExpandRecordColumn(TableFromList, "Column1", "metadata", "created_at", "id", "id_str", "text", "source", "truncated", "in_reply_to_status_id", "in_reply_to_status_id_str", "in_reply_to_user_id", "in_reply_to_user_id_str", "in_reply_to_screen_name", "user", "geo", "coordinates", "place", "contributors", "is_quote_status", "retweet_count", "favorite_count", "entities", "favorited", "retweeted", "lang", "possibly_sensitive", "quoted_status_id", "quoted_status_id_str", "quoted_status", "Column1.metadata", "Column1.created_at", "Column1.id", "Column1.id_str", "Column1.text", "Column1.source", "Column1.truncated", "Column1.in_reply_to_status_id", "Column1.in_reply_to_status_id_str", "Column1.in_reply_to_user_id", "Column1.in_reply_to_user_id_str", "Column1.in_reply_to_screen_name", "Column1.user", "Column1.geo", "Column1.coordinates", "Column1.place", "Column1.contributors", "Column1.is_quote_status", "Column1.retweet_count", "Column1.favorite_count", "Column1.entities", "Column1.favorited", "Column1.retweeted", "Column1.lang", "Column1.possibly_sensitive", "Column1.quoted_status_id", "Column1.quoted_status_id_str", "Column1.quoted_status")
in
    ExpandColumn

【参考方案1】：

v1.1 搜索 API 仅提供对过去 7 天推文的访问。您将需要使用不同的 API 从更远的地方检索数据：

premium API 30-days or full-archive search（这些是商业的，但有一个限制访问的免费层）

或

v2 full-archive search（这要求您有一个在Academic Research product track 上的帐户）

【参考方案2】：

首先您需要拥有高级搜索 API 访问权限，然后通过传递参数 toDate、fromDate 和 ma​​xResults 您可以获得所需的分页数据.

文档供参考Twitter Search API Document