如何在 C# 中从 cosmos db 获取准确的最新 5 分钟数据

Posted 2023-02-15

【中文标题】如何在 C# 中从 cosmos db 获取准确的最新 5 分钟数据

【英文标题】：How to get exact latest 5 minutes data from cosmos db in C#

【发布时间】：2021-01-30 18:19:03

【问题描述】：

我有一个网络作业，它连续运行并从网络套接字 api 读取数据。

以下是每隔 1 秒自动运行并将刻度数据添加到 cosmos db 的代码。

private static void OnTick(Tick TickData)
        

            var latestTickData = new MyObject()
            
                InstrumentID = TickData.InstrumentToken,
                Close = TickData.LastPrice,
                High = TickData.LastPrice,
                Low = TickData.LastPrice,
                Open = TickData.LastPrice,
                TimeStamp = TickData.Timestamp.HasValue ? TickData.Timestamp.Value : DateTime.Now
            ;

            // add data into cosmos

            Task.Run(() =>
            
                Program.documentClient.CreateDocumentAsync(UriFactory.CreateDocumentCollectionUri("tickerDatabase", "tickerContainer"), latestTickData);
            ).Wait();
        

现在我想读取最近 5 分钟的数据并取开高低收盘价。

对于最近 5 分钟的数据，目前我每 5 分钟运行一次计时器作业，它从 cosmos 读取数据并计算开高低收，但问题是这里的时间。

如果计时器作业延迟 1 分钟运行，那么该数据的寡妇也会发生变化并获得错误的值。

我的问题是，如何从 cosmos 中获取准确的 5 分钟最新数据？

当前计时器作业代码 -

myobject.cs

public class MyObject
    
        public uint InstrumentID  get; set; 
        public decimal Close  get; set; 
        public decimal High  get; set; 
        public decimal Low  get; set; 
        public decimal Open  get; set; 
        public DateTime TimeStamp  get; set; 
        public uint Volume  get; set; 

        public DateTime GetStartOfPeriodByMins(int numMinutes)
        
            int oldMinutes = TimeStamp.Minute;
            int newMinutes = (oldMinutes / numMinutes) * numMinutes;

            DateTime startOfPeriod = new DateTime(TimeStamp.Year, TimeStamp.Month, TimeStamp.Day, TimeStamp.Hour, newMinutes, 0);

            return startOfPeriod;
        
    

myfunction.cs

public static void ExecuteProcess([TimerTrigger("0 */5 * * * *")] TimerInfo myTimer)
        
            var option = new FeedOptions  EnableCrossPartitionQuery = true ;
            var queryable = Program.documentClient.CreateDocumentQuery<MyObject>
            (UriFactory.CreateDocumentCollectionUri("tickerDatabase", "tickerContainer"), option).ToList();


            var resultSet = queryable.GroupBy(i => i.GetStartOfPeriodByMins(5))
               .Select(gr =>
              new
              
                  StartOfPeriod = gr.Key,
                  Low = gr.Min(item => item.Low),
                  High = gr.Max(item => item.High),
                  Open = gr.OrderBy(item => item.TimeStamp).First().Open,
                  Close = gr.OrderBy(item => item.TimeStamp).Last().Close
              );

            var my5min = resultSet.LastOrDefault();

            Console.WriteLine("time " + my5min.StartOfPeriod + " open " + my5min.Open + " high " + my5min.High + " low " + my5min.Low + " close " + my5min.Close);

让我解释一下 1 分钟数据的问题（虽然我最初想要 5 分钟）。

以下是样本记录-

List<MyObject> test = new List<MyObject>();
            test.Add(new MyObject()  Open = 2939, High = 2939, Low = 2939, Close = 2939, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 01) );
            test.Add(new MyObject()  Open = 2933, High = 2933, Low = 2933, Close = 2933, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 01) );
            test.Add(new MyObject()  Open = 2936, High = 2936, Low = 2936, Close = 2936, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 03) );
            test.Add(new MyObject()  Open = 2944, High = 2944, Low = 2944, Close = 2944, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 05) );
            test.Add(new MyObject()  Open = 2944, High = 2944, Low = 2944, Close = 2944, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 08) );
            test.Add(new MyObject()  Open = 2939, High = 2939, Low = 2939, Close = 2939, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 10) );
            test.Add(new MyObject()  Open = 2939, High = 2939, Low = 2939, Close = 2939, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 15) );
            test.Add(new MyObject()  Open = 2932, High = 2932, Low = 2932, Close = 2932, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 25) );
            test.Add(new MyObject()  Open = 2939, High = 2939, Low = 2939, Close = 2939, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 26) );
            test.Add(new MyObject()  Open = 2939, High = 2939, Low = 2939, Close = 2939, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 28) );
            test.Add(new MyObject()  Open = 2932, High = 2932, Low = 2932, Close = 2932, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 30) );
            test.Add(new MyObject()  Open = 2941, High = 2941, Low = 2941, Close = 2941, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 32) );
            test.Add(new MyObject()  Open = 2939, High = 2939, Low = 2939, Close = 2939, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 35) );
            test.Add(new MyObject()  Open = 2941, High = 2941, Low = 2941, Close = 2941, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 40) );
            test.Add(new MyObject()  Open = 2937, High = 2937, Low = 2937, Close = 2937, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 42) );
            test.Add(new MyObject()  Open = 2939, High = 2939, Low = 2939, Close = 2939, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 45) );
            test.Add(new MyObject()  Open = 2937, High = 2937, Low = 2937, Close = 2937, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 48) );
            test.Add(new MyObject()  Open = 2939, High = 2939, Low = 2939, Close = 2939, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 50) );
            test.Add(new MyObject()  Open = 2939, High = 2939, Low = 2939, Close = 2939, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 52) );
            test.Add(new MyObject()  Open = 2937, High = 2937, Low = 2937, Close = 2937, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 54) );
            test.Add(new MyObject()  Open = 2935, High = 2935, Low = 2935, Close = 2935, TimeStamp = new DateTime(2020, 10, 15, 10, 01, 56) );

            test.Add(new MyObject()  Open = 2935, High = 2935, Low = 2935, Close = 2935, TimeStamp = new DateTime(2020, 10, 15, 10, 02, 12) );

将 1 分钟作为参数传递给 GetStartOfPeriodByMins(1) -

这里记录的时间是 10:1:56 和 10:2:12。

现在您可以观察到最后我们将在 resultSet 中获得 2 个记录集

如果定时器触发功能没有在特定时间运行，它只会取最后一条记录的平均值为 10:2:12，这是不对的

那么问题是如何匹配运行时间？

5 分钟后也会发生同样的情况。

1分钟以上样本数据的输出

所以我们需要确保该时间范围内的数据应该是完整的。

【问题讨论】：

Task.Run(...).Wait(); 不。不。不。不不不不不。制作 private static async Task 并等待 CreateDocumentAsync。

只是为了在不同的线程中运行它，我这样做了

有什么不同？无论如何它都是异步的。

我的意思是目前我正在手动运行它而不是托管测试它但是如果发生这种情况，让我用示例数据解释它

啊，所以你想要完整分钟...

【参考方案1】：

这里一种可能的解决方案是利用文档的_ts 属性。您可以直接使用start date 和end date 进行SQL 查询，跨度为5 分钟。除此之外，您还可以将start date 保存为数据库中的last used end date（成本可忽略不计）。查询应如下所示：

SELECT * FROM c where c._ts <= 1601890740 AND c._ts >= 1601890585

另外请注意，您必须在 POSIX 到 DateTime 之间进行一些来回转换。