用C#在azure函数的body中读取一个4MB的json文件，在body读取部分进程挂起

Posted 2023-02-25

【中文标题】用C#在azure函数的body中读取一个4MB的json文件，在body读取部分进程挂起

【英文标题】：Read a 4MB json file in the body of azure function by C#, and the Process hangs in body reading part

【发布时间】：2021-09-16 20:51:31

【问题描述】：

dotnet 版本：5.0.203 ide：JetBrain Rider

thread pool logs

当我调试函数时，我只能看到线程池中的线程启动和退出，而不是进入断点。

所有的读写都是由Task Async Await完成的。​​

之后我尝试了 MemoryStream，将阅读器的缓冲区大小增加到 40000+，它仍然挂起。

控制台日志：console logs

复制样本：

using System;
using System.Collections.Generic;
using System.IO;
using System.Net;
using System.Text;
using System.Threading.Tasks;
using Microsoft.AspNetCore.WebUtilities;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;

namespace XXXXXXXX.XXXXXX.Functions

    
    public static class TestFunction
    
        [Function("TestFunction")]
        public static async Task<HttpResponseData> Run([HttpTrigger(AuthorizationLevel.Function,  "post")]
            HttpRequestData req,
            FunctionContext executionContext)
        
            var logger = executionContext.GetLogger("TestFunction");
            logger.LogInformation("C# HTTP trigger function processed a request.");
            

            /* test01 */
            // var clDataJsonStr = await new HttpRequestStreamReader(req.Body, Encoding.UTF8, 4096).ReadToEndAsync();
            // var clDataJsonStr = await new HttpRequestStreamReader(req.Body, Encoding.UTF8, 4096).ReadToEndAsync();
            
            // req.Body.Seek(0, SeekOrigin.Begin);
            /* test02 */
            // var serializer = new JsonSerializer(); 
            // ClData clData;
            
            // using (var stream = new StreamReader(req.Body, Encoding.UTF8, false, 4096))
            // 
            //     using (JsonReader reader = new JsonTextReader(stream))
            //     
            //         while (await reader.ReadAsync())
            //         
            //             if (reader.TokenType != JsonToken.StartObject) continue;
            //             clData = serializer.Deserialize<ClData>(reader);
            //             logger.LogInformation(clData.ToString());
            //         
            //     
            // 
            
            /* test03 */
            // var stringReader = new StringReader(req.Body.ToString());
            // var str = await stringReader.ReadToEndAsync();
            
            // Console.WriteLine(str);

            /*test04*/
            var ms= new MemoryStream();
            await req.Body.CopyToAsync(ms);
            var jsonBytes = ms.ToArray();
            logger.LogInformation(jsonBytes.Length.ToString());
            
            var response = req.CreateResponse(HttpStatusCode.OK);
            response.Headers.Add("Content-Type", "text/plain; charset=utf-8");

            await response.WriteStringAsync("Welcome to Azure Functions!");

            return response;
            
        
    

测试 Json 文件：bycc.json

【问题讨论】：

如果答案有帮助，您可以将其标记为已接受。

下面的答案有帮助吗？

【参考方案1】：

您可以改用StreamReader 来读取 JSON 正文：

string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
Class1[] data = JsonConvert.DeserializeObject<Class1[]>(requestBody);

此代码适用于您的有效负载：

而且，这是上面截图中的Class1：

public class Class1
    
        public string ReportDay  get; set; 
        public string Type  get; set; 
        public string ReportTypeDescription  get; set; 
        public string DoorName  get; set; 
        public string DoorCode  get; set; 
        public string Cc  get; set; 
        public float TotalSaleAmountIncludingVisitors  get; set; 
        public float TotalSaleAmountIncludingVisitorsLy  get; set; 
        public float TotalSaleAmountIncludingVisitorsEvol  get; set; 
        public float NewCustomerSaleAmount  get; set; 
        public float NewCustomerSaleAmountLy  get; set; 
        public float NewCustomerSaleAmountEvol  get; set; 
        public float OldCustomerSaleAmount  get; set; 
        public float OldCustomerSaleAmountLy  get; set; 
        public float OldCustomerSaleAmountEvol  get; set; 
        public float VisitorSaleAmount  get; set; 
        public float VisitorSaleAmountLy  get; set; 
        public float VisitorSaleAmountEvol  get; set; 
        public float CustomerSaleRatio  get; set; 
        public float TotalSaleQtyIncludingVisitor  get; set; 
        public float TotalSaleQtyIncludingVisitorLy  get; set; 
        public float TotalSaleQtyIncludingVisitorEvol  get; set; 
        public int NewCustomerSaleQty  get; set; 
        public int NewCustomerSaleQtyLy  get; set; 
        public float NewCustomerSaleQtyEvol  get; set; 
        public int OldCustomerSaleQty  get; set; 
        public int OldCustomerSaleQtyLy  get; set; 
        public float OldCustomerSaleQtyEvol  get; set; 
        public int VisitorSaleQty  get; set; 
        public int VisitorSaleQtyLy  get; set; 
        public float VisitorSaleQtyEvol  get; set; 
        public int TotalActiveCustomerQty  get; set; 
        public int TotalActiveCustomerQtyLy  get; set; 
        public float TotalActiveCustomerQtyEvol  get; set; 
        public int NewCustomerQty  get; set; 
        public int NewCustomerQtyLy  get; set; 
        public float NewCustomerQtyEvol  get; set; 
        public int OldCustomerQty  get; set; 
        public int OldCustomerQtyLy  get; set; 
        public float OldCustomerQtyEvol  get; set; 
        public float NewCustomerAchievingRate  get; set; 
        public float AusNotIncludingVisitor  get; set; 
        public float AusNotIncludingVisitorLy  get; set; 
        public float AusEvol  get; set; 
        public float NewCustomerAus  get; set; 
        public float NewCustomerAusLy  get; set; 
        public float NewCustomerAusEvol  get; set; 
        public float OldCustomerAus  get; set; 
        public float OldCustomerAusLy  get; set; 
        public float OldCustomerAusEvol  get; set; 
        public float IptNotIncludingVisitor  get; set; 
        public float IptNotIncludingVisitorLy  get; set; 
        public float IptNotIncludingVisitorEvol  get; set; 
        public float NewCustomerIpt  get; set; 
        public float NewCustomerIptLy  get; set; 
        public float NewCustomerIptEvol  get; set; 
        public float OldCustomerIpt  get; set; 
        public float OldCustomerIptLy  get; set; 
        public float OldCustomerIptEvol  get; set; 
        public int IptEqOneCount  get; set; 
        public int IptEqOneCountLy  get; set; 
        public float IptEqOneCountEvol  get; set; 
        public float PurchaseFreq  get; set; 
        public float PurchaseFreqLy  get; set; 
        public float PurchaseFreqEvol  get; set; 
        public float RecruitmentRate  get; set; 
        public float WechatBindingRate  get; set; 
        public float WechatBindingRateEvol  get; set; 
        public float CompanyWechatBindingRate  get; set; 
        public float CompanyWechatBindingRateEvol  get; set; 
        public float NewCustomerRepurchaseRatePerMonth  get; set; 
        public int NewCustomerRepurchaseQtyPerMonth  get; set; 
        public float NewCustomerRepurchaseAmountPerMonth  get; set; 
        public float NewCustomerRepurchaseAusPerMonth  get; set; 
        public float NewCustomerRepurchaseIptPerMonth  get; set; 
        public float NewCustomerRepurchaseRatePerThreeMonth  get; set; 
        public int NewCustomerRepurchaseQtyPerThreeMonth  get; set; 
        public float NewCustomerRepurchaseAmountPerThreeMonth  get; set; 
        public float NewCustomerRepurchaseAusPerThreeMonth  get; set; 
        public float NewCustomerRepurchaseIptPerThreeMonth  get; set; 
        public float NewCustomerRepurchaseRatePerSixMonth  get; set; 
        public int NewCustomerRepurchaseQtyPerSixMonth  get; set; 
        public float NewCustomerRepurchaseAmountPerSixMonth  get; set; 
        public float NewCustomerRepurchaseAusPerSixMonth  get; set; 
        public float NewCustomerRepurchaseIptPerSixMonth  get; set; 
        public int SkuServiceTimeCount  get; set; 
        public int SkuServicePeopleTimeCount  get; set; 
        public int SkuServicePeopleCount  get; set; 
        public int PurchasePeopleCountAfterService  get; set; 
        public float PurchaseRateAfterService  get; set; 
        public float PurchaseAmountOfPeopleCountAfterService  get; set; 
        public int PotentialCustomer  get; set; 
        public float PotentialCustomerInversionRate  get; set; 
        public float RepurchaseRate  get; set; 
        public float RetentionRate  get; set; 
        public float AnnualSpending  get; set; 
    

【讨论】：

对于StreamReader 忽略IDisposable 可以吗？

是的，你可以。在这里查看乔的评论：***.com/questions/692263/…

这有帮助吗？

我的天蓝色函数是isolated-worker(.NET 5+)，req实际上是HttpRequestData而不是旧api的HttpRequest。会不会受到api变化的影响？

是的，这也适用于 HttpRequestData: docs.microsoft.com/en-us/dotnet/api/… 。更新了我的代码。请检查