使用 C# 将 FileHelpers 数据排序到 SQl 服务器中的不同列中
Posted
技术标签:
【中文标题】使用 C# 将 FileHelpers 数据排序到 SQl 服务器中的不同列中【英文标题】：Sorting FileHelpers data into different columns in SQl sever using C# 【发布时间】：2016-09-23 08:33:33 【问题描述】：我在 C# 中创建了一个代码，它从多个文件夹中读取分隔的 txt 文件，解析数据，然后使用正确的值更新 Microsoft SQL 服务器表。代码如下：
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using FileHelpers;
using System.Data.SqlClient;
using System.IO;
using System.Data;
namespace prototype_using_filehelpers
class ManagerReport
[DelimitedRecord("|")]
[IgnoreEmptyLines()]
[IgnoreFirst()]
public class ManagerReportNames
public int? MASTER_VALUE_ORDER;
public int? MASTER_VALUE;
public string RESORT;
public int? CS_HEADING_COUNT_MASTER;
public int? CS_FS_ARR_ROOMS_MASTER;
public int? CS_FS_DEP_ROOMS_MASTER;
public int? CS_FS_NO_ROOMS_MASTER;
public int? CS_FS_GUESTS_MASTER;
public int? CS_FS_TOTAL_REVENUE_MASTER;
public int? CS_FS_ROOM_REVENUE_MASTER;
public int? CS_FS_INVENTORY_ROOMS_MASTER;
public int? CF_FS_PERC_OCC_ROOMS_MASTER;
public int? CF_FS_AVG_ROOM_RATE_MASTER;
public int? LAST_YEAR_01;
public int? SUB_GRP_1_ORDER;
public string SUB_GRP_1;
public string DESCRIPTION;
public string AMOUNT_FORMAT_TYPE;
public string PRINT_LINE_AFTER_YN;
public int? HEADING_1_ORDER;
public int? HEADING_1;
[FieldOptional]
public string HEADING_2;
[FieldOptional]
[FieldConverter(ConverterKind.Decimal, ".")]
public decimal? SUM_AMOUNT;
[FieldOptional]
public decimal? FORMATTED_AMOUNT;
static void ProcessFilesCSVFiles(string copyPath, string destinationPath)
// first check if path exists
if (!Directory.Exists(copyPath))
// doesn't exist then exit, can't copy from something that doesn't exist
return;
var copyPathDirectory = new DirectoryInfo(copyPath);
// using the SearchOption.AllDirectories will search sub directories
var copyPathCSVFiles = copyPathDirectory.GetFiles("*.txt", SearchOption.AllDirectories);
// loops through directory looking for txt files
for (var i = 0; i < copyPathCSVFiles.Length; i++)
// get the file
var csvFile = copyPathCSVFiles[i];
//sets lines to the files full extention so it can then be called at a later time
string lines = csvFile.FullName;
// read the csv file line by line
FileHelperEngine engine = new FileHelperEngine(typeof(ManagerReportNames));
var records = engine.ReadFile(lines) as ManagerReportNames[];
foreach (var record in records)
Console.WriteLine(record.RESORT);
Console.WriteLine(record.FORMATTED_AMOUNT);
// SQL connection. Creates connection and command and inserts the values taken from the File Helper engine into the SQL table
SqlCommand cmd;
SqlConnection conn;
conn = new SqlConnection("Data Source=hureports01;Initial Catalog=hureports;Integrated Security=True");
conn.Open();
cmd = new SqlCommand("INSERT INTO test2 (property_id,val1) VALUES (@property_id,@val1)", conn);
cmd.Parameters.AddWithValue("@property_id", record.RESORT);
cmd.Parameters.AddWithValue("@val1", record.FORMATTED_AMOUNT);
cmd.ExecuteNonQuery();
// creates a variable that combines the the directory of the new folder with the file name
var destinationFilePath = Path.Combine(destinationPath, csvFile.Name);
// This loop prevents duplicates. If a file is already in the folder, it will delete the file already in there and move this one in.
// Shouldn't be an issue since each file will have a different name
if (File.Exists(destinationFilePath))
File.Delete(destinationFilePath);
// moves it to the new folder
csvFile.MoveTo(destinationFilePath);
static void Main(string[] args)
ProcessFilesCSVFiles(@"C:\Users\Documents", @"C:\Users\Documents\Complete");
代码有效，但我正在尝试实现一项新功能，但我不确定如何实现。目前，它正在使用两个值更新 SQL 服务器：RESORT 和 FORMATTED_AMOUNT。 RESORT 进入表列“property_id”，FORMATTED_AMOUNT 进入 val1。在我的表中，我还添加了 val2、val3 和 val4。 HEADING_2 可以有三个不同的值，或者为空。我将如何修复我的代码，以便当 HEADING_2 是第一个值时，相应的 FORMATTED_AMOUNT 将进入 val1，当 HEADING_2 是第二个值时，相应的 FORMATTED_AMOUNT 将进入 v2，当 HEADING_2 是第三个值时，相应的 FORMATTED_AMOUNT 将进入进入val3，最后当HEADING_2为null时，对应的FORMATTED_AMOUNT会进入val4
当前代码：
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using FileHelpers;
using System.Data.SqlClient;
using System.IO;
using System.Data;
namespace prototype_using_filehelpers
class ManagerReport
[DelimitedRecord("|")]
[IgnoreEmptyLines()]
[IgnoreFirst()]
[IgnoreLast(2)]
public class ManagerReportNames
public int? MASTER_VALUE_ORDER;
public int? MASTER_VALUE;
public string RESORT;
public int? CS_HEADING_COUNT_MASTER;
public int? CS_FS_ARR_ROOMS_MASTER;
public int? CS_FS_DEP_ROOMS_MASTER;
public int? CS_FS_NO_ROOMS_MASTER;
public int? CS_FS_GUESTS_MASTER;
public int? CS_FS_TOTAL_REVENUE_MASTER;
public int? CS_FS_ROOM_REVENUE_MASTER;
public int? CS_FS_INVENTORY_ROOMS_MASTER;
public int? CF_FS_PERC_OCC_ROOMS_MASTER;
public int? CF_FS_AVG_ROOM_RATE_MASTER;
public int? LAST_YEAR_01;
public int? SUB_GRP_1_ORDER;
public string SUB_GRP_1;
public string DESCRIPTION;
public string AMOUNT_FORMAT_TYPE;
public string PRINT_LINE_AFTER_YN;
public int? HEADING_1_ORDER;
public int? HEADING_1;
[FieldOptional]
public string HEADING_2;
[FieldOptional]
[FieldConverter(ConverterKind.Decimal, ".")]
public decimal? SUM_AMOUNT;
[FieldOptional]
public decimal? FORMATTED_AMOUNT;
static void ProcessFilesCSVFiles(string originalPath, string destinationPath)
// first check if path exists
if (!Directory.Exists(originalPath))
// doesn't exist then exit, can't copy from something that doesn't exist
return;
var copyPathDirectory = new DirectoryInfo(originalPath);
// using the SearchOption.AllDirectories will search sub directories
var copyPathCSVFiles = copyPathDirectory.GetFiles("*.txt", SearchOption.AllDirectories);
// loops through directory looking for txt files
for (var i = 0; i < copyPathCSVFiles.Length; i++)
// get the file
var csvFile = copyPathCSVFiles[i];
//sets lines to the files full extention so it can then be called at a later time
string lines = csvFile.FullName;
// read the csv file line by line
FileHelperEngine engine = new FileHelperEngine(typeof(ManagerReportNames));
var records = engine.ReadFile(lines) as ManagerReportNames[];
foreach (var record in records)
Console.WriteLine(record.RESORT);
Console.WriteLine(record.FORMATTED_AMOUNT);
// This allows us to split what column a row goes into based on whether it is a day month or year
string Heading = record.HEADING_2;
string Group = record.SUB_GRP_1;
string column;
if (Heading == "DAY" && Group == "OCC_PERC")
column = "Percent_Rooms_Occupied";
else if (Heading == "DAY" && Group == "OCC_PERC_WO_CH")
column = "Percent_Rooms_Occupied_minus_Comp_and_House";
else if (Heading == "DAY" && Group == "ADR_ROOM")
column = "ADR";
else if (Heading == "DAY" && Group == "ADR_ROOM_WO_CH")
column = "ADR_minus_Comp_and_House";
else if (Heading == "DAY" && Group == "ROOMREV_AVL_ROOMS_MINUS_OOO")
column = "Revenue_per_Available_Room_minus_OOO";
else if (Heading == "DAY" && Group == "TOTAL_REVENUE")
column = "Total_Revenue";
else if (Heading == "DAY" && Group == "ROOM_REVENUE")
column = "Room_Revenue";
else if (Heading == "DAY" && Group == "FOOD_BEV_REVENUE")
column = "Food_And_Beverage_Revenue";
else if (Heading == "DAY" && Group == "OTHER_REVENUE")
column = "Other_Revenue";
else if (Heading == "DAY" && Group == "PHYSICAL_ROOMS")
column = "Total_Rooms_in_Hotel";
else if (Heading == "DAY" && Group == "OCC_ROOMS")
column = "Rooms_Occupied";
else if (Heading == "DAY" && Group == "OCC_MINUS_COMP_HU")
column = "Rooms_Occupied_minus_Comp_and_House_Use";
else if (Heading == "DAY" && Group == "COMP_ROOMS")
column = "Complimentary_Rooms";
else
continue;
// SQL connection. Creates connection and command and inserts the values taken from the File Helper engine into the SQL table
SqlCommand cmd;
SqlConnection conn;
conn = new SqlConnection("Data Source=hureports01;Initial Catalog=hureports;Integrated Security=True");
conn.Open();
//var sqlCommand2 = string.Format(@"IF EXISTS (SELECT * FROM[HEWreport] WHERE Property_ID = @Property_ID) INSERT INTO[HEWreport] (Property_ID, < ColumnName >) VALUES(@Property_ID, @val");
var sqlCommand = string.Format(@"MERGE [HEWreport] AS target USING (select @Property_ID as Property_ID, @val as 0) AS source ON (target.Property_ID = source.Property_ID) WHEN MATCHED THEN UPDATE SET 0= source.0
WHEN NOT MATCHED THEN INSERT (Property_ID, 0) VALUES (source.Property_ID, source.0);", column);
cmd = new SqlCommand(sqlCommand, conn);
cmd.Parameters.AddWithValue("@Property_ID", record.RESORT);
cmd.Parameters.AddWithValue("@val", record.FORMATTED_AMOUNT);
cmd.ExecuteNonQuery();
conn.Close();
// creates a variable that combines the the directory of the new folder with the file name
var destinationFilePath = Path.Combine(destinationPath, csvFile.Name);
// This loop prevents duplicates. If a file is already in the folder, it will delete the file already in there and move this one in.
// Shouldn't be an issue since each file will have a different name
if (File.Exists(destinationFilePath))
File.Delete(destinationFilePath);
// moves it to the new folder
csvFile.MoveTo(destinationFilePath);
static void Main(string[] args)
ProcessFilesCSVFiles(@"C:\Users\Documents\Hotels", @"C:\Users\Documents\Completed Hotels");
如您所见，我完全改变了我想要做的事情，但这是相同的概念。除了定义列是 val1 或 val2，我还需要比 HEADING_2 中的更多，我还需要 SUB_GRP_1。我也尝试过使用参数，因为我还被告知 AddWithValue() 可能会导致问题，但这仍然不起作用
【问题讨论】：
【参考方案1】：假设您有一个将 HEADING_2 映射到列名的函数，如下所示
var columnName = GetColumnNameFromHEADING_2(HEADING_2_value);
您可以将代码更改如下：
var sqlCommand = string.Format("INSERT INTO test2 (property_id, 0) VALUES (@property_id,@val)", columnName);
cmd = new SqlCommand(sqlCommand, conn);
cmd.Parameters.AddWithValue("@property_id", record.RESORT);
cmd.Parameters.AddWithValue("@val", record.FORMATTED_AMOUNT);
如果您想按照评论中的说明合并行，请使用合并语句：
var sqlCommand = string.Format(@"
MERGE [test2] AS target
USING (select @property_id as property_id, @val as 0) AS source
ON (target.property_id = source.property_id)
WHEN MATCHED THEN
UPDATE
SET 0= source.0
WHEN NOT MATCHED THEN
INSERT (property_id, 0)
VALUES (source.property_id, source.0);
", columnName);
【讨论】：
谢谢。我将其添加到我的代码中。我只是想知道是否有办法让 HEADING_2 为 1 时进入 val1，而当 HEADING_2 为 2 时，它进入同一行的 val2。因为我的行有规律。对于 HEADING_2，它先是 1，然后是 2，然后是 3。它通常不为空。但是这三个连续的 HEADING_2 都是相关的。我只是想知道是否有一种方法可以将这三行都放在一个表行中，并在相应列中使用 FORMATTED_AMOUNT 的值 我不断收到此错误：System.Data.dll 中发生“System.Data.SqlClient.SqlException”类型的未处理异常附加信息：参数化查询“（@Property_ID nvarchar（5）， @val nvarchar(4000))MERGE [HEWrepor' 需要参数 '@val'，但未提供。 能否提供您目前拥有的源代码以及您形成的sqlCommand的结果？检查您是否提供了参数cmd.Parameters.AddWithValue("@val", record.FORMATTED_AMOUNT);
我添加了我当前的代码以及我正在尝试修复的内容。如果我将 val 设置为等于特定数字，它会工作并在每一列中打印该数字。它只是不适用于打印 FORMATTED_AMOUNT。我也知道我的文件包含的行中有这些确切的参数，所以它应该返回一个值
所以我能够解决我的问题。在我的文件中，我发现有两行都包含相同的 HEADING_2 和相同的 SUB_GRP_1，除了其中一个为空且一个有值。我只需添加一个参数即可解决此问题。我现在的问题是它只会在一行中读取。所以我可以第一次测试我的代码，它会工作。但是，除非我转到我的 SQL 表并删除该行并再次运行它，否则它将无法再次工作。目标是一次上传多个文件并将它们全部放在不同的行中。我已经实现了多个文件的读入，它仍然以上是关于使用 C# 将 FileHelpers 数据排序到 SQl 服务器中的不同列中的主要内容，如果未能解决你的问题，请参考以下文章