如何将 csv 文件从 Internet 加载到 Access 数据库中？

Posted 2023-05-07

【中文标题】如何将 csv 文件从 Internet 加载到 Access 数据库中？

【英文标题】：How to load csv files from the internet into an Access database?

【发布时间】：2013-03-01 15:05:19

【问题描述】：

我有以下包含股票代码的数组：Public Shared tickerArray() As String = "GOOG", "AAPL", "V", "MSFT"。我需要使用这个循环：For Each tickerValue In Form1.tickerArray 将每个股票代码的 csv 文件加载到 Microsoft Access 中的一个大表中。 csv 文件位于"http://ichart.yahoo.com/table.csv?s=" & tickerValue。我还需要将相应的股票代码加载到从 csv 文件导入的表格的每一行中，以使每一行都是唯一的。所以数据库中的列应该是：“Ticker, Date, Open, High, Low, Close, Volumne & Adj Close”。

我找到了有关将本地 csv 文件加载到 Access 的信息，但我似乎无法弄清楚如何通过 vb.net 将 csv 文件从 Internet 加载到 Access。

另外，我需要经常更新这个表，所以我只需要从 csv 文件中插入新的唯一行。没有重复。

有什么建议吗？谢谢！

更新： 这是我到目前为止的代码。

Imports System.Data
Imports System.Data.OleDb
Imports System.Net
Imports System.IO

Public Class Form1

Public Shared tickerArray() As String = "GOOG", "AAPL", "V", "MSFT"

Private Sub btnUpdate_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnUpdate.Click
    Dim myConnection As OleDbConnection
    Dim DBpath As String = 
    Dim sConnectionString As String = "Provider=Microsoft.ACE.OLEDB.12.0; Data Source=" & DBpath & ";Persist Security Info=True"
    myConnection = New OleDbConnection(sConnectionString)
    myConnection.Open()

    Dim strURL As String
    Dim strBuffer As String
    For Each tickerValue In Form1.tickerArray
        'Creates the request URL for Yahoo.
        strURL = "http://ichart.yahoo.com/table.csv?s=" & tickerValue
        strBuffer = RequestWebData(strURL)
        'Create array.
        Dim sReader As New StringReader(strBuffer)
        Dim List As New List(Of String)
        Do While sReader.Peek >= 0
            List.Add(sReader.ReadLine)
        Loop
        Dim lines As String() = List.ToArray
        sReader.Close()
        For Each Line In lines.Skip(1)
            MsgBox(Line)
            Dim myInsert As String = TextToInsert(Line, tickerValue)
            Dim cmd As New OleDbCommand(myInsert, myConnection)
            cmd.ExecuteNonQuery()
        Next
    Next
End Sub

Function TextToInsert(ByVal inputString As String, ByVal ticker As String)
    Dim Dt As String = inputString.Split(",")(0).Trim
    Dim Open As String = inputString.Split(",")(1).Trim
    Dim High As String = inputString.Split(",")(1).Trim
    Dim Low As String = inputString.Split(",")(1).Trim
    Dim Close As String = inputString.Split(",")(1).Trim
    Dim Volume As String = inputString.Split(",")(1).Trim
    Dim Adj_Close As String = inputString.Split(",")(1).Trim
    Dim SQLstr As String
    SQLstr = "INSERT INTO Historical (Ticker, Date, Open, High, Low, Close, Volume, Adj Close) " & "VALUES (" & "'" & ticker & "','" & Dt & "','" & Open & "','" & High & "'," & "'" & Low & "','" & Close & "','" & Volume & "','" & Adj_Close & "'" & ")"
    Return SQLstr
End Function

Private Function RequestWebData(ByVal pstrURL As String) As String
    Dim objWReq As WebRequest
    Dim objWResp As WebResponse
    Dim strBuffer As String
    'Contact the website
    objWReq = HttpWebRequest.Create(pstrURL)
    objWResp = objWReq.GetResponse()
    'Read the answer from the Web site and store it into a stream
    Dim objSR As StreamReader
    objSR = New StreamReader(objWResp.GetResponseStream)
    strBuffer = objSR.ReadToEnd
    objSR.Close()

    objWResp.Close()

    Return strBuffer
End Function

End Class

我收到错误代码“OleDBException 未处理。INSERT INTO 语句中的语法错误。”在线cmd.ExecuteNonQuery()请帮忙！

【问题讨论】：

先将csv文件下载成文件，再将文件加载到数据库中。

@LarsTech 我必须这样做吗？我希望我可以跳过它并直接进入数据库。下载文件似乎是在浪费时间和空间。谢谢。

它不一定是“物理”文件，但您仍然需要下载流，完成后，将这些结果推送到数据库中。这是一个两步过程。

@LarsTech 好的，听起来不错。然后我认为下载流没有问题。 (strURL = "http://ichart.yahoo.com/table.csv?s=" & tickerValuestrBuffer = RequestWebData(strURL)Dim sReader As New StringReader(strBuffer)) 我只是不知道将该文件插入数据库的语法。

日期是关键字，你必须把它放在括号里。另外，查找sql​​注入，因为你有很多。学习使用参数。

【参考方案1】：

您可以使用 System.Data.OleDb 命名空间来定义函数以插入数据库。 类似于（以非常粗略的方式）：

Dim myConnection As OleDbConnection

Dim sConnectionString  As String = "Provider=Microsoft.ACE.OLEDB.12.0; Data Source=" & DBpath & ";Persist Security Info=True"

myConnection = New OleDbConnection(sConnectionString)

myConnection.Open()

然后在csv中的每一行做一个循环

Dim myInsert as String= functionTextToInsert(inputString)

Dim cmd As New OleDbCommand(myInsert, myConnection)

cmd.ExecuteNonQuery()

函数TextToInsert(ByVal inputString as string)是一个函数，它将csv中的一行转换为插入字符串：“max;min;vol”->“插入到MYTABLE（Column_01，Column_02，Column_03）VALUES（最大值， min,vol);"

【讨论】：

这看起来很棒！能否详细介绍一下函数TextToInsert(ByVal inputString as string)？

我在原始帖子中添加了其他信息。我认为我的问题可能与我在 SQL 语句中调用的表有关。该表存在于 access 数据库中，但我不确定我的程序是否能够识别它。

您好，尝试打印插入语句；也许您对插入格式有一些问题：例如，您的值的小数点分隔符错误或缺少顶点；另一个问题可能是数据类型：记录中定义为数据或数字的奇怪格式字符串。

【参考方案2】：

试试这个：

Writing large number of records (bulk insert) to Access in .NET/C#

我从来没有使用过 DAO，所以在这里我帮不上什么忙，但是...

如果此 MSSQL 语法适用于 MS ACCESS，请使用命令对象并将包含您的 csv 作为 varchar(max) 或类似访问数据类型的参数传递给它，这将来自您的应用程序的 CSV 内容：

BULK
INSERT mytable
FROM @myCSVstring
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
GO

有两件事，SO 帖子指出 DAO 快得多，而且您的终结器可能不同，但我会从这些开始。

【参考方案3】：

我认为之前的答案过于复杂。您只需发送XMLHTTP 请求即可。这会将 CSV 数据作为字符串获取，然后可以将其解析为表格。

下面是一个 VBA 子例程。我在哪里找到这些技术的文档在代码 cmets 中。我已经在 Access 2019 中运行了它，它可以工作。

问题中指出的数据的修改可以在插入数据之前或之后进行。

Sub pImportOnlineCsvAsTable(sURL As String, sTableName As String, Optional sKey As String)

' Download a CSV file from the Internet and parse it into an Access table (all strings).

' Adapted from "www.***.com/questions/52142757/vba-download-csv-from-a-link/#52175815"
'   This shows how to download an online CSV file into an ADO stream, then save it as a local file.
' and "www.***.com/questions/33860833/convert-adodb-binary-stream-to-string-vba/#33883789"
'   This shows I don't need the stream, but can use the CSV data as a string and parse it into a table.

' Documentation on "XMLHTTP" object:
'   "https://docs.microsoft.com/en-us/previous-versions/windows/desktop/ms760305(v=vs.85)"

Dim oWinHttpReq As Object
Dim oDB As Database
Dim sSQL As String, sCSVdata As String, _
    asCSVrows As Variant, asCSVfields As Variant, _
    nCSVrows As Long, nCSVfields As Integer, _
    nCountRows As Long, nCountFields As Integer

Set oDB = CurrentDb()
Set oWinHttpReq = CreateObject("Microsoft.XMLHTTP")
oWinHttpReq.Open "get", sURL, False
oWinHttpReq.send
If oWinHttpReq.status = 200 Then
    sCSVdata = oWinHttpReq.ResponseText
    asCSVrows = Split(sCSVdata, Chr(10))
    nCSVrows = UBound(asCSVrows) + 1
    asCSVfields = Split(asCSVrows(0), ",")
    nCSVfields = UBound(asCSVfields) + 1
    sSQL = ""
    For nCountFields = 0 To (nCSVfields - 1)
        sSQL = sSQL & "[" & asCSVfields(nCountFields) & "] varchar(255)" & _
                    IIf(nCountFields = nCSVfields - 1, "", ", ")
      Next
    sSQL = "create table [" & sTableName & "] (" & sSQL & ");"
    oDB.Execute (sSQL)
    For nCountRows = 1 To (nCSVrows - 1)
        asCSVfields = Split(asCSVrows(nCountRows), ",")
        sSQL = ""
        For nCountFields = 0 To (nCSVfields - 1)
            sSQL = sSQL & """" & asCSVfields(nCountFields) & """" & _
                        IIf(nCountFields = nCSVfields - 1, "", ", ")
          Next
        sSQL = "insert into [" & sTableName & "] values (" & sSQL & ");"
        oDB.Execute (sSQL)
      Next
    If sKey <> "" Then _
        oDB.Execute "alter table [" & sTableName & "] add primary key ([" & sKey & "]);"
  End If

Set oWinHttpReq = Nothing
Set oDB = Nothing

End Sub     ' pImportOnlineCsvAsTable()