SQL Server FileStream （转载）

Posted 2020-10-17 PowerCoder

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了SQL Server FileStream （转载）相关的知识，希望对你有一定的参考价值。

从SQL SERVER 2008开始，SQL SERVER引入了一种新的文件组类型叫FileStream文件组，如下图所示：

那么这种文件组是用来做什么的呢？

以往我们对文件管理有两种方法：

数据库只保存文件的路径，具体的文件保存在文件服务器(NFS)上，使用时，编程实现从文件服务器读取文件；
将文件直接以varbinary(max)或image数据类型保存在数据库中。

　　上面两种文件存放方式都有问题：第一种方法因为会访问磁盘，故受I/O影响性能不是很好，而且不能很好的进行文件备份；第二种方法虽然解决了文件备份（数据库的备份）问题，但是由于字段的字节数太大，对数据库本身也会造成影响，性能也很低下。

　　微软在SQL Server 2008推出了一种新的方式 - FileStream，它不是一种新的数据类型，而是一种技术，它使SQL Server数据库引擎和NTFS文件系统成为了一个整体，它结合了上面两种方式的优点：FileStream使用NT系统来缓存文件数据，而对文件数据的操作可使用Transact-SQL语句对其进行插入、更新、查询、搜索和备份。

　　https://msdn.microsoft.com/en-us/library/gg471497(v=sql.110).aspx

　　FILESTREAM integrates the SQL Server Database Engine with an NTFS file system by storing varbinary(max) binary large object (BLOB) data as files on the file system. Transact-SQL statements can insert, update, query, search, and back up FILESTREAM data. Win32 file system interfaces provide streaming access to the data.

　　FILESTREAM uses the NT system cache for caching file data. This helps reduce any effect that FILESTREAM data might have on Database Engine performance. The SQL Server buffer pool is not used; therefore, this memory is available for query processing.

也就是说FileStream文件组是用来代替数据库中的varbinary(max)或image类型，用来存储文件数据的，它可以随着数据库备份和还原，也支持绑定数据库的事务，完美解决了在SQL SERVER数据库中存储二进制文件的难题。

一、FileStream配置

　　1. 配置SQL Server安装实例：Start -> All Programs -> Microsoft SQL Server 2008 R2 -> Configuration Tools -> SQL Server Configuration Manager

　　右击属性，切换到FILESTREAM标签，勾选如下配置

　　2. 打开SQL Server，并配置如下

　　以上也可以通过如下脚本执行：

Exec sp_configure filesteam_access_level, 2
RECONFIGURE

　　最后重启SQL Server Service

二、实例展示

　　创建FileStream类型文件/组

　　FILESTREAM data must be stored in FILESTREAM filegroups. A FILESTREAM filegroup is a special filegroup that contains file system directories instead of the files themselves. These file system directories are called data containers. Data containers are the interface between Database Engine storage and file system storage.

--Create filestreamgroup 
ALTER DATABASE [Archive]
ADD FILEGROUP [FileStreamGroup] CONTAINS FILESTREAM 
GO

--Create filestream and association with filestreamgroup above
ALTER DATABASE [Archive]
ADD FILE ( NAME = N\'FileStream\', FILENAME = N\'D:\\Company\\Data\\SQL Server\\FileStream\') TO FILEGROUP [FileStreamGroup]
GO

　　注意上面将FILEGROUP类型的文件组FileStreamGroup关联到FILENAME时，我们给FILENAME声明的实际上是一个文件夹：FILENAME = N\'D:\\Company\\Data\\SQL Server\\FileStream\'，而不是个具体的文件，该文件夹会存储FileStreamGroup文件组中所有的相关文件，如下图所示：

　　filestream.hdr 文件是 FILESTREAM 容器的头文件。filestream.hdr 文件是重要的系统文件。它包含 FILESTREAM 标头信息。请勿删除或修改此文件。

　　创建测试表（注意：如果表包含FILESTREAM列，则每一行都必须具有唯一的行ID）

--Create table
CREATE TABLE Archive.dbo.Attachment (
    [ID] [UNIQUEIDENTIFIER] ROWGUIDCOL NOT NULL PRIMARY KEY,
    [FileName] NVARCHAR(100) NULL,
    [CreateUser] NVARCHAR(100) NULL,
    [CreateDatetime] DATETIME NULL,
    [Content] VARBINARY(MAX) FILESTREAM NULL  
)
FILESTREAM_ON [FileStreamGroup]

　　什么是rowguidcol关键字

rowguidcol：指定列为全球惟一鉴别行号列（rowguidcol是Row Global UniqueIdentifier Column的缩写）。此列的数据类型必须为UNIQUEIDENTIFIER类型。一个表中数据类型为UNIQUEIDENTIFIER的列中只能有一个列被定义为rowguidcol列。rowguidcol属性不会使列值具有惟一性，也不会自动生成一个新的数据值给插入行。需要在INSERT语句中使用NEWID（）函数或指定列的默认值为NEWID（）函数。

　　插入一些测试数据

--Insert some records
INSERT INTO Attachment VALUES 
(NEWID(),\'File Name 1\',\'shg.cpan\', GETDATE(),NULL),
(NEWID(),\'File Name 1\',\'shg.cpan\', GETDATE(),CAST(\'\' AS VARBINARY(MAX))),
(NEWID(),\'File Name 1\',\'shg.cpan\', GETDATE(),CAST(\'This is a attachment, which contains all introduction for filestream\' AS VARBINARY(MAX)))

　　从前台插入一些数据

using (SqlConnection conn = new SqlConnection("server=10.7.15.172;database=Archive;uid=sa;pwd=1234;Connect Timeout=180"))
{
    conn.Open();
    using (SqlCommand cmd = conn.CreateCommand())
    {
        string id = Guid.NewGuid().ToString();
        cmd.CommandText = "INSERT INTO Attachment VALUES(\'" + id + "\',\'File Name 2\',\'shg.cpan\',\'" + DateTime.Now + "\',@content)";
        SqlParameter param = new SqlParameter("@content", SqlDbType.VarBinary, 1000000000);
        param.Value = File.ReadAllBytes(@"D:\\Folder\\131 u_ex151207.log");
        cmd.Parameters.Add(param);
        cmd.ExecuteNonQuery();
    }
    conn.Close();
}

　　检索数据

SELECT DATALENGTH(CONTENT)/(1024.0 * 1024.0) AS MB,* FROM ATTACHMENT

　　结果

　　文件系统

　　上面的文件都是上传的真实文件，只不过没有后缀，如果重命名加上后缀，即可读取，如最后一个是excel文件，加上.xls，即可用Excel软件打开此文件。

　　下面我们再插入一条记录

INSERT INTO Attachment VALUES 
(NEWID(),\'Win32API\',\'shg.cpan\', GETDATE(),CAST(\'This is a attachment, which contains all introduction for filestream\' AS VARBINARY(MAX)))

　　文件名00000016-0000016e-000c是如何和数据关联的呢

　　DBCC IND (Archive, Attachment, -1)

　　我们看一下PagePID为110的页情况（PageType为1表明是数据页）

DBCC TRACEON (3604);
DBCC PAGE (Archive, 1, 110, 3);
GO

　　看到了什么？CreateLSN即是我们在文件系统中看到的文件名00000016:0000016e:000c，这样数据库中的纪录就和文件联系起来了

三、使用 Win32 管理 FILESTREAM 数据

https://technet.microsoft.com/zh-cn/library/cc645940(v=sql.105).aspx

可以使用 Win32 在 FILESTREAM BLOB 中读取和写入数据。您需要执行以下步骤：

读取 FILESTREAM 文件路径;
读取当前事务上下文;
获取 Win32 句柄，并使用该句柄在 FILESTREAM BLOB 中读取和写入数据

读取 FILESTREAM 文件路径

DECLARE @filePath varchar(max)

SELECT @filePath = Content.PathName()
FROM Archive.dbo.Attachment
WHERE FileName = \'Win32API\'
PRINT @filepath

\\\\CHUNTING-PC\\MSSQLSERVER\\v1\\Archive\\dbo\\Attachment\\%!Content\\583FFDB4-921B-4340-8247-130174488DC8

读取当前事务上下文

DECLARE @txContext varbinary(max)

BEGIN TRANSACTION
SELECT @txContext = GET_FILESTREAM_TRANSACTION_CONTEXT()
PRINT @txContext
COMMIT

0x9D84E776FD943D419C99727C7AAA5B00

获取 Win32 句柄，并使用该句柄在 FILESTREAM BLOB 中读取和写入数据

若要获取 Win32 文件句柄，请调用 OpenSqlFilestream API。此 API 是从 sqlncli.dll 文件中导出的。可以将返回的句柄传递给以下任何 Win32 API：ReadFile、WriteFile、TransmitFile、SetFilePointer、SetEndOfFile 或 FlushFileBuffers。下面的示例说明了如何获取 Win32 文件句柄并使用它在 FILESTREAM BLOB 中读取和写入数据。

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using System.IO;
using System.Linq;
using System.Security.AccessControl;
using System.Text;

namespace FileStreamConsoleApp
{
    class Program
    {
        static void Main(string[] args)
        {
            SqlConnection sqlConnection = new SqlConnection("server=10.7.15.172;database=Archive;Connect Timeout=180;Integrated Security=true");

            SqlCommand sqlCommand = new SqlCommand();
            sqlCommand.Connection = sqlConnection;

            try
            {
                sqlConnection.Open();

                //The first task is to retrieve the file path of the SQL FILESTREAM BLOB that we want to access in the application.
                sqlCommand.CommandText = "SELECT Content.PathName() FROM Archive.dbo.Attachment WHERE FileName = \'Win32API\'";

                String filePath = null;

                Object pathObj = sqlCommand.ExecuteScalar();
                if (DBNull.Value != pathObj)
                {
                    filePath = (string)pathObj;
                }
                else
                {
                    throw new System.Exception("Chart.PathName() failed to read the path name for the Chart column.");
                }

                //The next task is to obtain a transaction context. All FILESTREAM BLOB operations occur within a transaction context to maintain data consistency.

                //All SQL FILESTREAM BLOB access must occur in a transaction. MARS-enabled connections have specific rules for batch scoped transactions,
                //which the Transact-SQL BEGIN TRANSACTION statement violates. To avoid this issue, client applications should use appropriate API facilities for transaction management, 
                //management, such as the SqlTransaction class.

                SqlTransaction transaction = sqlConnection.BeginTransaction("mainTranaction");
                sqlCommand.Transaction = transaction;

                sqlCommand.CommandText = "SELECT GET_FILESTREAM_TRANSACTION_CONTEXT()";//注意这里返回的就是事务上下文，该事务就是上面sqlConnection创建的事务transaction

                Object obj = sqlCommand.ExecuteScalar();
                byte[] txContext = (byte[])obj;//事务上下文是数据库二进制类型


                //The next step is to obtain a handle that can be passed to the Win32 FILE APIs.
                SqlFileStream sqlFileStream = new SqlFileStream(filePath, txContext, FileAccess.ReadWrite);//这里要将事务上下文也传给SqlFileStream，这样该SqlFileStream就和前面创建的数据库事务transaction绑定了

                byte[] buffer = new byte[512];

                int numBytes = 0;

                //Write the string, "EKG data." to the FILESTREAM BLOB. In your application this string would be replaced with the binary data that you want to write.

                string someData = "EKG data.";
                Encoding unicode = Encoding.GetEncoding(0);

                sqlFileStream.Write(unicode.GetBytes(someData.ToCharArray()), 0, someData.Length);

                //Read the data from the FILESTREAM BLOB.

                sqlFileStream.Seek(0L, SeekOrigin.Begin);

                numBytes = sqlFileStream.Read(buffer, 0, buffer.Length);

                string readData = unicode.GetString(buffer);

                if (numBytes != 0)
                {
                    Console.WriteLine(readData);
                }
                //Because reading and writing are finished, FILESTREAM must be closed. This closes the c# FileStream class, 
                //but does not necessarily close the the underlying FILESTREAM handle. 
                sqlFileStream.Close();

                //The final step is to commit or roll back the read and write operations that were performed on the FILESTREAM BLOB.

                sqlCommand.Transaction.Commit();
            }
            catch (System.Exception ex)
            {
                Console.WriteLine(ex.ToString());
            }
            finally
            {
                sqlConnection.Close();
            }
            Console.ReadKey();
        }
    }
}

注意此处的连接方式，必须是integrated security，如果设置成如下

SqlConnection conn = new SqlConnection("server=10.7.15.172;database=Archive;uid=sa;pwd=1234;Connect Timeout=180")

会报错提示权限问题

　　这是因为实际的文件保存在服务器的硬盘上，读取时读到的文件是"\\\\xxxx"，还是通过网上邻居，所以要使用信任连接

Only the account under which the SQL Server service account runs is granted NTFS permissions to the FILESTREAM container. We recommend that no other account be granted permissions on the data container.

Note
SQL logins will not work with FILESTREAM containers. Only NTFS authentication will work with FILESTREAM containers.

另一篇文章 http://stackoverflow.com/questions/1398404/sql-server-filestream-access-denied

插入Attachment的完整C#语句

using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using System.IO;
using System.Transactions;

namespace FileStreamConsoleApp
{
    public class ArchiveAttachment
    {
        private const string ConnStr = "server=10.7.15.172;database=Archive;Connect Timeout=180;Integrated Security=true";

        public static void InsertAttachment(string Id, string fileName, string fileFullName, string createUser, DateTime createDatetime)
        {
            string InsertTSql = @"INSERT INTO Attachment(Id, FileName, CreateUser, CreateDatetime, Content) VALUES(@Id, @FileName, @CreateUser, @CreateDatetime, 0x); 
                                    SELECT Content.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT() FROM Attachment WHERE Id = @Id";

            string serverPath;
            byte[] serverTransaction;

            using (TransactionScope ts = new TransactionScope())
            {
                using (SqlConnection conn = new SqlConnection(ConnStr))
                {
                    conn.Open();

                    using (SqlCommand cmd = new SqlCommand(InsertTSql, conn))
                    {
                        cmd.Parameters.Add("@Id", SqlDbType.NVarChar).Value = Id;
                        cmd.Parameters.Add("@FileName", SqlDbType.NVarChar).Value = fileName;
                        cmd.Parameters.Add("@CreateUser", SqlDbType.NVarChar).Value = createUser;
                        cmd.Parameters.Add("@createDatetime", SqlDbType.DateTime).Value = createDatetime;
                        using (SqlDataReader sdr = cmd.ExecuteReader())
                        {
                            sdr.Read();
                            serverPath = sdr.GetSqlString(0).Value;
                            serverTransaction = sdr.GetSqlBinary(1).Value;
                            sdr.Close();
                        }
                    }
                    SaveAttachment(fileFullName, serverPath, serverTransaction);
                }
                ts.Complete();
            }
        }

        private static void SaveAttachment(string clientPath, string serverPath, byte[] serverTransaction)
        {
            const int BlockSize = 1024 * 512;

            using (FileStream source = new FileStream(clientPath, FileMode.Open, FileAccess.Read))
            {
                using (SqlFileStream dest = new SqlFileStream(serverPath, serverTransaction, FileAccess.Write))
                {
                    byte[] buffer = new byte[BlockSize];
                    int bytesRead;
                    while ((bytesRead = source.Read(buffer, 0, buffer.Length)) > 0)
                    {
                        dest.Write(buffer, 0, bytesRead);
                        dest.Flush();
                    }
                    dest.Close();
                }
                source.Close();
            }
        }
    }
}