缓冲文件(用于更快的磁盘访问)
Posted
技术标签:
【中文标题】缓冲文件(用于更快的磁盘访问)【英文标题】:Buffered files (for faster disk access) 【发布时间】:2011-08-04 02:46:56 【问题描述】:我正在处理大文件并且直接写入磁盘很慢。由于文件很大,我无法将其加载到 TMemoryStream 中。
TFileStream 没有缓冲,所以我想知道是否有可以提供缓冲流的自定义库,或者我应该只依赖操作系统提供的缓冲。操作系统缓冲可靠吗?我的意思是,如果缓存已满,则可能会从缓存中刷新旧文件(我的),以便为新文件腾出空间。
我的文件在 GB 范围内。它包含数百万条记录。不幸的是,这些记录不是固定大小的。因此,我必须进行数百万次读取(4 到 500 个字节之间)。读取(和写入)是顺序的。我不会在文件中上下跳动(我认为这是缓冲的理想选择)。
最后,我必须将这样的文件写回磁盘(同样是数百万次小写入)。
赞美大卫·赫弗南! David 提供了一段很棒的代码来提供缓冲磁盘访问。 人们必须拥有他的 BufferedFileStream!它是黄金。并且不要忘记投票。 谢谢大卫。
Speed tests:
Input file: 317MB.SFF
Delphi stream: 9.84sec
David's stream: 2.05sec
______________________________________
More tests:
Input file: input2_700MB.txt
Lines: 19 millions
Compiler optimization: ON
I/O check: On
FastMM: release mode
**HDD**
Reading: **linear** (ReadLine) (PS: multiply time with 10)
We see clear performance drop at 8KB. Recommended 16 or 32KB
Time: 618 ms Cache size: 64KB.
Time: 622 ms Cache size: 128KB.
Time: 622 ms Cache size: 24KB.
Time: 622 ms Cache size: 32KB.
Time: 622 ms Cache size: 64KB.
Time: 624 ms Cache size: 256KB.
Time: 625 ms Cache size: 18KB.
Time: 626 ms Cache size: 26KB.
Time: 626 ms Cache size: 1024KB.
Time: 626 ms Cache size: 16KB.
Time: 628 ms Cache size: 42KB.
Time: 644 ms Cache size: 8KB. <--- no difference until 8K
Time: 664 ms Cache size: 4KB.
Time: 705 ms Cache size: 2KB.
Time: 791 ms Cache size: 1KB.
Time: 795 ms Cache size: 1KB.
**SSD**
We see a small improvement as we go towards higher buffers. Recommended 16 or 32KB
Time: 610 ms Cache size: 128KB.
Time: 611 ms Cache size: 256KB.
Time: 614 ms Cache size: 32KB.
Time: 623 ms Cache size: 16KB.
Time: 625 ms Cache size: 66KB.
Time: 639 ms Cache size: 8KB. <--- definitively not good with 8K
Time: 660 ms Cache size: 4KB.
______
Reading: **Random** (ReadInteger) (100000 reads)
SSD
Time: 064 ms. Cache size: 1KB. Count: 100000. RAM: 13.27 MB <-- probably the best buffer size for ReadInteger is 4bytes!
Time: 067 ms. Cache size: 2KB. Count: 100000. RAM: 13.27 MB
Time: 080 ms. Cache size: 4KB. Count: 100000. RAM: 13.27 MB
Time: 098 ms. Cache size: 8KB. Count: 100000. RAM: 13.27 MB
Time: 140 ms. Cache size: 16KB. Count: 100000. RAM: 13.27 MB
Time: 213 ms. Cache size: 32KB. Count: 100000. RAM: 13.27 MB
Time: 360 ms. Cache size: 64KB. Count: 100000. RAM: 13.27 MB
Conclusion: don't use it for "random" reading
2021 年更新: 逐个字符读取时,新的 System.Classes.TBufferedFileStream 似乎快了 70%。
【问题讨论】:
内存映射文件? 如果该文件仅由您的应用程序使用,您可以考虑将记录存储在数据库中 【参考方案1】:如果你有这种代码:
while Stream.Position < Stream.Size do
您可以通过将 FileStream.Size 缓存到变量来优化它,它会加快速度。 Stream.Size 使用三个虚函数调用来找出实际大小
【讨论】:
缓慢的是调用 Windows 文件 API 的事实,而不是使用虚拟方法。实现缓存是解决方案。 @ArnaudBouchez,Size 不是简单的虚方法。它调用 kernel32.FileSeek 3 次!添加 Postion(它也调用 FileSeek)——你只需要在一行代码中调用 4 个内核。这比读/写中的内核调用多四倍。 @Alex 确实。这正是我的观点 - 请再次阅读我的评论。这就是为什么我建议缓存大小,甚至在编写内容时更多地计算循环内的当前位置。尽可能避免 API 调用总是一个好主意!【参考方案2】:Windows 文件缓存非常有效,尤其是在您使用 Vista 或更高版本时。 TFileStream
是 Windows ReadFile()
和 WriteFile()
API 函数的松散包装,对于许多用例来说,唯一更快的是内存映射文件。
但是,在一种常见情况下,TFileStream
会成为性能瓶颈。也就是说,如果您在每次调用流读取或写入函数时读取或写入少量数据。例如,如果您一次读取一个整数数组,那么在调用 ReadFile()
时一次读取 4 个字节会产生很大的开销。
同样,内存映射文件是解决这个瓶颈的好方法,但是另一种常用的方法是读取一个更大的缓冲区,比如几千字节,然后在内存缓存中解决未来读取流的问题,而不是进一步致电ReadFile()
。这种方法只适用于顺序访问。
根据您更新的问题中描述的使用模式,我认为您可能会发现以下类会提高您的性能:
unit BufferedFileStream;
interface
uses
SysUtils, Math, Classes, Windows;
type
TBaseCachedFileStream = class(TStream)
private
function QueryInterface(const IID: TGUID; out Obj): HResult; stdcall;
function _AddRef: Integer; stdcall;
function _Release: Integer; stdcall;
protected
FHandle: THandle;
FOwnsHandle: Boolean;
FCache: PByte;
FCacheSize: Integer;
FPosition: Int64;//the current position in the file (relative to the beginning of the file)
FCacheStart: Int64;//the postion in the file of the start of the cache (relative to the beginning of the file)
FCacheEnd: Int64;//the postion in the file of the end of the cache (relative to the beginning of the file)
FFileName: string;
FLastError: DWORD;
procedure HandleError(const Msg: string);
procedure RaiseSystemError(const Msg: string; LastError: DWORD); overload;
procedure RaiseSystemError(const Msg: string); overload;
procedure RaiseSystemErrorFmt(const Msg: string; const Args: array of const);
function CreateHandle(FlagsAndAttributes: DWORD): THandle; virtual; abstract;
function GetFileSize: Int64; virtual;
procedure SetSize(NewSize: Longint); override;
procedure SetSize(const NewSize: Int64); override;
function FileRead(var Buffer; Count: Longword): Integer;
function FileWrite(const Buffer; Count: Longword): Integer;
function FileSeek(const Offset: Int64; Origin: TSeekOrigin): Int64;
public
constructor Create(const FileName: string); overload;
constructor Create(const FileName: string; CacheSize: Integer); overload;
constructor Create(const FileName: string; CacheSize: Integer; Handle: THandle); overload; virtual;
destructor Destroy; override;
property CacheSize: Integer read FCacheSize;
function Read(var Buffer; Count: Longint): Longint; override;
function Write(const Buffer; Count: Longint): Longint; override;
function Seek(const Offset: Int64; Origin: TSeekOrigin): Int64; override;
end;
TBaseCachedFileStreamClass = class of TBaseCachedFileStream;
IDisableStreamReadCache = interface
['0B6D0004-88D1-42D5-BC0F-447911C0FC21']
procedure DisableStreamReadCache;
procedure EnableStreamReadCache;
end;
TReadOnlyCachedFileStream = class(TBaseCachedFileStream, IDisableStreamReadCache)
(* This class works by filling the cache each time a call to Read is made and
FPosition is outside the existing cache. By filling the cache we mean
reading from the file into the temporary cache. Calls to Read when
FPosition is in the existing cache are then dealt with by filling the
buffer with bytes from the cache.
*)
private
FUseAlignedCache: Boolean;
FViewStart: Int64;
FViewLength: Int64;
FDisableStreamReadCacheRefCount: Integer;
procedure DisableStreamReadCache;
procedure EnableStreamReadCache;
procedure FlushCache;
protected
function CreateHandle(FlagsAndAttributes: DWORD): THandle; override;
function GetFileSize: Int64; override;
public
constructor Create(const FileName: string; CacheSize: Integer; Handle: THandle); overload; override;
property UseAlignedCache: Boolean read FUseAlignedCache write FUseAlignedCache;
function Read(var Buffer; Count: Longint): Longint; override;
procedure SetViewWindow(const ViewStart, ViewLength: Int64);
end;
TWriteCachedFileStream = class(TBaseCachedFileStream, IDisableStreamReadCache)
(* This class works by caching calls to Write. By this we mean temporarily
storing the bytes to be written in the cache. As each call to Write is
processed the cache grows. The cache is written to file when:
1. A call to Write is made when the cache is full.
2. A call to Write is made and FPosition is outside the cache (this
must be as a result of a call to Seek).
3. The class is destroyed.
Note that data can be read from these streams but the reading is not
cached and in fact a read operation will flush the cache before
attempting to read the data.
*)
private
FFileSize: Int64;
FReadStream: TReadOnlyCachedFileStream;
FReadStreamCacheSize: Integer;
FReadStreamUseAlignedCache: Boolean;
procedure DisableStreamReadCache;
procedure EnableStreamReadCache;
procedure CreateReadStream;
procedure FlushCache;
protected
function CreateHandle(FlagsAndAttributes: DWORD): THandle; override;
function GetFileSize: Int64; override;
public
constructor Create(const FileName: string; CacheSize, ReadStreamCacheSize: Integer; ReadStreamUseAlignedCache: Boolean); overload;
destructor Destroy; override;
function Read(var Buffer; Count: Longint): Longint; override;
function Write(const Buffer; Count: Longint): Longint; override;
end;
implementation
function GetFileSizeEx(hFile: THandle; var FileSize: Int64): BOOL; stdcall; external kernel32;
function SetFilePointerEx(hFile: THandle; DistanceToMove: Int64; lpNewFilePointer: PInt64; dwMoveMethod: DWORD): BOOL; stdcall; external kernel32;
TBaseCachedFileStream
constructor TBaseCachedFileStream.Create(const FileName: string);
begin
Create(FileName, 0);
end;
constructor TBaseCachedFileStream.Create(const FileName: string; CacheSize: Integer);
begin
Create(FileName, CacheSize, 0);
end;
constructor TBaseCachedFileStream.Create(const FileName: string; CacheSize: Integer; Handle: THandle);
const
DefaultCacheSize = 16*1024;
//16kb - this was chosen empirically - don't make it too large otherwise the progress report is 'jerky'
begin
inherited Create;
FFileName := FileName;
FOwnsHandle := Handle=0;
if FOwnsHandle then begin
FHandle := CreateHandle(FILE_ATTRIBUTE_NORMAL);
end else begin
FHandle := Handle;
end;
FCacheSize := CacheSize;
if FCacheSize<=0 then begin
FCacheSize := DefaultCacheSize;
end;
GetMem(FCache, FCacheSize);
end;
destructor TBaseCachedFileStream.Destroy;
begin
FreeMem(FCache);
if FOwnsHandle and (FHandle<>0) then begin
CloseHandle(FHandle);
end;
inherited;
end;
function TBaseCachedFileStream.QueryInterface(const IID: TGUID; out Obj): HResult;
begin
if GetInterface(IID, Obj) then begin
Result := S_OK;
end else begin
Result := E_NOINTERFACE;
end;
end;
function TBaseCachedFileStream._AddRef: Integer;
begin
Result := -1;
end;
function TBaseCachedFileStream._Release: Integer;
begin
Result := -1;
end;
procedure TBaseCachedFileStream.HandleError(const Msg: string);
begin
if FLastError<>0 then begin
RaiseSystemError(Msg, FLastError);
end;
end;
procedure TBaseCachedFileStream.RaiseSystemError(const Msg: string; LastError: DWORD);
begin
raise EStreamError.Create(Trim(Msg+' ')+SysErrorMessage(LastError));
end;
procedure TBaseCachedFileStream.RaiseSystemError(const Msg: string);
begin
RaiseSystemError(Msg, GetLastError);
end;
procedure TBaseCachedFileStream.RaiseSystemErrorFmt(const Msg: string; const Args: array of const);
var
LastError: DWORD;
begin
LastError := GetLastError; // must call GetLastError before Format
RaiseSystemError(Format(Msg, Args), LastError);
end;
function TBaseCachedFileStream.GetFileSize: Int64;
begin
if not GetFileSizeEx(FHandle, Result) then begin
RaiseSystemErrorFmt('GetFileSizeEx failed for %s.', [FFileName]);
end;
end;
procedure TBaseCachedFileStream.SetSize(NewSize: Longint);
begin
SetSize(Int64(NewSize));
end;
procedure TBaseCachedFileStream.SetSize(const NewSize: Int64);
begin
Seek(NewSize, soBeginning);
if not Windows.SetEndOfFile(FHandle) then begin
RaiseSystemErrorFmt('SetEndOfFile for %s.', [FFileName]);
end;
end;
function TBaseCachedFileStream.FileRead(var Buffer; Count: Longword): Integer;
begin
if Windows.ReadFile(FHandle, Buffer, Count, LongWord(Result), nil) then begin
FLastError := 0;
end else begin
FLastError := GetLastError;
Result := -1;
end;
end;
function TBaseCachedFileStream.FileWrite(const Buffer; Count: Longword): Integer;
begin
if Windows.WriteFile(FHandle, Buffer, Count, LongWord(Result), nil) then begin
FLastError := 0;
end else begin
FLastError := GetLastError;
Result := -1;
end;
end;
function TBaseCachedFileStream.FileSeek(const Offset: Int64; Origin: TSeekOrigin): Int64;
begin
if not SetFilePointerEx(FHandle, Offset, @Result, ord(Origin)) then begin
RaiseSystemErrorFmt('SetFilePointerEx failed for %s.', [FFileName]);
end;
end;
function TBaseCachedFileStream.Read(var Buffer; Count: Integer): Longint;
begin
raise EAssertionFailed.Create('Cannot read from this stream');
end;
function TBaseCachedFileStream.Write(const Buffer; Count: Integer): Longint;
begin
raise EAssertionFailed.Create('Cannot write to this stream');
end;
function TBaseCachedFileStream.Seek(const Offset: Int64; Origin: TSeekOrigin): Int64;
//Set FPosition to the value specified - if this has implications for the
//cache then overriden Write and Read methods must deal with those.
begin
case Origin of
soBeginning:
FPosition := Offset;
soEnd:
FPosition := GetFileSize+Offset;
soCurrent:
inc(FPosition, Offset);
end;
Result := FPosition;
end;
TReadOnlyCachedFileStream
constructor TReadOnlyCachedFileStream.Create(const FileName: string; CacheSize: Integer; Handle: THandle);
begin
inherited;
SetViewWindow(0, inherited GetFileSize);
end;
function TReadOnlyCachedFileStream.CreateHandle(FlagsAndAttributes: DWORD): THandle;
begin
Result := Windows.CreateFile(
PChar(FFileName),
GENERIC_READ,
FILE_SHARE_READ,
nil,
OPEN_EXISTING,
FlagsAndAttributes,
0
);
if Result=INVALID_HANDLE_VALUE then begin
RaiseSystemErrorFmt('Cannot open %s.', [FFileName]);
end;
end;
procedure TReadOnlyCachedFileStream.DisableStreamReadCache;
begin
inc(FDisableStreamReadCacheRefCount);
end;
procedure TReadOnlyCachedFileStream.EnableStreamReadCache;
begin
dec(FDisableStreamReadCacheRefCount);
end;
procedure TReadOnlyCachedFileStream.FlushCache;
begin
FCacheStart := 0;
FCacheEnd := 0;
end;
function TReadOnlyCachedFileStream.GetFileSize: Int64;
begin
Result := FViewLength;
end;
procedure TReadOnlyCachedFileStream.SetViewWindow(const ViewStart, ViewLength: Int64);
begin
if ViewStart<0 then begin
raise EAssertionFailed.Create('Invalid view window');
end;
if (ViewStart+ViewLength)>inherited GetFileSize then begin
raise EAssertionFailed.Create('Invalid view window');
end;
FViewStart := ViewStart;
FViewLength := ViewLength;
FPosition := 0;
FCacheStart := 0;
FCacheEnd := 0;
end;
function TReadOnlyCachedFileStream.Read(var Buffer; Count: Longint): Longint;
var
NumOfBytesToCopy, NumOfBytesLeft, NumOfBytesRead: Longint;
CachePtr, BufferPtr: PByte;
begin
if FDisableStreamReadCacheRefCount>0 then begin
FileSeek(FPosition+FViewStart, soBeginning);
Result := FileRead(Buffer, Count);
if Result=-1 then begin
Result := 0;//contract is to return number of bytes that were read
end;
inc(FPosition, Result);
end else begin
Result := 0;
NumOfBytesLeft := Count;
BufferPtr := @Buffer;
while NumOfBytesLeft>0 do begin
if (FPosition<FCacheStart) or (FPosition>=FCacheEnd) then begin
//the current position is not available in the cache so we need to re-fill the cache
FCacheStart := FPosition;
if UseAlignedCache then begin
FCacheStart := FCacheStart - (FCacheStart mod CacheSize);
end;
FileSeek(FCacheStart+FViewStart, soBeginning);
NumOfBytesRead := FileRead(FCache^, CacheSize);
if NumOfBytesRead=-1 then begin
exit;
end;
Assert(NumOfBytesRead>=0);
FCacheEnd := FCacheStart+NumOfBytesRead;
if NumOfBytesRead=0 then begin
FLastError := ERROR_HANDLE_EOF;//must be at the end of the file
break;
end;
end;
//read from cache to Buffer
NumOfBytesToCopy := Min(FCacheEnd-FPosition, NumOfBytesLeft);
CachePtr := FCache;
inc(CachePtr, FPosition-FCacheStart);
Move(CachePtr^, BufferPtr^, NumOfBytesToCopy);
inc(Result, NumOfBytesToCopy);
inc(FPosition, NumOfBytesToCopy);
inc(BufferPtr, NumOfBytesToCopy);
dec(NumOfBytesLeft, NumOfBytesToCopy);
end;
end;
end;
TWriteCachedFileStream
constructor TWriteCachedFileStream.Create(const FileName: string; CacheSize, ReadStreamCacheSize: Integer; ReadStreamUseAlignedCache: Boolean);
begin
inherited Create(FileName, CacheSize);
FReadStreamCacheSize := ReadStreamCacheSize;
FReadStreamUseAlignedCache := ReadStreamUseAlignedCache;
end;
destructor TWriteCachedFileStream.Destroy;
begin
FlushCache;//make sure that the final calls to Write get recorded in the file
FreeAndNil(FReadStream);
inherited;
end;
function TWriteCachedFileStream.CreateHandle(FlagsAndAttributes: DWORD): THandle;
begin
Result := Windows.CreateFile(
PChar(FFileName),
GENERIC_READ or GENERIC_WRITE,
0,
nil,
CREATE_ALWAYS,
FlagsAndAttributes,
0
);
if Result=INVALID_HANDLE_VALUE then begin
RaiseSystemErrorFmt('Cannot create %s.', [FFileName]);
end;
end;
procedure TWriteCachedFileStream.DisableStreamReadCache;
begin
CreateReadStream;
FReadStream.DisableStreamReadCache;
end;
procedure TWriteCachedFileStream.EnableStreamReadCache;
begin
Assert(Assigned(FReadStream));
FReadStream.EnableStreamReadCache;
end;
function TWriteCachedFileStream.GetFileSize: Int64;
begin
Result := FFileSize;
end;
procedure TWriteCachedFileStream.CreateReadStream;
begin
if not Assigned(FReadStream) then begin
FReadStream := TReadOnlyCachedFileStream.Create(FFileName, FReadStreamCacheSize, FHandle);
FReadStream.UseAlignedCache := FReadStreamUseAlignedCache;
end;
end;
procedure TWriteCachedFileStream.FlushCache;
var
NumOfBytesToWrite: Longint;
begin
if Assigned(FCache) then begin
NumOfBytesToWrite := FCacheEnd-FCacheStart;
if NumOfBytesToWrite>0 then begin
FileSeek(FCacheStart, soBeginning);
if FileWrite(FCache^, NumOfBytesToWrite)<>NumOfBytesToWrite then begin
RaiseSystemErrorFmt('FileWrite failed for %s.', [FFileName]);
end;
if Assigned(FReadStream) then begin
FReadStream.FlushCache;
end;
end;
FCacheStart := FPosition;
FCacheEnd := FPosition;
end;
end;
function TWriteCachedFileStream.Read(var Buffer; Count: Integer): Longint;
begin
FlushCache;
CreateReadStream;
Assert(FReadStream.FViewStart=0);
if FReadStream.FViewLength<>FFileSize then begin
FReadStream.SetViewWindow(0, FFileSize);
end;
FReadStream.Position := FPosition;
Result := FReadStream.Read(Buffer, Count);
inc(FPosition, Result);
end;
function TWriteCachedFileStream.Write(const Buffer; Count: Longint): Longint;
var
NumOfBytesToCopy, NumOfBytesLeft: Longint;
CachePtr, BufferPtr: PByte;
begin
Result := 0;
NumOfBytesLeft := Count;
BufferPtr := @Buffer;
while NumOfBytesLeft>0 do begin
if ((FPosition<FCacheStart) or (FPosition>FCacheEnd))//the current position is outside the cache
or (FPosition-FCacheStart=FCacheSize)//the cache is full
then begin
FlushCache;
Assert(FCacheStart=FPosition);
end;
//write from Buffer to the cache
NumOfBytesToCopy := Min(FCacheSize-(FPosition-FCacheStart), NumOfBytesLeft);
CachePtr := FCache;
inc(CachePtr, FPosition-FCacheStart);
Move(BufferPtr^, CachePtr^, NumOfBytesToCopy);
inc(Result, NumOfBytesToCopy);
inc(FPosition, NumOfBytesToCopy);
FCacheEnd := Max(FCacheEnd, FPosition);
inc(BufferPtr, NumOfBytesToCopy);
dec(NumOfBytesLeft, NumOfBytesToCopy);
end;
FFileSize := Max(FFileSize, FPosition);
end;
end.
【讨论】:
@Remy 感谢您的编辑,我总是把这些函数的名称弄错! 亲爱的耶稣。 Delphi 类:11.2 秒。您的课程:1.6 秒。 @AndriyM 这里根本没有火箭科学。文件缓冲优化早在我出生之前就已经存在,看起来很像这样。事实上,我怀疑这段代码很简单。 @All 感谢您提示我修复此代码,以便它作为独立单元编译。 它有效,但并不十分微妙。从 3 秒到 0.6,因子 5,非常感谢 :-)【参考方案3】:为了大家的利益:Embarcadero 在Delphi 10.1 Berlin的最新版本中添加了TBufferedFileStream
(see the documentation)。
不幸的是,我不能说它与这里给出的解决方案有什么竞争,因为我还没有购买更新。我也知道这个问题是在 Delphi 7 上提出的,但我相信对 Delphi 自己的实现的引用在将来会很有用。
【讨论】:
@david 的缓冲文件流非常好用。但很高兴知道!我现在没有钱(和愿望)来升级,但有一天那门课可能会派上用场。 Delphi 急需这样的课程。 Delphi 中的所有文件访问都非常慢。 +1【参考方案4】:所以 TFileStream 很慢,它从磁盘读取所有内容。 而且 TMemoryStream 不够大。 (如果你这么说)
那么为什么不使用一个 TFileStream 将一个高达 100MB 的块加载到一个 TMemoryStream 中进行处理呢?这可以通过一个简单的预解析器来完成,它只查看数据中的大小标头,但这会恢复您的问题。
让你的代码意识到它的大文件可能出现异常并完全避免这种情况并不是一件坏事:允许它处理来自 TMemoryStream 的(不完整的)块,这也会带来线程增强(硬盘访问不是瓶颈)如果需要。
【讨论】:
仅供参考 Delphi 的 Tmemorystream 也不是太理想,因为它总是以 8k 的增量增长。 FPC 呈指数增长,直到增量达到一定大小。使用经过调整的 FPC 覆盖 Delphi 的 tmemorystream.realloc 可以加快大量写入,因为重新分配更少。 David 的 BufferedFileStream 已经实现了缓冲磁盘访问。【参考方案5】:TFileStream
类在内部使用 CreateFile
函数,该函数始终使用缓冲区来管理文件,除非您指定了 FILE_FLAG_NO_BUFFERING
标志(请注意,您不能直接使用 TFileStream 指定此标志) .为了
更多信息,您可以查看这些链接
Windows File Buffering
您也可以尝试 TGpHugeFileStream
,它是 Primoz Gabrijelcic 的 GpHugeFile
单元的一部分。
【讨论】:
这就是 RRUZ 所暗示的,但在许多情况下它根本不是真的。如果您一次读取小片段,则调用 ReadFile 的开销会变得很大。 @Altar,不,我不是说,我说的是TFileStream
使用缓冲区来保存数据,在大多数情况下都可以。现在,如果您想提高性能,您可以从头开始编写一个对象(类)以使用更大的缓冲区访问和写入文件,或者使用像 TGpHugeFileStream
这样的类。
TFileStream 不使用缓冲区。它只是 ReadFile/WriteFile 的轻量级包装器。 Windows 具有文件缓存,这些 API 例程可以从中受益。
@RRUZ 我不认为这算作讨论中的缓冲区!
@Altar 在现已删除的评论中声明您需要读/写访问权限。对吗?以上是关于缓冲文件(用于更快的磁盘访问)的主要内容,如果未能解决你的问题,请参考以下文章