我想写一个分割文件转换成固定大小的块的方法,但我不能excceed的 2147483590 ( Integer.MaxValue极限 - 57 的)每创建缓冲区的原因时,字节
的构造函数只接受一个整数每个块。
我读过其他S.O.建议答案谈到创建小的块。(如:100 MB),然后追加块,以获得真正所需的国标块大小,但我不知道这是否是正确的方法或如何追加的块
Somone的可以帮助我?这里是我做了什么:
公用Sub SplitFile(BYVAL的InputFile作为字符串,
BYVAL CHUNKSIZE长,
可选BYVAL ChunkName作为字符串=没什么,
可选BYVAL ChunkExt作为字符串=没什么,
可选BYVAL覆盖由于布尔= FALSE)
输入文件'的FileInfo实例。
昏暗fInfo作为新IO.FileInfo(与InputFile)
块的总量来创建。
昏暗ChunkCount作为整数= CINT(Math.Floor(fInfo.Length / CHUNKSIZE))
最后一个块的'的其余字节。
昏暗LastChunkSize只要= fInfo.Length - (ChunkCount * CHUNKSIZE)
缓冲区读取数据块。
昏暗ChunkBuffer为字节()=新的字节(CHUNKSIZE - 1L){}
缓冲区读取最后一个区块。
昏暗LastChunkBuffer为字节()=新的字节(LastChunkSize - 1L){}
零填充字符串来枚举块文件。
昏暗零作为字符串=的String.Empty
给定的文件名,每个数据块。
昏暗ChunkFile作为字符串=的String.Empty
该数据块文件名前缀。
ChunkName = IF(String.IsNullOrEmpty(ChunkName)
IO.Path.Combine(fInfo.DirectoryName,IO.Path.GetFileNameWithoutExtension(fInfo.Name)),
IO.Path.Combine(fInfo.DirectoryName,ChunkName))
块文件扩展名。
ChunkExt = IF(String.IsNullOrEmpty(ChunkExt)
fInfo.Extension.Substring(1I)
ChunkExt)
如果CHUNKSIZE大于文件大小则...
如果CHUNKSIZE> = fInfo.Length然后
抛出新发生OverflowException('CHUNKSIZE'应该比文件大小更小。)
退出小组
elseif的CHUNKSIZE> 2147483590I然后'(Integer.MaxValue - 57)
抛出新发生OverflowException('CHUNKSIZE超限。)
退出小组
结束如果CHUNKSIZE<> ...
如果没有文件覆盖是允许的话...
如果不覆盖然后
对于ChunkIndex作为整数= 0I要(ChunkCount)
零=新字符串(0,CStr的(ChunkCount).Length - CStr的(ChunkIndex + 1).Length)
如果块文件已经存在,那么...
如果IO.File.Exists(的String.Format({0} {1} {2}。,ChunkName,零和放大器; CStr的(ChunkIndex + 1I),ChunkExt))然后
抛出新IO.IOException(的String.Format(文件已经存在:{0},ChunkFile))
退出小组
结束如果IO.File.Exists
接下来ChunkIndex
结束如果覆盖
打开文件,开始读取字节。
使用的InputStream作为新IO.FileStream(fInfo.FullName,IO.FileMode.Open)
BinaryReader在使用作为新IO.BinaryReader(InputStream中)
BinaryReader.BaseStream.Seek(0L,IO.SeekOrigin.Begin)
对于ChunkIndex作为整数= 0I要ChunkCount
零=新字符串(0,CStr的(ChunkCount).Length - CStr的(ChunkIndex + 1).Length)
ChunkFile =的String.Format({0} {1} {2}。,ChunkName,零和放大器; CStr的(ChunkIndex + 1I),ChunkExt)
如果ChunkIndex<> ChunkCount然后阅读CHUNKSIZE字节。
InputStream.Position =(CHUNKSIZE * CLng函数(ChunkIndex))
BinaryReader.Read(ChunkBuffer,0I,CHUNKSIZE)
别人的阅读LastChunkSize剩余的字节。
InputStream.Position =(CHUNKSIZE * ChunkIndex)+ 1
BinaryReader.Read(LastChunkBuffer,0I,LastChunkSize)
结束如果ChunkIndex<> ChunkCount
创建块文件写入的字节数。
使用的OutputStream作为新IO.FileStream(ChunkFile,IO.FileMode.Create)
使用的BinaryWriter作为新IO.BinaryWriter(OutputStream中)
如果ChunkIndex<> ChunkCount然后
BinaryWriter.Write(ChunkBuffer)
其他
BinaryWriter.Write(LastChunkBuffer)
结束如果
OutputStream.Flush()
最终使用的BinaryWriter
使用完的OutputStream
报告的进展...
的RaiseEvent ProgressChanged(CDbl((100I / ChunkCount)* ChunkIndex))
接下来ChunkIndex
最终使用BinaryReader在
使用完的InputStream
结束小组
解决方案
正如我已经写在我的评论,你可以只写数据到块,直到它们的尺寸足够大。阅读与较小的缓冲区做(我从你的问题有些code部分)在一个循环中,而计算有多少字节已经被写入。
打开文件,开始读取字节。
使用的InputStream作为新IO.FileStream(fInfo.FullName,IO.FileMode.Open)
BinaryReader在使用作为新IO.BinaryReader(InputStream中)
昏暗OneMegabyte作为整数= 1024 * 1024定义的一个MB长度
帐户,用于其中块大小小于一兆字节被请求的情况下
昏暗BUFFERSIZE作为整数
如果CHUNKSIZE< OneMegabyte然后
BUFFERSIZE = CINT(CHUNKSIZE)
其他
BUFFERSIZE = OneMegabyte
结束如果
昏暗BytesWritten只要= 0'计数当前文件的长度
暗淡ChunkIndex为整数= 0'保持块的数目的轨道
虽然InputStream.Position< InputStream.Length
ChunkFile =的String.Format({0} {1} {2}。,ChunkName,零和放大器; CStr的(ChunkIndex + 1I),ChunkExt)定义文件名
BytesWritten = 0'复位长度计
创建块文件写入的字节数。
使用的OutputStream作为新IO.FileStream(ChunkFile,IO.FileMode.Create)
使用的BinaryWriter作为新IO.BinaryWriter(OutputStream中)
虽然BytesWritten< CHUNKSIZE AndAlso InputStream.Position< InputStream.Length'读,直到你已经达到了输入的末尾
昏暗的ReadBytes()作为字节= BinaryReader.ReadBytes(BUFFERSIZE)读一兆字节
BinaryWriter.Write(的ReadBytes)写这个兆字节
BytesWritten + = ReadBytes.Count'增量大小计数器
结束在
OutputStream.Flush()
最终使用的BinaryWriter
使用完的OutputStream
ChunkIndex + = 1'增量文件柜
结束在
最终使用BinaryReader在
使用完的InputStream
I'm trying to write a method that splits a file into fixed-size chunks, but I can't excceed the limit of 2147483590 (Integer.MaxValue - 57) per each chunk when creating the Buffer 'cause the Byte
constructor only accepts an Integer.
I've read a suggestion in other S.O. answer that talked about creating little chunks (eg: 100 mb) and then append the chunks to get the real desired GB chunk-size, but I don't know whether that's the proper way or how to "append" the chunks.
Somone could help me?, here is what I've done:
Public Sub SplitFile(ByVal InputFile As String,
ByVal ChunkSize As Long,
Optional ByVal ChunkName As String = Nothing,
Optional ByVal ChunkExt As String = Nothing,
Optional ByVal Overwrite As Boolean = False)
' FileInfo instance of the input file.
Dim fInfo As New IO.FileInfo(InputFile)
' The total amount of chunks to create.
Dim ChunkCount As Integer = CInt(Math.Floor(fInfo.Length / ChunkSize))
' The remaining bytes of the last chunk.
Dim LastChunkSize As Long = fInfo.Length - (ChunkCount * ChunkSize)
' The Buffer to read the chunks.
Dim ChunkBuffer As Byte() = New Byte(ChunkSize - 1L) {}
' The Buffer to read the last chunk.
Dim LastChunkBuffer As Byte() = New Byte(LastChunkSize - 1L) {}
' A zero-filled string to enumerate the chunk files.
Dim Zeros As String = String.Empty
' The given filename for each chunk.
Dim ChunkFile As String = String.Empty
' The chunk file basename.
ChunkName = If(String.IsNullOrEmpty(ChunkName),
IO.Path.Combine(fInfo.DirectoryName, IO.Path.GetFileNameWithoutExtension(fInfo.Name)),
IO.Path.Combine(fInfo.DirectoryName, ChunkName))
' The chunk file extension.
ChunkExt = If(String.IsNullOrEmpty(ChunkExt),
fInfo.Extension.Substring(1I),
ChunkExt)
' If ChunkSize is bigger than filesize then...
If ChunkSize >= fInfo.Length Then
Throw New OverflowException("'ChunkSize' should be smaller than the Filesize.")
Exit Sub
' ElseIf ChunkSize > 2147483590I Then ' (Integer.MaxValue - 57)
' Throw New OverflowException("'ChunkSize' limit exceeded.")
' Exit Sub
End If ' ChunkSize <>...
' If not file-overwrite is allowed then...
If Not Overwrite Then
For ChunkIndex As Integer = 0I To (ChunkCount)
Zeros = New String("0", CStr(ChunkCount).Length - CStr(ChunkIndex + 1).Length)
' If chunk file already exists then...
If IO.File.Exists(String.Format("{0}.{1}.{2}", ChunkName, Zeros & CStr(ChunkIndex + 1I), ChunkExt)) Then
Throw New IO.IOException(String.Format("File already exist: {0}", ChunkFile))
Exit Sub
End If ' IO.File.Exists
Next ChunkIndex
End If ' Overwrite
' Open the file to start reading bytes.
Using InputStream As New IO.FileStream(fInfo.FullName, IO.FileMode.Open)
Using BinaryReader As New IO.BinaryReader(InputStream)
BinaryReader.BaseStream.Seek(0L, IO.SeekOrigin.Begin)
For ChunkIndex As Integer = 0I To ChunkCount
Zeros = New String("0", CStr(ChunkCount).Length - CStr(ChunkIndex + 1).Length)
ChunkFile = String.Format("{0}.{1}.{2}", ChunkName, Zeros & CStr(ChunkIndex + 1I), ChunkExt)
If ChunkIndex <> ChunkCount Then ' Read the ChunkSize bytes.
InputStream.Position = (ChunkSize * CLng(ChunkIndex))
BinaryReader.Read(ChunkBuffer, 0I, ChunkSize)
Else ' Read the remaining bytes of the LastChunkSize.
InputStream.Position = (ChunkSize * ChunkIndex) + 1
BinaryReader.Read(LastChunkBuffer, 0I, LastChunkSize)
End If ' ChunkIndex <> ChunkCount
' Create the chunk file to Write the bytes.
Using OutputStream As New IO.FileStream(ChunkFile, IO.FileMode.Create)
Using BinaryWriter As New IO.BinaryWriter(OutputStream)
If ChunkIndex <> ChunkCount Then
BinaryWriter.Write(ChunkBuffer)
Else
BinaryWriter.Write(LastChunkBuffer)
End If
OutputStream.Flush()
End Using ' BinaryWriter
End Using ' OutputStream
' Report the progress...
' RaiseEvent ProgressChanged(CDbl((100I / ChunkCount) * ChunkIndex))
Next ChunkIndex
End Using ' BinaryReader
End Using ' InputStream
End Sub
解决方案
As I have written in my comment you can just write data to the chunks until their size is large enough. Reading is done with a smaller buffer (I took some code parts from your question) in a loop while counting how many bytes have already be written.
' Open the file to start reading bytes.
Using InputStream As New IO.FileStream(fInfo.FullName, IO.FileMode.Open)
Using BinaryReader As New IO.BinaryReader(InputStream)
Dim OneMegabyte As Integer = 1024 * 1024 'Defines length of one MB
'Account for cases where a chunksize smaller than one MegaByte is requested
Dim BufferSize As Integer
If ChunkSize < OneMegabyte Then
BufferSize = CInt(ChunkSize)
Else
BufferSize = OneMegabyte
End If
Dim BytesWritten As Long = 0 'Counts the length of the current file
Dim ChunkIndex As Integer = 0 'Keep track of the number of chunks
While InputStream.Position < InputStream.Length
ChunkFile = String.Format("{0}.{1}.{2}", ChunkName, Zeros & CStr(ChunkIndex + 1I), ChunkExt) 'Define filename
BytesWritten = 0 'Reset length counter
' Create the chunk file to Write the bytes.
Using OutputStream As New IO.FileStream(ChunkFile, IO.FileMode.Create)
Using BinaryWriter As New IO.BinaryWriter(OutputStream)
While BytesWritten < ChunkSize AndAlso InputStream.Position < InputStream.Length 'Read until you have reached the end of the input
Dim ReadBytes() As Byte = BinaryReader.ReadBytes(BufferSize) 'Read one megabyte
BinaryWriter.Write(ReadBytes) 'Write this megabyte
BytesWritten += ReadBytes.Count 'Increment size counter
End While
OutputStream.Flush()
End Using ' BinaryWriter
End Using ' OutputStream
ChunkIndex += 1 'Increment file counter
End While
End Using ' BinaryReader
End Using ' InputStream
相关推荐
最新文章