Welcome to the Xceed Community | Help
Community Search  
More Search Options

Realtime file streaming with ASP

Sort Posts: Previous Next
  •  03-18-2005, 9:31 AM Post no. 6236

    Realtime file streaming with ASP

    Hello there !
    I just downloaded the evaluation version of the products, in order to, well ... evaluate if Xceed component suite is the right solution for me. I surfed the documentation but I'm not quite sure I'll be able to do what is needed, so I thought I'd bette ask here. Hope this is the right place :) (I have to compress and stream files, so this place looks ok)

    I'm working on an ASP application, one of its features being the storage of files on the server. Users can then choose any number of files from the web interface and get all these files nicely arranged in a zip file.

    Problem 1 : files on the server don't have the name they should have in the zip file (for example "0001.dat" should be zipped under the name "MyFile.doc") . Is there a way to add a file to a zip file giving at the same time a "friendly" name in the zip ? Of course I'm not able to rename the file "0001.dat" directly on the server ...

    Problem 2 : since the zip files have to be generated "on the fly", this needs time, cpu and ram (a lot for big files), and the browser has to wait till the zip file is actually created before getting the "save as..." box. For huge files, customers won't even wait long enough and call tech support telling that they can't download this vital file they need for yesterday. So question is : is there a way to begin streaming the zip file even though the compression phase is not finished yet ?

    My dream would be something like :
    xZip.AddFile "0001.dat", "RealFileName1"
    xZip.AddFile "0002.dat", "RealFileName2"
    xZip.StreamToClient 'this would begin sending data to the browser immediately

    Hope you can tell me if I'll be able to do something in that spirit with your products
    Thanks
    Sebastien
  •  03-18-2005, 11:18 AM Post no. 6237 in reply to 6236

    Re: Realtime file streaming with ASP

    Hi Seb!

    First, let's make sure we're talking about the same thing here. You posted in the "Xceed Streaming Compression" forum, but what you really want to do is create ZIP files, right? Xceed Streaming Compression doesn't create zip files, it simply compresses data in blobs or files, with simple formatting.

    So, taking in consideration you wish to zip files and send that resulting zip file:

    #1: The ZipPreprocessingFile event is the place to rename files as they are being added to the zip file. You have full power on what path and filename the files will appear to have inside the zip file. Simply change the "sFilename" parameter.

    #2: Xceed Zip ActiveX does not allow zipping in memory, or in a buffer, or in a streamed way. It zips on disk. In order to start sending data right away, you could try doing this: When the request for the zip file occurs, you first send those four bytes: 0x50 0x4B 0x03 0x04. When the zip operation completes, you send all the zip file <b>except</b> the first four bytes. That's somekind of a hack. Are you sure there isn't another way to make that "save as" box appear?

    Too bad you're not using ASP.NET. With Xceed Zip for .NET, you could zip entirely in memory and stream the zip file real easily.
  •  03-22-2005, 4:50 AM Post no. 6238 in reply to 6237

    Re: Realtime file streaming with ASP

    Hi Martin

    Thanks for the quick answer. I realize now that I was not in the good thread. Sorry for that.

    #1 : I didn't look at the events, that's why I missed the renaming part. When you're working in ASP you don't think about events too much : to my knowledge I'm quite sure you can't have events called in ASP VBScript (with WScript you have the ConnectObject method which would allow you to do that, but it's not availaible with ASP). One solution may be to wrap the Zipping part in an ActiveX component that would store all informations in some kind of collection and then handle the events in its context. I have to think about that solution

    #2 : my goal is precisely to avoid using too much memory and CPU. I would even consider using level 0 compression if that allowed me to stream the file directly to the client (that would mean reading for example 1Mb of the first file, send it to the client, redo that until all files have been entirely read). The purpose of zipping in the beginning was first to be able to give our customers a way to get for example 10 files easily without having to click on 10 links, then only to save bandwidth ...

    Otherwise Xceed's components look awesome (especially the ftp/winsock part : I've been doing some network programming lately and I can see the benefits of such libraries). I will definetely look further for you components.

    Thanks,
    Sebastien

    PS : I'd love to be programming with the .NET framework, unfortunately I'm not able to migrate this application to .NET for the moment (it's quite huge and would require A LOT of work ...). That will probably come sooner or later and I look forward to that moment :)
  •  03-22-2005, 8:15 AM Post no. 6239 in reply to 6238

    Re: Realtime file streaming with ASP

    Hi Sebastien!

    Effectively, you will have to wrap the zipping part into an ActiveX to benefit from Xceed Zip events. You can handle events directly in VBScript, but then, you can't set "in/out" parameters that are not variants. You can only read such parameters.

    Using "deflate" versus "store" isn't such a big overhead. It adds about 128kb + 32kb of overhead on processing, and isn't a factor of the file size. But if CPU time is important, use the lowest compression level. It's a good compromise between "bandwidth/time to send" versus "time to process".
  •  03-23-2005, 5:57 AM Post no. 6240 in reply to 6239

    Re: Realtime file streaming with ASP

    Hi !

    Thanks for you answers. In fact I looked at a neat doc from someone who explained how to do kind of what I wanted with php and was looking for a commercial component doing just that, but I misread the listing and in fact I realize now that, in the gzip format at least, the crc has to come *before* the file data and not only *after* ...
    So I believed that when not using any compression, it would have been possible to do something like :
    Send File1 Header (length, name, destination name, etc.)
    Send File1 Data (doing the CRC calculation at the same time)
    Send File1 Footer (CRC, etc.)
    Repeat for N Files

    This would have allowed to save a lot of memory and cpu time. Unfortunately this is not the case (but I'm quite sure there must be a reason for that, I'm no master at designing file formats ;) )

    Thanks anyway,
    Sebastien

  •  03-23-2005, 8:27 AM Post no. 6241 in reply to 6240

    Re: Realtime file streaming with ASP

    Watch out! The CRC16 in the GZip header is the <b>header's</b> crc16. There is a CRC32 at <b>the end</b> of the compressed data. If you choose the GZip format, you do can send the compressed data as it is being compressed.

    BTW, my previous posts assumed you wanted to create a ZIP file, not a GZ file. You could use Xceed Streaming Compression (not Xceed Zip) to send a GZ file as it is being created. However, be aware that some applications do not support multi-file GZ files. They only see the first file in the GZ file.
  •  03-24-2005, 3:32 AM Post no. 6242 in reply to 6241

    Re: Realtime file streaming with ASP

    OK !!
    So much for me, I didn't looked carefully enough at the Streamin Compression documentation. I just saw the "Streaming compression from file to memory (VB example)" now and the ReadFile method looks just like what I'm looking for !!
    I assume that if I want to then send the data I got from ReadFile to the Client I have first to create the header myself, right ?
    I might have more time this afternoon (GMT+1) so I'll dig into that direction at this moment.

    Thanks for the info, I think I'm finally seing the light at the end of the tunnel for that problem ;).
    Sebastien
  •  03-24-2005, 9:25 AM Post no. 6243 in reply to 6242

    Re: Realtime file streaming with ASP

    Actually, forget about the header. The Xceed Streaming Compression Library will take care of this. This library offers support for a few compression formats, like GZip. Here is a sample ASP page that creates a GZ file on the fly. I'm no ASP whiz, so I've added a few shortcuts and it lacks a few features!

    <code><%@ Language=VBScript %>
    <%
    ' I don't know the real ContentType, you'll have to search for it.
    Response.ContentType = "application/zip"

    Dim oComp, oGZip
    Set oComp = CreateObject( "Xceed.StreamingCompression" )
    Set oGZip = CreateObject( "Xceed.GZipCompressionFormat" )

    ' Make sure to license!
    Call oComp.License( "SCO11-GHNSD-BBNAP-W4NA" )

    ' Use the GZip compression format instead of the default "Xceed" format.
    Set oComp.CompressionFormat = oGZip

    ' I'm no good with the VBScript FileSystem, so I've hardcoded two filenames
    ' The third "0" below is the value for "cfpCompress"
    Dim vaData, vaRead

    ' The GZip format offers some header information
    oGZip.FileName = "first.txt"
    vaData = oComp.ReadFile( Server.MapPath( "first.txt" ), 0, 0, 0, True, vaRead )
    Call Response.BinaryWrite( vaData )

    oGZip.FileName = "second.txt"
    vaData = oComp.ReadFile( Server.MapPath( "second.txt" ), 0, 0, 0, True, vaRead )
    Call Response.BinaryWrite( vaData )

    ' I don't know how to rename the "URL", so the "save as" dialog box will suggest
    ' this asp's filename. Just rename to "foo.gz"
    Call Response.End()
    %>
    </code>

    Now, remember: Applications like WinZip or WinRAR don't support multi-file GZ files. And the current version of Xceed Streaming Compression doesn't either support decompressing such files easily. I'm currently working on a fix.
  •  03-24-2005, 12:45 PM Post no. 6244 in reply to 6243

    Re: Realtime file streaming with ASP

    Once again many thanks. I wasn't expecting asp code, that's great.

    For information you juste have to specify :
    Response.AddHeader "Content-Disposition", "attachment;filename=""test.zip"""
    to have a nice "save as" box with "test.zip" as default file name.

    I didn't know multi-files GZ files couldn't be open with WinZip. Netiher does Windows XP's "compressed folder" feature know how to handle GZ files...
    That's too bad.

    So if I sum up my tests, in order to be able to specify each file's name in the compressed file and stream it as it is compressed I have no other choice than using GZip compression format, but I can't create multi-files, which was the original requirement.
    If I use any other compression format I'm not able to create a valid Zip File, and anyways I won't be able to change the file name, which is also very important for me ...

    So much for the light at the end of the tunnel ... :'(

    I assume there must be no way to do what I have to do ...
    What is amazing is that my issue must be a common one (google told me ;) )
    Since the HTTP protocol doesn't allow you to download multiple files at once, you must wrap theses files in some kind of an enveloppe (and zip files seem perfect for that). GMail creates ZIP files too for example. But that nobody has some way to be able to do compression and streaming at the same time is startling.

    I still have one or two more ideas I want to test, I'll let you know if I find something.

    Anyway I wanted to thank you very much for you help which is beyond anything I expected. It's very nice to be able to talk with people who know their business 8)

    Sebastien
View as RSS news feed in XML
Contact | Site Map | Reviews | Legal Terms of Use | Trademarks | Privacy Statement Copyright 2011 Xceed Software Inc.