How to realize large file (hundreds of M or several G) download?

The principle of fragmentation upload is to realize large file upload by splitting large files into small files and breaking the limit of server memory and time. Large file download, in fact, is as wonderful as large file upload,convert scanned pdf to word online free large files which also needs to break the limit of server memory and time.

The reason why uploading and downloading has to break the limit of memory and time is actually related to the buffer. A buffer is an area where data is transferred between devices with different storage speeds or devices with different priorities. Depending on the type of operation, buffers can be categorized into input buffers and output buffers. All input or output content needs to pass through the buffer before subsequent operations can be performed. The use of buffers reduces the number of processes waiting for each other, resulting in better performance.

There are three levels of buffers in PHP, the program buffer, the PHP-fpm buffer and the web server buffer (Nginx/Apache). When we print data using output functions like echo and print, the data flows from the program buffer to the PHP-FPM buffer, the Nginx buffer, and the browser buffer, and by default, each buffer flushes to the next buffer at the same time that the output buffer for that level is filled or the file is closed.

PHP's output buffers are not opened by default. In this case, all input data should be written to disk immediately and all output data should be read from disk immediately. This is because operating the disk is more performance intensive and slower, and for large files there are bound to be memory overflow or timeout problems. So in order to avoid this situation, in the download, we need to open the output buffer bit by bit to read the contents of the file, a little bit of reading after sending a little bit to the client.

PHP output data buffer switch and size settings by php.ini control, which output_buffering control system output a buffer to open and size of the problem settings, implict_flush control technology output as a buffer has been full of their own whether they can automatically refresh to the next layer. Of course, we can also use PHP in the ob series of study function relationship for the choice of manual operation. ob_start is to open the PHP buffer, ob_flush is to refresh the PHP buffer, flush is to refresh the Webserver's buffer, ob_end_flush is to empty and close the PHP buffer.

It should be noted that web servers and browsers have their own output buffers. Typically, the default output_buffering is 4069 characters or more, i.e., the output content must reach 4069 characters before the web can flush the output buffer to the next level. Therefore, to ensure that content can be output from the client immediately after using the flush function, the output content must be equal to or greater than 4096 characters.

By doing the above, we break the memory and time constraints of the server and realize the download of large files.

fragmentation upload PHP FPM

2