Mon 30th Mar 2026

Handling large file upload to a PHP and NodeJS server

Software Architecture Web development Website
Image for blog: Handling large file upload to a PHP and NodeJS server

In this article I go into how you can implement chunked file uploading, this is especially useful when you are dealing with large file uploads.

Background

I had developed a document management site and we came across an issue where one user had a 200MB file to upload, each time they wanted to submit there was always a crash which made the site unusable. This actually led me into looking at the black box of how the file uploading works because I usually go down rabbit holes when something like this occurs.


How the file selection and upload process normally works in the browser

Step 1: Clicking on the file upload button


This makes the browser ‘talk’ with the operating system(windows, mac or linux) to open its native file picker. This lets the user browse around available files and almost always opens where the last uploaded file for a succesful form submission was picked


Step 2: Picking the file


The user then picks a file for upload. Upon doing this the browser creates a file object in its memory with the file’s metadata ie the name, size, type (e.g., image/jpeg), and last modified date. At this point, the whole file isn’t loaded into memory then.


Step 3: Clicking on submit to send the file


Browsers use a standard called multipart/form-data to send the file and it has the following components. Once the submit button is clicked, the browser then prepares the data to be sent. The data will have:

  1. Headers: This also will have a Content-length header to tell the server how big the file is
  2. A Boundary: The browser generates a long, unique string called a boundary. This acts like a digital separator.
  3. A Payload: It wraps the file, and any other input supplied, in a "envelope"


Step 4: Server Processing


Many modern web frameworks (like Express in Node.js, or certain Python/PHP setups) are configured by default to "buffer" the request body(or payload). The server waits until the entire payload has arrived. Once the whole thing is there, it hands it over to the code as one giant variable in memory (RAM that is). This is where the issue was, because we had a 1GB RAM VPS and picture taking 20% of it for one user while still retaining other processes eg the server OS itself, the programming language daemons and some other background tasks.

At the time the budget was tight and resolving this issue for the long term was better than just pooling resources to expand the server’s resource.


Exploring possible solutions:


Direct-to-Cloud Uploads (Presigned URLs)


In this method - instead of sending the large file to your server, your server simply gives the browser a "VIP Pass" (a Presigned URL) to upload directly to a storage provider like AWS S3, Google Cloud Storage, or Azure Blobs.

The Workflow:

  1. Browser asks your server: "Can I upload this 200gb file :) ?"
  2. Server returns a temporary, secure URL.
  3. Browser PUTs the file directly to the cloud provider.

This works because major cloud providers are purpose built to handle massive streams of data. My server never even touches the large file, so it won’t crash. I would just need the link to the file which is a simple string.

I didn’t explore this because of budget reasons though it is really a good option.


TUS Protocol (Resumable Uploads)


I was actually surprised by this and wondered why never came across it earlier. I believe it mostly developed for users with mobile connections or those in public WiFi where the connection can “blink”.

How it works: It breaks the file into a "fingerprint." If the connection drops, the browser asks the server, "How many bytes did you get?" The server says "190MB," and the browser sends only the remaining 10MB.

The Benefit: It’s incredibly stable for mobile users or people with spotty connections which would be a good portion of our users.

At the time I explored its repositiories and I kind of saw it as a new tech and didn’t want to test it further


Chunking the file for upload


In this method, the file is chunked into pieces and sent to the server for processing. Think of a whole unsliced bread vs a sliced bread and the server has to ‘process’ the whole of it. I mean one is easier to get through, right?

The benefit to this is that RAM usage: remains constant regardless of whether the file is 1MB or 1GB. Also it was ,in theory at the time, set to use the programming language constructs of the servers that had been setup so no new library addition yay!


Abstract Implementation

Step 1: The front end HTML


This simply houses the form and the JS required to chunk it

<!-- the form -->
<form action="" method="post" enctype="multipart/form-data">
<input type="file" name="file" id="" />
<button type="button" onsubmit="handleFileUpload()">Submit</button>
<div
style="width:200px;height: 10px; background-color: grey; margin-top: 100px;"
>
<div
style="width:0%; height: 100%; background-color: black;transition: all 0.5s ease-in"
id="progress"
></div>
<span id="progressIndicator">0</span>
</div>
</form>


Step 2: The front end JS


The JavaScript for the front end along with comments for each step

const chunkFileServerURL = "http://localhost:8001/chunk_file";
const fileCombineServerURL = "http://localhost:8001/file";
const fileInput = document.querySelector('input[type="file"]');
// for simplicity reasons I've used the time to get a signature for each request
// in actual implementation you'd need a more robust setup
const signingDateTime = new Date().getTime();
let progress = 0;
/**
* Splices the file into chunks of 512kB and hands over to upload
*/
async function handleFileUpload() {
const file = fileInput.files[0];
// bytes are 1024 to make 1 kilobyte size of each chunk around 500kB per chunk
const chunkSize = 1024 * 512;
let start = 0;
let chunkIndex = 0;
while (start < file.size) {
await uploadChunk(file.slice(start, start + chunkSize), chunkIndex);
start += chunkSize;
chunkIndex += 1;
progress = (start / file.size) * 100;
document.querySelector("#progress").style.width = `${progress}%`;
document.querySelector(
"#progressIndicator"
).innerText = `${progress.toFixed(2)}%`;
}
const extension = file.name.split(".").pop();
await combineChunks(extension);
}
/**
* Uploads the chunk to the specified URL for chunk uploads
*/
async function uploadChunk(chunk, chunkIndex) {
const formData = new FormData();
formData.append(`${signingDateTime}_chunk_${chunkIndex}`, chunk);
await fetch(chunkFileServerURL, {
method: "POST",
body: formData,
});
}
/**
* Sends the signal for combination of the chunks in the server
*/
async function combineChunks(extension) {
document.querySelector("#progress").style.width = `100%`;
document.querySelector("#progressIndicator").innerText = `Combining...`
const formData = new FormData();
formData.append("signing_time", signingDateTime);
formData.append(`file_extension`, extension);
await fetch(fileCombineServerURL, {
method: "POST",
body: formData,
});
document.querySelector("#progressIndicator").innerText = `Uploaded`;
}


Step 3.1: The back end JS


We had a service in Node JS so I had to implement it in node JS using express as below

const express = require('express');
const fileUpload = require('express-fileupload');
const fs = require('fs');
const cors = require('cors');
const app = express();
app.use(cors());
app.use(fileUpload());
const port = 8001;
const CHUNKS_DIRECTORY = './chunks/';
const FILES_DIRECTORY = './files/';
app.post('/chunk_file', (req, res) => {
for (const file of Object.keys(req.files)) {
const sampleFile = req.files[file];
const uploadPath = CHUNKS_DIRECTORY + file;
sampleFile.mv(uploadPath, function (err) {
if (err)
return res.status(500).send(err);
res.send('File uploaded!');
});
}
})
app.post('/file', (req, res) => {
const files = fs.readdirSync(CHUNKS_DIRECTORY);
const signingString = req.body.signing_time;
const extension = req.body.file_extension;
const finalPath = FILES_DIRECTORY + signingString + '.' + extension;
const writeStream = fs.createWriteStream(finalPath);
const matchingFiles = files
.filter((file) => file.startsWith(signingString))
.sort((a, b) => a.localeCompare(b, undefined, { numeric: true }));
// sorted because the order isn't guaranteed
for (const chunkName of matchingFiles) {
const chunkPath = CHUNKS_DIRECTORY + chunkName;
const chunkBuffer = fs.readFileSync(chunkPath);
writeStream.write(chunkBuffer);
fs.unlinkSync(chunkPath);
}
writeStream.end();
writeStream.on('finish', () => {
res.send({ message: 'Stitched successfully via Stream!', path: finalPath });
});
writeStream.on('error', (err) => {
throw err;
})
})
app.listen(port, () => {
console.log(`Example app listening on port ${port}`)
})


Step 3.2: The back end PHP


The other service in the backend doing the uploads was in PHP. This was implemented as below:

// ...
// uploading the chunks to the folder
foreach ($_FILES as $key => $value) {
move_uploaded_file($value["tmp_name"], './chunks/'. $key);
}
// ...
// below is the code to recombine the chunks NOTE: in another file
// ...
$directoryFiles = scandir('./chunks');
// no need to order since scandir automatically orders
$signing = $_POST['signing_time'];
$extension = $_POST['file_extension'];
$filePaths = [];
foreach ($directoryFiles as $fileName) {
if(substr($fileName, 0 , strlen($signing)) == $signing){
array_push($filePaths, $fileName);
}
}
$file = fopen('./files/'.$signing. "." . $extension, 'w');
foreach ($filePaths as $path) {
$data = file_get_contents('./chunks/'. $path);
fwrite($file, $data);
}
fclose($file);
foreach ($filePaths as $path) {
unlink('./chunks/'. $path);
}


Some bonus comments


With the above abstract implementation I was able to develop a system to further perform retries continue from a halted upload by signing the upload and file via local storage and identifying which chunks were remaining.

It was a noticeable improvement and the feedback to users of the upload process was welcomed since earlier it was sort of a black box.


Do you have some more to add or want a few things tweaked? Let me know below

Share on Twitter

Any comment or feedback please share below!

Table Of Contents