I'm looking to set up a process where, when a client uploads a file to S3 using a presigned URL, the file's SHA256 checksum is automatically verified. My workflow involves the client requesting a presigned URL while also sending the SHA256 checksum of the file they intend to upload. After saving this checksum in the database, the server generates the presigned URL using the provided checksum. However, I found out that S3 does not reject uploads of different files than what the checksum indicates. This is disappointing as it means any file can be uploaded, regardless of the checksum. I tried incorporating the x-amz-checksum-sha256 header during upload, but I faced a 'There were headers present in the request which were not signed' error. I'm hoping to find a way to have S3 automatically calculate the SHA256 on upload, so I can retrieve it later with commands like HeadObjectCommand or GetObjectAttributesCommand to verify the upload against the database record. I'm not interested in using the CRC64 checksum that AWS provides.
4 Answers
It sounds like there might be a bigger issue at play here regarding file integrity needs. If you just want assurance that files aren’t tampered with during upload, consider engaging your account team to explore other options. It’s likely there's a better solution than just checksum validation that can enhance your process.
You could technically configure the presigned URL with permissions tied to a specific SHA-256 checksum. This way, if the uploaded file doesn’t match the calculated checksum, S3 will deny the upload. However, clever clients could still find ways around it if they know the rules.
You might want to consider setting up an AWS Lambda function that triggers on file uploads to S3. This way, you can calculate the checksum right after the upload and validate it against what you expect. This isn't a perfect solution and it adds complexity, but it could help enforce checks on uploaded files.
Currently, S3 doesn’t support the use of SHA256 checksums for uploads with presigned URLs. Instead, you should provide an MD5 digest of the file you're uploading, which can be used for integrity verification. For more about object integrity, check the official AWS documentation.
What about the x-amz-content-sha256 header? Does it come with the same limitations? I suspect it might lead to similar signature errors if it doesn't match the payload's SHA256.
Yes, it seems that x-amz-content-sha256 might cause issues as well if it’s not signed correctly. It's a tricky area to navigate!
That's an interesting thought, but it doesn't fully protect against clients uploading incorrect files. They might just need to adjust their approach.