How can I verify a client-provided hash with S3 presigned uploads?
Asked by
MellowQuartz47
I'm building an image upload service that uses S3 presigned URLs so files can go directly from clients to storage instead of passing through my backend. The client currently sends the file size, content type such as image/png, and a file hash that I use for deduplication. However, none of that hash information is trustworthy: the client can submit one arbitrary hash to my upload endpoint and then upload a completely different file through the presigned URL. What's the best way to ensure the uploaded object matches the expected hash without routing the entire file through my backend?
Answers
You can calculate or verify the hash after S3 receives the object. For example, trigger a Lambda from an object-created event, calculate the digest, and compare it with the expected value. You can also use S3’s checksum information or batch checksum computation, then reject or delete objects that fail validation.
Reply to this answer
Require checksums at the storage layer as well. Configure the upload permissions so requests without a checksum header are denied, then tell clients to provide the checksum when using the presigned URL. S3 will validate that the checksum matches the uploaded data, so a client cannot successfully upload different content under a claimed hash.
Reply to this answer
You don’t have to trust a hash submitted to your API as proof of the file contents. Treat it as an expected value, validate the object after upload, and remove or quarantine it if the checksum does not match. This keeps the file transfer direct while preserving a verification step.
Reply to this answer
Include the expected SHA-256 checksum in the presigned PUT request. S3 supports the x-amz-checksum-sha256 header, and it will reject the upload if the uploaded bytes do not match that checksum. The client must calculate the checksum first, send the same value in the signed request, and include the required header when uploading.
Reply to this answer
Add your answer
Answers are reviewed before publication.
LunarPine22: S3 can expose checksum metadata for the object, so you may be able to compare the stored checksum rather than processing the file through your application server.