- Published on
Multer, Amazon S3 and AWS Amplify
- Authors
- Name
- interglobalmedia
- @letsbsocial1
So many of us that were first introduced to file and image uploads using Nodejs, a backend database, probably Expressjs and deploying to Heroku, for example, used the npm package multer to enable us to upload images locally via the client. We soon found out that that alone was not a viable solution on Heroku. We needed to supplement the multer npm package for local uploads with the multer-s3 npm package for uploading images to Amazon Simple Storage Service (aka S3) for storing files on remote.
But it gets better. No one ever seems to mention, but only experience (by checking), the fact that multer-s3, when configured to delete files, only deletes them locally. The remote file on S3 remains.
One can upload and replace images, but the replaced images are not deleted on remote. I found that out the hard way when I had created a user authenticated blog using multer, multer-s3, MongoDB, Nodejs, Expressjs , and hosting it on Heroku. I verified the information by visiting the issues tab for multer-s3 on Github. In addition, If you read through the documentation very carefully, you will notice there is no reference to deleting files or images. Only uploads.
So I started a quest on how I could successfully implement multipart file uploads (which is required) in which files and images could be uploaded on S3 AND deleted permanently via client interface as well. There is nothing worse than a platform that seems to delete a user's image when the user hits the delete button, but it only is deleted locally and on the client, but not on remote hosting. Not good for the user because they wanted it removed, and not good for the owner of the site who does not want to pay extra for unwanted files.
I think I may have just come across at least the beginning of my answer using AWS Amplify, S3 storage via Amplify, and React for the frontend. The post is entitled GraphQL Tutorial - How to Manage Image & File Uploads & Downloads with AWS AppSync & AWS Amplify by Nader Dabit, Twitter handle @dabit3, developer advocate at AWS Cloud. It does not cover dynamically deleting a file from S3, but I do know that there is a way of doing so. The post I came across at least gets me started and bypasses the use of multer-s3, which is the root of my and many others' original problem.
When I went into the Amplify docs under the Storage section, there were sections on PUT, GET, and REMOVE requests for uploading, getting, and removing files. There are also Amplify UI components one can use with React or Angular, as well as a section on customizing the image upload path. To learn more, visit the Amplify Javascript - Storage documentation. It will take more experimentation on my part to achieve this multipart dynamic file deletion goal on AWS. I am just in the beginning stages.
Then one has to beg the question, "If I am using AWS Amplify along with serverless, and therefore also hosting there, perhaps I should look into where using the Amazon native database DynamoDB would work for my application as well instead of MongoDB (in such a use case)?" It might and might not be the right fit in every situation, but it is worth comparing the two.
Whichever platform one chooses to use, one has to weigh the pros and cons of it compared to other comparable platforms, and the pros should outweigh the cons. As I discussed with some people at a serverless event at the AWS Loft NYC last night, no solution is perfect. The question is, "Which platform, given the compromises any may entail, will most encompass the fulfillment of one's needs and budget?".
The event I attended last night was a Jamstack event entitled Serverless with Gatsby and AWS AppSync + Serverless with Vue JS at the AWS Loft NYC. Nader Dabit spoke about GatsbyJS and AWS AppSync, and Divya Sasidharan, Twitter handle @shortdiv and developer advocate at @Netlify, spoke about Serverless, Vuejs, and hosting on @Netlify.
I will be embedding this episode of Plugging in The Holes along with a transcript in the form of a post on interglobalmedianetwork.com for your hearing and reading pleasure. I will be including the related resource links mentioned in the podcast of course. Always do. Bye for now!