Recently, I had to implement file uploads in one of my applications. Now, you would probably think… just use
django-storages and be done with it. Yes, but actually no. There is one big problem with the set up of that package: You are always routing the files through your own server. This could be nice for validating files before storing them, but for large files or people with bad internet connections, this could be a major issue. Servers time out easily and bandwitdh is often limited. Extending the time-out period is a bad idea for many reasons, for one being that it is much easier to DDoS a server when the server is working longer on a request.
So, what now? Create presigned urls on the server and then let users upload to that specific URL. AWS S3 has API endpoints for this. It’s fairly easy to set up and takes the heavy lifting from your server. On top of that, it could protect you against path traversal attacks.
Now, I will use AWS S3 for this tutorial, but you can use any provider that supports the full S3 API (Ceph and Min.io should both work fine). There are many reasons to why someone would use AWS, but personally, I try to stay away from them as much as possible.
For this project, I am using a blank Django project with DRF installed. Make sure to install
pip install boto3.
Up next, create a new file called
s3.py and put this class in it:
Also add these things in your settings:
Please do not enter the credentials right into your settings files, but use something like
django-environ to pull them from a
.env file or environment variables.
The class we just created will help us to easily call the boto3 code. We can now do
S3().get_presigned_url('pictures/hello.png') to get an upload endpoint for
pictures/hello.png. Note that it will overwrite items when you upload to the same url twice, so it’s a good idea to add a unique identifier, such as the primairy key or a uuid. Also note that the examples below are very basic and straight to the point. You will probably have to extend it to your needs.
To save files to models, you could create a model like this:
With that, the file would be deleted automatically from S3 once you delete the object. We generate the key automatically based on the
id. This also means that people can’t update the names with the current set up as that would require the file to be moved/renamed.
On the back end side:
You could add extra validators in the serializer to make sure it matches a certain extension (beaware though, validating that won’t block all upload attacks). But I will not get into that too much here.
file_url field will automatically add a temporary file link to that serializer, so you can use it on the front end.