Skip to content

Cannot open same S3File instance using different modes #1474

@devmonkey22

Description

@devmonkey22

I'm using S3Storage backend with a Django model with a FileField field. With a model instance with a FileField tied to this storage, I need to be able to open (read) the same file as text and later binary (i.e. to compute hash) from the same model instance. However, Django FileFields (really FieldFiles) keep a reference to the underlying File (aka S3File) even after closing which makes it require being able to reopen with a different mode. When closed, the S3File._file is cleared out, but S3File._mode remains, thus preventing a future open from changing modes before it opens the file again.

Currently I receive the S3File.open error "Cannot reopen file with a new mode." which was added in #1321 to fix #1319 and released in version 1.14.2.

Example Code:

data = "abcdefghijklmnopqrstuvwxyz\n"
model = SomeModel.objects.create()
model.file_contents.save("data.txt", ContentFile(data*2))

# Simulate reading binary (to compute SHA256)
sha_hash = hashlib.new('sha256')
with model.file_contents.open('rb') as f:
	for chunk in f.chunks():
		sha_hash.update(chunk)

# This fails with ValueError: Cannot reopen file with a new mode.
with model.file_contents.open('r') as f:
	line_count = 0
	for line in f.readlines():
		self.assertEqual(line, data)
		line_count += 1
	self.assertEqual(line_count, 2)

This raises:

Traceback (most recent call last):
  File ".../test_models.py", line 233, in test_s3_reopen
    with msg.data.open('r') as f:
  File ".../python3.10/site-packages/django/db/models/fields/files.py", line 81, in open
    self.file.open(mode)
  File ".../python3.10/site-packages/storages/backends/s3.py", line 152, in open
    raise ValueError("Cannot reopen file with a new mode.")
ValueError: Cannot reopen file with a new mode.

I'll push a PR shortly that should resolve this. Let me know if it causes unintended issues allowing S3File to be reopened that I didn't notice.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions