Short answer: put the upload path on object storage, keep document metadata and the deletion deadline in your database, and make retention enforcement an observable workflow. For an edtech system handling signed enrollment documents, the decisive comparison is large-file throughput under a deadline, not which storage API looks shortest.
The bill is mostly a byte-retention problem. A useful planning example is 10,000 signed packets at 25 MB each: 250,000 MB remains retained until the deadline, before replicas, temporary upload parts, and downloads are counted. A smaller request fee will not change that dominant term. A lifecycle rule or equivalent expiry policy does.
The trade-off is uncomfortable but clear: keep the original object until its legal or institutional deadline, then remove it and retain only the minimum audit record needed to show what happened. If an investigation arrives after deletion, the bytes are gone by design. That is the cost of honoring a deletion promise, and it should be accepted explicitly rather than hidden in an operator script.
What should the application own, and what should storage own?
The application should own identity, authorization, document state, checksum, retention deadline, and an immutable event trail. Object storage should own the bytes and their transport. That boundary keeps a 500 MB upload out of the relational database while leaving policy decisions queryable.
A row might contain an opaque object key, not a student name or an email address. Store the media type and expected size alongside it. The object key can be generated from a document identifier and a random suffix; it should not expose a course, learner, or signature status in a URL.
The upload sequence needs an intentional state machine: create a pending record, upload to a temporary key, verify size and checksum, promote or copy to the final key, then mark the record available. A failed transfer stays pending and expires separately. Do not let a client choose the final key and then infer completion from the existence of a path.
from dataclasses import dataclass
from datetime import datetime, timezone
from hashlib import sha256
from uuid import uuid4
@dataclass
class UploadPlan:
document_id: str
object_key: str
expires_at: datetime
def plan_upload(document_id: str, expires_at: datetime) -> UploadPlan:
if expires_at <= datetime.now(timezone.utc):
raise ValueError("retention deadline must be in the future")
return UploadPlan(
document_id=document_id,
object_key=f"pending/{document_id}/{uuid4().hex}",
expires_at=expires_at,
)
def checksum(stream) -> str:
digest = sha256()
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
The code does not upload anything by itself. The adapter boundary stays stable across deployment models; changing the underlying object service does not change the retention record. The adapter must still enforce the same post-upload checks.
How should an app compare self-hosted and managed object storage for file uploads?
Yes, but the name on the bucket is only one variable. Throughput is shaped by client concurrency, multipart thresholds, disk layout, network path, TLS termination, CPU available for checksums, and how many copies the system writes. A single giant request is easy to reason about and easy to lose. Multipart transfer gives the client retryable units, but it also creates orphaned parts and a cleanup obligation.
For a 2 GB signed archive, I would begin with a measured multipart policy rather than a universal chunk size: bound the number of simultaneous parts, retry a failed part, and record upload completion separately from application success. The right value comes from a load test that resembles the school network and the deployment topology. A benchmark from a laptop says very little about a regional classroom upload.
The comparison then becomes concrete. A self-hosted object layer gives the team control over disks, placement, upgrades, and network distance, but the team also owns capacity planning, failure recovery, monitoring, and hardware replacement. DigitalOcean Spaces provides a managed object-storage boundary and S3-compatible access, while the operator accepts that placement and service behavior are constrained by the managed offering. Amazon S3 is another managed reference point with documented lifecycle behavior. None of these facts answers the throughput question alone; the test must measure the path your application actually uses.
A fair test uploads the same encrypted fixture through the same client, records time to first byte and completion, injects a failed part, and repeats the run at the intended concurrency. Keep the fixture representative: a signed PDF bundle has a different read pattern from a scanned transcript archive, and a school network with intermittent uplink loss exposes retry behavior that a datacenter benchmark hides. Measure p50 and p95 completion time, retry volume, CPU, and network saturation. Also run a delete-after-deadline test. Storage that uploads quickly but leaves expired documents reachable has failed the real job. This is a test plan, not a vendor score.
| Deployment model | Operational control | Main boundary | Useful fit |
|---|---|---|---|
| Self-hosted object storage | Disks, placement, and network path | Capacity, recovery, upgrades, and hardware | A team that operates storage directly |
| Managed object storage | Provider-operated infrastructure | Placement and service limits | A team that wants a smaller storage-operations surface |
| Database blob storage | One primary data boundary | Database size, backup, and large-transfer pressure | Small objects with simple access patterns |
The table is a decision aid, not a ranking.
Measure it.
How do you make an explicit deletion deadline real?
Treat the deadline as data and as storage policy. The database deadline drives the application view of eligibility; an object-lifecycle rule provides a second enforcement mechanism for the bytes. AWS documents lifecycle management as a way to transition or expire objects, but lifecycle processing should not be presented to users as an exact-time timer. The application should expose a pending-deletion state and verify the result asynchronously.
Keep the policy narrow. Temporary multipart material needs a shorter cleanup window than a signed final document. Final objects need a deletion deadline derived from the retention rule, not from the upload timestamp guessed by a worker. When a deadline changes, record who changed it, why, the old value, and the new value. A deletion worker can then reconcile database rows with object metadata and emit a result for every attempt.
from datetime import datetime, timezone
def deletion_candidate(deadline: datetime, now: datetime | None = None) -> bool:
current = now or datetime.now(timezone.utc)
return deadline <= current
def deletion_event(document_id: str, deadline: datetime, result: str) -> dict:
return {
"event": "document_deletion",
"document_id": document_id,
"deadline": deadline.isoformat(),
"result": result,
"recorded_at": datetime.now(timezone.utc).isoformat(),
}
The event is evidence of an attempted action, not proof that the bytes disappeared. A reconciler should check the object store, tolerate an already-absent object, and alert on an unexpected object after its deadline. Keep the audit event free of document contents and unnecessary personal data.
What fails at the boundary between upload and retention?
The awkward cases deserve more design time than the happy path. A client can disconnect after the last byte but before the application receives confirmation. A retry can create two objects for one document. A database transaction can commit while the final object promotion fails. A deadline can be shortened while a long upload is still running.
Use idempotency keys for the application command, and make object promotion conditional on the expected checksum and size. Reconcile pending records instead of guessing from request logs. Give temporary objects an independent expiry policy. Keep authorization checks in front of both metadata and signed download URLs; a deletion policy is not an access policy.
Encryption deserves the same separation. Encrypt in transit, encrypt stored objects, and keep key access auditable. If a document must be made unreadable before physical deletion, key revocation can be a separate control, but it should not be described as equivalent to deleting the object unless the governing policy says so.
A decision rule I can defend
Choose self-hosting when the team can operate storage failure domains, capacity, upgrades, backups, and network performance as part of its normal service ownership. Choose managed object storage when that operational boundary is more valuable than placement control, and validate the provider's limits and lifecycle semantics with a test fixture. The labels do not remove the work; they move it.
For this edtech use case, I would approve a design only after it can answer five questions in a review: what metadata proves the deadline, what happens to an interrupted multipart upload, how is a 2 GB object retried, how is deletion verified, and which alert fires when an expired object remains? The implementation should fail closed for authorization and fail visibly for retention.
The durable design is therefore modest: a database for policy and state, object storage for bytes, multipart transfer for large files, lifecycle enforcement for expiry, and reconciliation for the gaps between systems. Keep the signed document until its declared deadline. Then delete it, record the result, and accept that auditability means preserving evidence of the action rather than preserving the document forever.
Top comments (0)