Design and Empirical Evaluation of a Network-Centric, On-Premises Architecture for Earth Observation Data Access
2026-08-20 • Distributed, Parallel, and Cluster Computing
Distributed, Parallel, and Cluster ComputingNetworking and Internet ArchitecturePerformance
AI summaryⓘ
The authors studied how to efficiently access large amounts of Earth observation data stored locally, since moving it to the cloud isn't practical for some organizations. They built a system using fast local storage and networks, then tested how network speed affects data access performance. They found that network capacity limits speed up to about 10 Gbps per server, but beyond that, the server's memory setup becomes more important for performance. This means investing in faster networks only helps if the servers have matching memory resources.
Earth observation dataon-premises infrastructurepublic cloudMinIO object storage100 GbE fabricPostGISOGC API-EDRnetwork bandwidthstorage throughputendpoint memory topology
Authors
João Pinelo, João Gonçalves, Denis Willett, Amit Ruhela, Derek Steinmoeller, Uriel Mendoza, Pelumi S. Alao, Ronald Soares Lopes, Rogerio Atem de Carvalho, Pedro Mattos
Abstract
Earth observation (EO) programmes generate data at volumes that exceed the transfer and storage capacity of most institutional networks. Public cloud platforms address this for well-resourced organisations, but institutions across the Atlantic basin face constraints in connectivity, sovereignty and funding that make on-premises infrastructure the only viable path. Cloud-native data formats enable efficient partial reads, yet their performance depends on the bandwidth of the underlying network fabric, a dependency rarely measured in isolation. This paper presents a replicable, network-centric architecture for on-premises EO data access, evaluated at its first operational deployment: the AIR Data Centre, founding node of the Atlantic Cloud. The system comprises a MinIO object storage cluster on a 100 GbE fabric, a PostGIS metadata catalogue and an OGC API-EDR access layer. We characterise the fabric under sustained parallel load, evaluate object storage throughput for EO-representative workloads, and compare measured performance against throttled baselines on identical hardware, isolating network bandwidth as the sole variable. Multi-site replication benchmarks with partner institutions characterise the federation primitive the model depends on. Network bandwidth is the dominant constraint on storage throughput for bulk EO data access up to a threshold; beyond it, endpoint memory topology rather than capacity governs how much bandwidth a system can use. For this hardware class that threshold lies above 10 Gbps per server. Below it, network capacity alone sets what the facility can deliver; above it, the return on further network investment depends on endpoint memory provisioning, which can be deferred and bought later.