The need to share data across different workloads is a constant in IT, but the preferred way of sharing this data has evolved during the last few years. Where most developers would have used file systems (NFS, Samba…) ten years ago, the choice has switched naturally to Object Storage as its features and abstractions were added.
In parallel with this evolution, more and more uses cases c level contact listinvolving publicly sharing data have arisen. The amount of data has exploded, in almost all applications, and IT people don’t want to manage storage anymore (maintenance, failure, scalling…).
As now, all the major languages have their object storage libraries (at least for S3 APIs), and tones of applications integrate by default the connectors to those services. And as many IT people, I switched my reflexes to think Object Storage first, and handle some exceptions where network file systems are a better solution.
When applications need storage performances…
During many years, performances were not the main focus. The goal was to take advantage of unlimited space and public access to store images, documents, code…
Nowadays, part of the applications need more speed because the objects are bigger and plentiful. That’s the case for Big Data use cases and IA scenarios but also for the majority of modern applications.
- Those who work on big data analysis know that you can wait hours before getting your result in many cases. Working on data to analyze users behavior, system deviation, cost optimization requires powered compute resources but also high performance storage. Big Data the fundamentals of resilience at ovhcloud (1/5) clusters needs to access the date quickly to provide a result in time.
- Training IA models and fine tuning the predictions requires a lot of data analysis also that takes hours and slow down your progress if your storage solution is the bottleneck.
- Serving media that are bigger and bigger (4K video, HD pictures on smartphones, …) on high speed network (fiber, 5G) challenge the previous solutions.
OVHcloud Standard Object Storage hosts petabytes of data across the world on a strong and resilient infrastructure but our classical offer was not design to address those use cases. The performances of our classical object storage where not enough in those situations.
A new Object Storage service for demanding applications
So what is high performances for an object storage service? What are we speaking about?
There are mainly two things to consider when you need speed on your objects. The first one is the API. This is the first component that answers the requests you’ll send to the cluster, so the simplest API call should react as a flash. That’s the assurance that all the engine and automation tools on the cluster are optimized and not consuming precious milliseconds. The second one is the bandwidth. You’ll expect that now your object is identified, it should be delivered to you as soon as possible using the maximum capacity of disks and network.
So we are excited to provide OVHcloud High Performance Object Storage that fit those requirements in the upcoming weeks. The general availability should be the 21rst February at first in Strasbourg, then mid-March in Gravelines.
Some numbers
We’ll make some tests and compare that new service with the existing service at OVHcloud and the market leader on Object Storage, AWS S3. We’ll do those tests from OVHcloud aero leads network on a Public Cloud instance B2-120 which have a 10G public interface. This instance is located in Gravelines (north of France). The three tested services are located in central Europe, Strasbourg (France) for OVHcloud ones and Frankfurt (Germany, closed to Strasbourg) for AWS. The distance between the instance and the cluster are almost the same.