Micron Technology, specializing in DRAM and Flash Mapamy, published The production of the storage engine
HSE is optimized not only to achieve maximum performance, but also to ensure the durability of various classes of SSD drives. The high speed of work is achieved due to the hybrid storage model – the most relevant data are cashed in RAM, which reduces the number of appeals to the drive. The engine can be used for low-level data storage in NOSQL DBMS, software storage facilities (SDS, Software-Defined Storage) type CepH and Scaley Ring, Big Data Processing Platforms, High-Proding Calculations (HPCs), and IOT devices (IoTs) ) and solutions for machine learning systems. As an example of integrating the engine into third-party projects prepared version of the documentary subdivorous DBMS Mongodb, translated to the use of HSE.
The main features of HSE:
- Support for typical and extended operators for data processing in key/value format;
- Full support of transactions with the possibility of insulating the storage cuts through the creation of snapshots (snapshots can also be used to maintain independent collections in one storage);
- Possibility of using cursors to enumulate data in views based on snapshot;
- Data model optimized for mixed load types;
- Flexible mechanisms for managing the reliability of storage;
- custom -made data orchestration schemes (distribution according to various types of memory present in the vault);
- Library with C API, which can dynamically contact any applications. The presence of bindings for Python and Java;
- Support for storing keys and data in compressed form.
- The ability to scale data and hundreds of billions of keys in the storage;
- Effective processing of thousands of parallel operations;
- A significant increase in throughput, decreased delays and strengthening record/reading for various types of load compared to typical alternative solutions;
- The ability to use different classes in one storehouse to optimize performance and extend the duration of the drive.