1. CSN
  2. All Volumes and Issues
  3. Volume 6, Number 2, 2024
  4. Evaluation of Efficiency and Performance of Serialization Formats for Distributed Systems

Evaluation of Efficiency and Performance of Serialization Formats for Distributed Systems

CSN.
2024;
Volume 6, Number 2
: pp. 141-157
https://doi.org/10.23939/csn2024.02.141
Authors:
  1. Eduard Maltsev
  2. Oleksandr Muliarevych
1
Lviv Polytechnic National University, Ukraine
2
Lviv Polytechnic National University

The conducted study allows us to evaluate the impact of various serialization formats on the performance of inter-service communication, focusing on serialization speed, data bandwidth efficiency, and latency in environments integrating middleware, characteristic of microservice architectures. Through an empirical analysis of a wide range of serialization formats and comparisons with traditional standards, it is demonstrated that the compactness of serialized data formats is more critical for reducing end-to-end latency than serialization speed itself. Despite high serialization speed, protocols such as FlatBuffers and Cap'n Proto show lower performance in distributed environments due to larger message sizes, in contrast to the more balanced performance observed in protocols like Avro, Thrift, and Protobuf. The purpose of the article is to review existing data formats and message processing and transmission protocols, and through practical experiments, demonstrate the importance of optimizing message sizes to enhance network efficiency and bandwidth capacity.

data encoding
performance evaluation
message transmission protocols
distributed system
data formats
  1. B. Marii, and I. Zholubak, “Features of Development and Analysis of REST Systems,” ACPS, vol. 7, no. 2, pp. 121–129, Dec. 2022, DOI: 10.23939/acps2022.02.121.
  2. S. Weerasinghe and I. Perera, “Optimized Strategy in Cloud-Native Environment for Inter-Service Communication in Microservices,” Int. J. Onl. Eng., vol. 20, no. 01, pp. 40–57, Jan. 2024, DOI: 10.3991/ijoe.v20i01.44021.
  3. D. P. Proos and N. Carlsson, "Performance Comparison of Messaging Protocols and Serialization Formats for Digital Twins in IoV," 2020 IFIP Networking Conference (Networking), Paris, France, 2020, pp. 10-18.
  4. Buono, V., & Petrovic, P. (2021). Enhance Inter-service Communication in Supersonic K-Native REST-based Java Microservice Architectures (Dissertation). url https://urn.kb.se/resolve?urn=urn:nbn:se:hkr:diva-22135
  5. L. Morschel et al., “dCache – Efficient Message Encoding For Inter-Service Communication in dCache: Evaluation of Existing Serialization Protocols as a Replacement for Java Object Serialization,” EPJ Web Conf., vol. 245, p. 05017, 2020, DOI: 10.1051/epjconf/202024505017.
  6. D. Friesel and O. Spinczyk, “Data Serialization Formats for the Internet of Things,” Electronic Communications of the EASST, p. Volume 80: Conference on Networked Systems 2021 (NetSys 2021), Sep. 2021, DOI: 10.14279/TUJ.ECEASST.80.1134.
  7. Á. Luis, P. Casares, J. J. Cuadrado-Gallego, and M. A. Patricio, “PSON: A Serialization Format for IoT Sensor Networks,” Sensors, vol. 21, no. 13, p. 4559, Jul. 2021, DOI: 10.3390/s21134559.
  8. J. C. Viotti and M. Kinderkhedia, “A Survey of JSON-compatible Binary Serialization Specifications.” arXiv, Jan. 10, 2022. DOI: 10.48550/arXiv.2201.02089.
  9. P. K. Kumar, R. Agarwal, R. Shivaprasad, D. Sitaram, and S. Kalambur, “Performance Characterization of Communication Protocols in Microservice Applications,” in 2021 International Conference on Smart Applications, Communications and Networking (SmartNets), Glasgow, United Kingdom: IEEE, Sep. 2021, pp. 1–5. DOI: 10.1109/SmartNets50376.2021.9555425.
  10. J. C. Viotti and M. Kinderkhedia, “Benchmarking JSON BinPack,” 2022, DOI: 10.48550/ARXIV.2211.12799.
  11. B. Huang and Y. Tang, “Research on optimization of real-time efficient storage algorithm in data information serialization,” PLoS ONE, vol. 16, no. 12, p. e0260697, Dec. 2021, DOI: 10.1371/journal.pone.0260697.
  12. T. Ahmad, Z. A. Ars, and H. P. Hofstee, “Benchmarking Apache Arrow Flight -- A wire-speed protocol for data transfer, querying and microservices.” arXiv, Apr. 08, 2022. DOI: 10.48550/arXiv.2204.03032.
  13. A. B. Dauda, M. S. Adam, M. A. Mustapha, A. M. Mabu, and S. Mustafa, “Soap serialization effect on communication nodes and protocols,” 2020, DOI: 10.48550/ARXIV.2012.12578.
  14. D. Evans, “Energy-Efficient Transaction Serialization for IoT Devices,” j. of Comput. sci. res., vol. 2, no. 2, pp. 1–16, May 2020, DOI: 10.30564/jcsr.v2i2.1620.
  15. J. C. Viotti and M. Kinderkhedia, “A Benchmark of JSON-compatible Binary Serialization Specifications,” 2022, DOI: 10.48550/ARXIV.2201.03051.
  16. Protocol Buffers Version 3 Language Specification. Accessed: Feb. 20, 2024. [Online]. Available: https://protobuf.dev/reference/protobuf/proto3-spec/
  17. C. Currier, “Protocol Buffers,” in Mobile Forensics – The File Format Handbook: Common File Formats and File Systems Used in Mobile Devices, C. Hummert and D. Pawlaszczyk, Eds., Cham: Springer International Publishing, 2022, pp. 223–260. DOI: 10.1007/978-3-030-98467-0_9.
  18. X. Wang and Z. Xie, “The Case For Alternative Web Archival Formats To Expedite The Data-To-Insight Cycle,” in Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020, in JCDL ’20. New York, NY, USA: Association for Computing Machinery, Aug. 2020, pp. 177–186. DOI: 10.1145/3383583.3398542.
  19. T. Li, H. Shi, and X. Lu, “HatRPC: hint-accelerated thrift RPC over RDMA,” in Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, in SC ’21. New York, NY, USA: Association for Computing Machinery, Nov. 2021, pp. 1–14. DOI: 10.1145/3458817.3476191.
  20. Sorokin, K., "Benchmark comparing various data serialization libraries," [Online]. Available: https://github.com/thekvs/cpp-serializers. [Accessed: March 1, 2024].
  21. J. C. Hamerski, A. R. P. Domingues, F. G. Moraes and A. Amory, "Evaluating Serialization for a Publish-Subscribe Based Middleware for MPSoCs," 2018 25th IEEE International Conference on Electronics, Circuits and Systems (ICECS), Bordeaux, France, 2018, pp. 773-776, DOI: 10.1109/ICECS.2018.8618003.
  22. J. Peltenburg, Á. Hadnagy, M. Brobbel, R. Morrow, and Z. Al-Ars, “Tens of gigabytes per second JSON-to-Arrow conversion with FPGA accelerators,” in 2021 International Conference on Field-Programmable Technology (ICFPT), Dec. 2021, pp. 1–9. DOI: 10.1109/ICFPT52863.2021.9609833.