Developing a Performance Evaluation Benchmark for Event Sourcing Databases

2024;
: pp. 159 - 168
1
Lviv Polytechnic National University, Software Engineering Department
2
Lviv Polytechnic National University, Software Engineering Department

In the domain of software architecture, Event Sourcing (ES) has emerged as a significant paradigm, especially for systems requiring high levels of auditability, traceability, and intricate state management. Systems such as financial transaction platforms, inventory management systems, customer relationship management (CRM) software, and any application requiring a detailed audit trail can significantly benefit from this approach. Numerous aspects of ES remain unexplored, as they have yet to be thoroughly investigated by scientific research. The unique demands of such systems, particularly in terms of database performance and functionality, are not adequately addressed by existing database benchmarks. By establishing benchmarks, organizations can compare different databases to determine which best meets their needs for applications. This aids in selecting the most appropriate technology based on empirical data rather than assumptions or marketing claims.This paper introduces a novel benchmarking framework specifically designed for evaluating databases in the context of event sourcing. The framework addresses critical aspects unique to ES, including event append performance, efficient handling of Projections (separate databases for read operations), strong consistency, ordered data insertion, and robust versioning controls. Through rigorous testing and analysis, this framework aims to fill the gap in existing database benchmarking tools, providing a more accurate and relevant assessment for ES systems. We also conducted experiments that not only demonstrated the effectiveness of our approach but also yielded meaningful results, substantiating its practicality and applicability.

  1. Alongi, F., Bersani, M. M., Ghielmetti, N., Mirandola, R., & Tamburri, D. A. (2022). Event-sourced, observable software architectures: An experience report. John Wiley & Sons Ltd., 2127–2151. https://doi.org/10.1002/spe.3116.
  2. Overeem, M., Spoor, M., Jansen, S., & Brinkkemper, S. (2021). An Empirical Characterization of Event Sourced Systems and Their Schema Evolution – Lessons from Industry. J. Syst. Softw. https://doi.org/10.1016/j.jss.2021.110970.
  3. Sfaxi, L., & Ben Aissa, M. (2021). Designing and implementing a Big Data benchmark in a financial context: application to a cash management use case. Computing, 103, 1983–2005. https://doi.org/10.1007/s00607- 021-00933-x.
  4. Aluko, V., & Sakr, S. (2019). Big SQL systems: an experimental evaluation. Cluster Computing, 1–31. https://doi.org/10.1007/s10586-019-02914-4.
  5. Bajaber, F., Sakr, S., Batarfi, O., Altalhi, A., & Barnawi, A. (2020). Benchmarking big data systems: A survey. Computer Communications, 149, 241–251. https://doi.org/10.1016/j.comcom.2019.10.002
  6. Han, R., John, L., & Zhan, J. (2017). Benchmarking Big Data Systems: A Review. IEEE Transactions on Services Computing, PP, 1–1. https://doi.org/10.1109/TSC.2017.2730882
  7. Yang, K., Wu, T., Shen, Q., Cui, W., & Zhang, G. (2020). Benchmark Researches from the Perspective of Metrology. https://doi.org/10.1007/978-3-030-49556-5_31
  8. Deari, R., Zenuni, X., Ajdari, J., Ismaili, F., & Raufi, B. (2018). Analysis And Comparision of Document-Based Databases with Relational Databases: MongoDB vs MySQL. 2018 International Conference on Information Technologies (InfoTech), 1–4. https://doi.org/10.1109/InfoTech.2018.8510719.
  9. Khan, W.; Kumar, T.; Zhang, C.; Raj, K.; Roy, A. M.; Luo, B. (2023). SQL and NoSQL Database Software Architecture Performance Analysis and Assessments – A Systematic Literature Review. Big Data Cogn. Comput. 7, 97. https://doi.org/10.3390/bdcc7020097
  10. Mukherjee, S. (2019). The battle between NoSQL Databases and RDBMS. http://dx.doi.org/10.2139/ssrn.3393986
  11. Qu, L., Wang, Q., Chen, T., Li, K., Zhang, R., Zhou, X., … Zhou, A. (2022). Are current benchmarks adequate to evaluate distributed transactional databases? BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 2, 100031. https://doi.org/10.1016/j.tbench.2022.100031.
  12. (n. d.). .NET load testing framework. NBomber. Retrieved March 15, 2024, from https://nbomber.com/
  13. (n. d.). Marten. Martendb. Retrieved March 15, 2024, from https://martendb.io/
  14. (2023, August 16). Event sourcing library for .NET. Nuget.org. Retrieved March 15, 2024, from https://www.nuget.org/packages/NEventStore/
  15. Malyi, R. (2024, February 16). Test project source code. GitHub. Retrieved March 15, 2024, from https://github.com/RomanMalyi/DatabaseComparison