A. Khalygov, B. Lysov
https://doi.org/10.15407/elmodel.48.04.028
Èlektron. model. 2026, 48(4):28-46
ABSTRACT
Tools and methods for the automated collection, processing, and integration of data for scientific analytical systems in critical infrastructure are analyzed. A three-level methodology is proposed, which includes: collecting information from official, commercial, and unofficial APIs, web resources, and streaming platforms; preprocessing of data—cleaning, normalization, anonymization, and enrichment through cross-source integration; and the creation of a consolidated knowledge base for further use in predictive models and decision support systems. It has been shown that combining web scraping, streaming technologies (Kafka, Flink, Spark Streaming), and intelligent processing allows for working with heterogeneous structured and unstructured data in near real time. Practical examples from the energy, transportation, and cybersecurity sectors demonstrate the effectiveness of this approach. Future development prospects lie in integrating these methods with machine learning and explainable AI to improve the accuracy and transparency of analytical systems.
KEYWORDS
automated data collection, web scraping, streaming platforms, critical infrastructure, data processing.
REFERENCES
- Lysov, B.S., Hus’kova, V.H., & Khalyhov, A.A. (2025). Application of Data Mining Approaches for Preliminary Analysis of Cyber Threats at Critical Infrastructure Facilities. Scientific Works of Donetsk National Technical University, 1(40), 28-35. Drohobych: DonNTU. ISSN 1996-1588 (Print), ISSN 3083-6050 (Online).
https://doi.org/10.31474/1996-1588-2025-1-40-28-34 - Lysov, B.S., Hus’kova, V.H., Prosian’kina-Zharova, T.I., & Khalyhov, A.A. (2025). Information Technology for Classification of Monitoring Data of Systems and Data Transmission Networks for Cyber Threat Detection. Bulletin of NTU “KhPI”. Series: Computer Science and Modeling, 1(13), 95-108. Kharkiv: NTU “KhPI”. DOI: 10.20998/2411-0558.2025.01.08. ISSN 2079-0031 (Print), ISSN 2411-0558 (Online).
https://doi.org/10.20998/2411-0558.2025.01.08 - Huang, T., Sethu, H., & Kandasamy, N. (2016). A new approach to dimensionality reduction for anomaly detection in data traffic. IEEE Transactions on Network and Service Management, 13(4), 651-665.
https://doi.org/10.1109/TNSM.2016.2597125 - Kudo, T., Morita, T., Matsuda, T., & Takine, T. (2013). PCA-based robust anomaly detection using periodic traffic behavior. In Proceedings of the 2013 IEEE International Conference on Communications Workshops (ICC) (pp. 1-6). Budapest, Hungary, June 9-13.
https://doi.org/10.1109/ICCW.2013.6649443 - Brown, M., Barrington-Leigh, C., & Brown, Z. (2012). Kernel regression for real-time building energy analysis. Journal of Building Performance Simulation, 5(4), 263-276.
https://doi.org/10.1080/19401493.2011.577539 - Abedi, A., Rajkumar, V.S., Ştefanov, A., et al. (2023). Towards real-time distinction of power system faults and cyber attacks. IEEE PESGM 2023, 1-5. https://doi.org/10.1109/ PESGM52003.2023.10253241
https://doi.org/10.1109/PESGM52003.2023.10253241 - Brahma, S., Kavasseri, R., Cao, H., et al. (2016). Real-time identification of dynamic events in power systems using PMU data, and potential applications — models, promises, and challenges. IEEE Transactions on Power Delivery, 32(1), 294–301.
https://doi.org/10.1109/TPWRD.2016.2590961 - Davarifar, M., Rabhi, A., & Hajjaji, A. (2014). Real-time diagnosis of PV systems by using the sequential probability ratio test (SPRT). In Proceedings of the 2014 International Power Electronics and Motion Control Conference (EPE-PEMC) (pp. 508-513). IEEE. https://doi.org/10.1109/EPEPEMC.2014.6980544
https://doi.org/10.1109/EPEPEMC.2014.6980544 - Zhang, Y., Xiang, Y., & Wang, L. (2017). Power system reliability assessment incorporating cyber attacks against wind farm energy management systems. IEEE Transactions on Smart Grid, 8(6), 2343-2357.
https://doi.org/10.1109/TSG.2016.2523515 - Huang, N. E., Shen, Z., Long, S. R., Wu, M. C., Shih, H. H., Zheng, Q., Yen, N.-C., Tung, C. C., & Liu, H. H. (1998). The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 454(1971), 903-995.
https://doi.org/10.1098/rspa.1998.0193 - Yuan, Y., Li, Z., & Ren, K. (2011). Modeling load redistribution attacks in power systems. IEEE Transactions on Smart Grid, 2(2), 382-390.
https://doi.org/10.1109/TSG.2011.2123925 - Bashkov, Y., Dmitrieva, O., & Huskova, N. (2022). Parallel implementation of evolutionary partial differential equations by collocation optical-electronic schemes. Proceedings of SPIE — Photonics Applications in Astronomy, Communications, Industry, and High- Energy Physics Experiments, 12476(1). https://doi.org/10.1117/12.2664488
- Dmytriyeva, O., Huskova, V., & Khalyhov, A. (2024). Multi-adaptive step control in surrogate simulations of difference block compositions of a specified order. In Proceedings of the 14th International Conference on Dependable Systems, Services and Technologies (DESSERT 2024) (pp. 1-7). Athens, Greece. https://doi.org/10.1109/DESSERT65323. 2024.11122213
https://doi.org/10.1109/DESSERT65323.2024.11122213 - Ukraine Electricity Data [Електронний ресурс] // Kaggle. Режим доступу: https://www. kaggle.com/datasets/paradisen/ukraine-electricity/data
- Energy Map Dataset: Ukraine Energy System Data [Електронний ресурс] // Energy Map. Режим доступу: https://energy-map.info/en/datasets/8998f2ed-379f-4fa9-9076-88782b32ee4f
- Electric power consumption (kWh per capita) — Ukraine [Електронний ресурс] // The World Bank. Режим доступу: https://data.worldbank.org/indicator/EG.USE.ELEC.KH. PC?locations=UA
- Ukraine Real-Time Electricity Data Explorer [Електронний ресурс] // International Energy Agency (IEA). Режим доступу: https://www.iea.org/data-and-statistics/data-tools/ukraine-real- time-electricity-data-explorer
Received 12.05.2026