Leveraging reconfigurable computing in distributed real-time computation systemsLeveraging reconfigurable computing in distributed real-time computation systems Πλήρης Δημοσίευση σε Συνέδριο Conference Full Paper 2018-10-262016enThe community of Big Data processing typically performs realtime computations on data streams with distributed systems such as the Apache Storm. Such systems offer substantial parallelism; however, the communication overhead among nodes for the distribution of the workload places an upper limit to the exploitable parallelism. The contribution of the present work is the integration of a reconfigurable platform with the Apache Storm, which is the main platform of the Big Data streaming processing community. By exploiting the internal bandwidth of FPGAs we show that the computational limits for stream processing are significantly increased vs. conventional distributed processing without compromising on the platform of choice or its seamless operation in a dynamic pipeline. The integration of a Maxeler MPC-C Series platform with the Apache Storm, as presented in detail, yields on the Hayashi-Yoshida correlation algorithm an impressive tenfold increase in real-time streaming input capacity, which corresponds to a hundred-fold computational load. Our methodology is sufficiently general to apply to any class of distributed systems or reconfigurable computers, and this work presents quantitative results of the expected I/O performance, depending on the means of network connection.http://creativecommons.org/licenses/by/4.0/Workshops of the EDBT/ICDT 2016 Joint ConferenceCEUR Workshop Proceedings Nydriotis Apostolos Νυδριωτης Αποστολος Malakonakis Pavlos Μαλακωνακης Παυλος Pavlakis Nikolaos Παυλακης Νικολαος Chrysos Grigorios Χρυσος Γρηγοριος Ioannou Aikaterini Ιωαννου Αικατερινη Sotiriadis Evripidis Σωτηριαδης Ευριπιδης Garofalakis Minos Γαροφαλακης Μινως Dollas Apostolos Δολλας Αποστολος CEUR-WS Distributed computational system High performance computing Hybrid computational platform Multi FPGA platform Storm Streaming big data