Federation of a distributed infrastructure
The efficient exploitation of the Exabyte–a-day data rates, such as those expected to be generated by the SKA, represents the biggest challenge for ICTs (Information and Communication Technologies) in the world to date. SKA pathfinders will reach the Terabyte-a-day scale, remarkably increasing the leap in the volume and complexity of present astronomical archives. These volumes preclude standard storage and analysis approaches, and need analysis and data mining tasks to live within data repositories, transforming data providers also into services providers, and hence supplying all the individual components of internet-based workflows. This transformation needs the joint contribution of different computing paradigms (HPC clusters, grid computing, or cloud computing), and to investigate how standard communication procedures could integrate the existing heterogeneous network of resources into federated distributed infrastructures for computing and storage, capable of addressing user needs and technical issues.
The AMIGA group not only has the need, but also the technical expertise in Grid and software development to approach this challenge, so that in collaboration with specialized groups providing the complementary skills can lead to a relevant Spanish contribution to the international state of the art. These technical research require the combined expertise of the AMIGA group in 3D data analysis and VO with groups skilled in the federation of HPC and Grid architectures. In this context we are working with the Fundación Centro de Supercomputación Castilla y León (FCSCL), with a proven expertise in the domain of virtualization techniques and cloud computing solutions, as demonstrated e.g. by their contribution to NUBA project. They are leading this ICT research exploring possible issues in network, storage and processing in order to deploy AMIGA4GAS scientific workflows on a heterogeneous distributed infrastructure. This infrastrucrure will integrate Spanish High Performance Computing centers and the IberGrid initiative in a federated infrastructure of heterogeneous resources. The Barcelona Supercomputing Center (BSC) and the Instituto de Telecomunicaçoes in Portugal collaborate in these tasks both with knowledge and infrastructure. The AMIGA group contributes providing migrated workflows to the Grid and technical requirements.
- Web services as building blocks for Science Gateways in Astrophysics. S. Sanchez-Exposito et al. Proceedings of the International Workshop on Science Gateways, June 2015
- EGI Federated Cloud for calibrating and analysing Radio-Astronomy data (Poster) S. Sanchez-Exposito et al. EGI2015.
- Deploying astronomical workflows in heterogeneous distributed computing infrastructures S. Sánchez-Expósito et al. IWSG 2013 – International Workshop on Science Gateways, June 2013
- Workflows and infrastructures in the Exaescale era S. Sánchez-Expósito et al. 6ª Reunión plenaria de la Red Española de e-Ciencia, May 2012
- Towards multidimensional VO archives in the exascale era J.E. Ruiz et al. JENAM 2010
This work is being supported by Grant AYA2008-06181-C02 and AYA2011-30491-C02-01, co-financed by MICINN and FEDER funds, and the Junta de Andalucía (Spain) grant P08-FQM-4205, as well as under the EU 7th Framework Programme in the area of Digital Libraries and Digital Preservation. (ICT-2009.4.1) Project reference: 270192.