ICHEC Response to Top500 Media Coverage

ICHEC are concerned by recent coverage in RTE.ie and Silicon Republic about Ireland being ranked as number one in the world in terms of number of Top500 supercomputers per capita and the misleading picture it paints of HPC in Ireland. 

Titles such as “Ireland now ranked first globally in supercomputers per capita” and quotes such as “It is important that Ireland is seen for what it is – a world leader in High Performance Computing” are not only surprising but also misleading and potentially damaging in how Ireland is positioned in the global and European HPC landscape. These articles only reflect that modest systems have been bought by private sector organisations based in Ireland (these are closed systems) and that Ireland has a small population.

In practice, Ireland is in the bottom 20% in Europe in terms of open, publicly-funded computational resources, which is what really matters to publicly-funded researchers and citizens. We have no such systems in the Top 500 since 2014. Irish researchers have similar levels of access to publicly-funded, open HPC resources as their peers in South Africa. This is despite ICHEC being awarded a €5.4 million by Science Foundation Ireland in 2016 for national HPC infrastructure.

The Irish Centre for High-End Computing has campaigned relentlessly to inform Irish policy makers and funders about the importance of High-Performance Computing (HPC), its relevance to the economy and the need to keep up with the rest of the world as the impact of not doing so would damage FDI and job creation, as well as placing Irish Exchequer-funded research at a significant disadvantage compared to their peers. As a result we are concerned that the above statements will undermine our significant campaign of awareness with funders and policy makers. Significant members of the academic users communities have already contacted us to express their dismay and highlight the risk of such portrayal having a significant negative impact on the funding of a well resources HPC infrastructure nationally they rely on for their research. 

Both RTE.ie and Silicon Republic amended the original publications to include a quote from ICHEC. 

Preliminary comments

• The main objective of the TOP500 list is to provide a ranked list of general purpose systems that are in common use for high end applications. It is updated twice a year as part of the International Super Computing (Europe) and SuperComputing (US) conferences. It also provides a comprehensive set of tools to extract sublists and statistics per country, per technology (CPU, interconnect, etc.) as well as per sector.
• While pursuing the highest possible ranking in the Top500 is misguided (as it typically results in unbalanced systems which are less useful to end-users), a per capita analysis provides a good indication of the level of competitiveness a country can offer in relation to its competitors - in other words, it provides a reasonable approximation of the level of processing power available to individuals researchers in country A, compared to their counterparts in countries B or C..
• The number of systems listed in the Top 500 list for any given country is far less significant that their cumulative computational power. Furthermore, the variation in compute power across the list is highly non-linear. E.g. Collectively the top 5 supercomputers deliver more compute power than all 250 supercomputers combined in the bottom half of the list. The most powerful supercomputer delivers 54% more processing power than the entire 100 supercomputers at the bottom of the list (401-500). It is therefore clear that the number of systems in the Top500 constitutes a very poor indicator of the level of processing power “available” in this country.
• Performance is measured through a benchmark called Linpack. The resulting number is called the Rmax, and measured in TFlops/s. It provides a more useful and reliable measure of performance than the peak performance as it is closer to performance levels available to users. It also varies depending on other factors such as interconnect type and topology, RAM, quality of installation (tuning), etc. rather than the CPU’s spec sheet alone.

Methodology

The supporting data provided below were either directly obtained or calculated from the sublist generator at the Top 500 web site referenced earlier.

1. Data for a number of countries in Europe (the Czech Republic, Finland, Germany, the Republic of Ireland, the Netherlands, Norway, Sweden, Switzerland and the United Kingdom) and Asia (Japan, Singapore, South Korea and Taiwan) were extracted from the Top500 sublist generator
2. For each system, we have recorded the measured performance (Rmax) in TFlops/s, the sector (Research, Industry or Other, typically closed Government systems) and verified the type of interconnect (internal network), as it provides a reasonable indicator of the type of use for the system – e.g., 10G Ethernet suggests high throughput applications (traditionally associated with clouds) rather than capability applications (traditionally associated with supercomputing).
3. As Ireland currently has no research supercomputer in the Top 500, we have utilised the measured performance of Fionn (Ireland’s national platform for academic research).

Analysis

As reported in the media, there are indeed four “Irish systems” present in the top 500. Data are supplied below:

The above data show that:

1. These four systems are private industry systems, not available to carry open research – in fact, details of their owners and locations are not disclosed. Those are of no relevance to the Irish research community;
2. The two most powerful systems (ranked #283 and #284) are likely to be utilised for trivially parallel applications / high throughput applications rather than conventional supercomputing workloads, as suggested by their choice of 10G Ethernet interconnect;
3. Ireland has currently no system available to open research in the Top 500. The most powerful system is Fionn, on which the national HPC Service is provided to all Irish HEIs. Fionn was ranked #358 the time of its installation (Nov. 2013). With a measured compute power of 140.4TFlops, its last Top 500 ranking was #476 in June 2014, three and a half year ago.

Similar tables have been prepared for the Czech Republic, Finland, Germany, the Republic of Ireland, Japan, the Netherlands, Norway, Singapore, South Korea, Sweden, Switzerland, Taiwan and the United Kingdom). Raw numbers have been obtained and their per capita and per GDP values calculated. Finally, comparisons have been made in terms of the compute resources available for open research compared to Ireland. The full data are available by request for verification.

* Not in current Top500

The data in the rightmost column indicates an approximation of how many more resources are available to individual researchers in countries listed, compared to their counterparts in Ireland. So for example, a researcher based in Switzerland would have access to 86 times more resources than their colleagues in Ireland. While other examples are less dramatic, we can note that the difference is 10x to 21x for larger countries such as the Japan, the UK or Germany, and remains at 6x to 8x for Scandinavian countries.