Mind the Gap – Examining an Enterprise IT Mindset in a Research Environment

With high-performance computing (HPC) increasingly becoming an integral cog in research settings, be that academic, research or the world of technology and design, the requirement for the right knowledge and swift support of such systems has become paramount.

There are many factors at play in such environments that can impact HPC performance and return on investment (ROI), budgets, the cloud v on-premise debate, resource, and a perceived shortage in skills among them. Here, Owen Thomas, co-founder of Red Oak Consulting, examines how an enterprise IT mindset within a research environment could be damaging to operations, and crucially, impact the very outcomes HPC is designed to achieve.  

Research departments, particularly within higher education, encounter a range of challenges, from navigating budgetary constraints, software licensing, and integrating and managing diverse software and hardware components, often across a huge footprint.  In overcoming these challenges and providing an alternative to more established information systems in a cost-effective manner, the initial appeal of cloud computing is generally accepted by most research and higher education establishments as attractive, citing low entry costs, elasticity, standardisation, and low carbon amongst its benefits.

At the same time, the lack of familiarisation and knowledge of cloud HPC can attract concerns around vendor lock-in, reliability, lack of skills, insufficient support of cloud service providers, policies on the cloud, privacy, and the complexity of cloud technologies. In addition, security concerns loom large, as institutions must ensure data confidentiality and integrity, not least where internationally significant intellectual property is involved. Compliance with relevant regulations is paramount too, as is the perceived reliability of cloud infrastructure and services as they clearly impact long-term operational sustainability.

However, a scarcity of cloud-savvy personnel, experience of patchy support from cloud service providers, and the intricate nature of HPC and cloud technologies, add layers of complexity.

Moreover, enterprise IT teams in these establishments can often view the world in different ways from their colleagues in research IT, with both sides having valid claims on management and support.

Some of this may be down to control of budgets, which, on the one hand is understandable, not least in the present climate.  In justifying budgets for HPC and a return on investment, a degree of circumspection or even doubt is understandable.

Although centralised resources might be allocated for funding, say, a £5 million HPC project, the continuous expenses associated with supporting a project within a higher education institution could be threefold. Nevertheless, if these academic institutions do not produce internationally recognised research, a substantial part of their purpose is jeopardised. Consequently, it is necessary to ensure that research is bolstered by the essential tools and technology, enabling the execution of such critical work.

When Enterprise and research IT meet

When enterprise and research IT meet, especially in higher education, there’s often a different lens on HPC. And in our experience, it’s a skills and experience mismatch, rather than an unwillingness to let someone new move in on their territory.

Many organisations acknowledge the existence of a shortage in both skills and resources, yet they harbour reservations about outsourcing – it’s a conundrum of workforce development versus workforce management. While the United Kingdom boasts several exceptional business computing degree programs at its universities, the availability of HPC-focused degree options appears scarce, with only Edinburgh University notably offering such a programme.

It isn’t as simple as honing the skills an IT manager may have undoubtedly have, coupled with years of experience and learning, it’s really that HPC is a different beast altogether, and brings with it challenges not seen so much in an enterprise or corporate setting. It requires a distinct set of competencies and a profound understanding of the intricacies involved. Consequently, many proficient research staff members in higher education eventually transition to the corporate private sector, driven by the allure of greater funding and rewards.

The focus therefore, should be upon training in-house support to be proficient in meeting the demands placed on it by HPC. The challenge ultimately revolves around retaining vital in-house expertise in partnership with outsourced support that brings with it the requisite experience and skills to bridge the gap. It should be a two-way street which fosters training, mentorship, and professional development opportunities to in-house IT staff.

The security dilemma

While the skills shortage clearly needs addressing, often, the enterprise mindset will rightfully, have concerns around security – and the management of it, particularly when HPC is migrated to the cloud. While here at Red Oak we have certainly witnessed ‘risky’ practices within a HPC environment, such as users setting up their own cloud accounts with Microsoft Azure, or AWS for instance, in general, an IT administrator will be weary of their security protocols being infringed.

This was evident according to IBM’s 2022 Data Breach Report, which stated that the average global cost of a successful cyber-attack reached a record high of $4.35 million (£3.41 million). Clearly, security issues will remain one of the most urgent matters of concern among IT administrators in academic and other organisations when it comes to cloud computing.

Crucially, universities and research departments need complete peace of mind that their HPC environment, whether in the cloud or on-premise, is delivering the desired outcomes and ROI.

If appropriately outsourced within a robust managed service arrangement, users can draw on the deep-level expertise and the enterprise and research IT departments can work in tandem, rather than become engaged in a Mexican standoff with a disconnect between the two.

To find out more about how Red Oak can support enterprise IT, download our free report ‘Incorporating the Cloud into the HPC Mix’ here.