High Performance Computing Infrastructure in Japan

High Performance Computing Infrastructure in Japan

Kento Aida National Institute of Informatics

Kento Aida, National Institute of Informatics

2

Overview  of  HPCI

Kento Aida, National Institute of Informatics

3

Introduction n  High Performance Computing Infrastructure (HPCI) Ø  national project promoted by Ministry of Education, Culture, Sports, Science and Technology (MEXT) in Japan Ø  distributed computing infrastructure for high performance computing ü “K computer”, supercomputers and high performance storage

Ø  first production level infrastructure for high performance computing in Japan

n  roadmap Ø  – Mar 2011

basic design

ü network, authentication, user management, shared storage, testbed for advanced software

Ø  Apr – Dec 2011 detailed design Ø  Jan – Aug 2012 test operation Ø  Sep 2012 – production level operation

Kento Aida, National Institute of Informatics

4

Services (1) account registration

(2) single sign-on

HPCI account

cert.

ü  input HPCI account and password   ü  operation through a web   browser

ü  application ü  account

         

(3) login to resources ü  no password ü  run jobs on supercomputers ü  access files on shared storages

                                              computer    

HPCI shared storage

Kento Aida, National Institute of Informatics

5

System Overview user management HPCI ID registration review proposals

authentication

CA system

HPCI acct.

shib. SP

apply certificate

acct. registration

portal

certificate repository

single sign-on helpdesk

shib. SP

HPCI Secretariat (RIST)

AICS (K-computer) Supercomputer Centers in 9 Universities

                                              computer computer  resource resource   shared storage

AICS, U. Tokyo

             

computer resource

shib. shib. IdP shib. IdP IdP

NII

network infrastructure

More resources will be connected after 2012.

Kento Aida, National Institute of Informatics

As of Nov. 2012

Computing Resources RIKEN AICS： K computer (10.62PF, 1.27PiB/30PiB) Kyoto Univ. XE6 (300.8 TF, 59 TB) GreenBlade8000(242.5TF, 38TB) 2548X(10.6TF, 24TB)

Hokkaido Univ.： SR16000/M1(51.6TF/172TF, 6.6TB/ 22TB) BS2000 (5.76TF/44TF, 1.92TB/14TB) RENKEI-VPE: VM Hosting

Osaka Univ.： SX-9 (16TF, 10TB) SX-8R (5.3TF, 3.3TB) PCCluster (6.1TF, 2.0TB)

Kyushu Univ.： FX10 (68.1TF/181.6TF, 9.2TB/24TB) CX400 (44.2TF/510.1TF, 16.4TB/184.5TB) SR16000 L2 (25.3TF, 5.5TB)

source: M. Hirakawa, AICS

Nagoya Univ.： FX1(30.72TF, 24TB) HX600(25.6TF, 10TB) M9000(3.84TF, 3TB)

Tohoku Univ.： SX-9(29.4TF, 18TB) Express5800 (1.74TF, 3TB) Univ. of Tsukuba： T2K (95.4Tflops, 20TB) HA-PACS (802Tflops, 34.3TB) FIRST (36.1TFlops, 1.6TB) Univ. of Tokyo： FX10 (1.13PF, 150TB) SR16000/M1(54.9TF, 10.94TB) T2K (75.36TF/140TF, 16TB/31.25TB) EastHubPCCluster(10TF/13TF, 5.71TB/ 8.15TB) GPU Cluster(CPU 4.5TF, GPU 16.48TF, 1.5TB) WestHubPCCluster(12.37TF,8.25TB) Tokyo Institute ofHosting Technology： RENKEI-VPE:VM TSUBAME2.0 (0.24PF/2.4PF, 10TB/ 100TB) RENKEI-VPE : VM Hosting

Storage HPCI WEST HUB

HPCI EAST HUB University of Tokyo

AICS, RIKEN

•  12 PB+ storage

•  10 PB+ storage

Hokkaido University

Gfarm2 is used as the global shared file system Kyushu University

Tohoku University University of Tsukuba Tokyo Institute of Technology Nagoya University Osaka University Kyoto University

source: Y. Ishikawa, Univ. of Tokyo

Network (SINET4) SINET4: Science Information NETwork 4

9

SINET4 (cont’d) n  connection to 700+ academic sites n  IX for commercial networks n  80Gbps backbone between Tokyo and Osaka Ø  134（30Gbps） in Tokyo Ø  22（11Gbps） in Osaka n  L3VPN, L2VPN/VPLS, QoS CA portal

user

user

univerisity

university

user

user

IX （Tokyo)

QoS

IX （Osaka)

commercial network

VPN non-comercial network university

university

AICS LAN storage user compt. resource

storage user compt. resource

storage user compt. resource

resource provider

resource provider

storage user compt. resource

Kento Aida, National Institute of Informatics

10

Cloud Service n  VM hosting Ø  repository for research results Ø  pre/post processing Ø  testbed for prototype system software

source: S. Takizawa, Tokyo Tech.

Kento Aida, National Institute of Informatics

11

Authen3ca3on  System

Kento Aida, National Institute of Informatics

12

Overview of Authentication System n  access to web portals: Shibboleth Ø  management of certificates, user support, cloud service

n  access to remote computers: GSI Ø  login to remote computers, access to shared storage

n  bridge between shibboleth and GSI: web portal user

portal IdP, HPCI account pass word single  sign-­‐on

% gsi-ssh host.univ.ac.jp

(1)  sign-on to the portal (cert. issuing system) (2)  generate a proxy certificate and download the proxy certificate (3) ssh login to remote computers ü  no need to give local account name and password

• login to remote computers • access to shared storage Kento Aida, National Institute of Informatics

13

Architecture NII ü apply user cert. ü single sigh-on

cert. management system

portal (Shib. SP)

cert. repository

proxy cert. repository

browser

CA system (Shib. SP)

Shib. DS SINET 4 ü login to resources

supercomputer centers, RIKEN portal (Shib. SP) proxy cert. repository

GSI-SSH client

supercomputer centers, RIKEN

Shib. IdP account DB

GSI-SSH server

Kento Aida, National Institute of Informatics

14

Architecture (cont’d) NII ü apply user cert. ü single sigh-on

cert. management system

portal (Shib. SP)

cert. repository

proxy cert. repository

browser

CA system (Shib. SP)

Shib. DS SINET 4 ü login to resources

supercomputer centers, RIKEN portal (Shib. SP) proxy cert. repository

GSI-SSH client

supercomputer centers, RIKEN

Shib. IdP account DB

GSI-SSH server

Kento Aida, National Institute of Informatics

15

Software role Certificate Authority

system

software

CA system

NAREGI-CA

certificate management

custom software

certificate repository

MyProxy

ID federation

Shibboleth

Portal （NII，supercomputer centers）

portal (cert. issuing system)

custom software

Proxy certificate repository

MyProxy

ID federation

Shibboleth

Identity Provider （supercomputer centers, AICS）

ID federation

Shibboleth

Resource Provider （supercomputer centers, AICS）

middleware to access resources

GSI-SSH Gfarm

Kento Aida, National Institute of Informatics

16

Summary and Future Plan n  Summary Ø  This talk presents a design of HPCI focusing on the authentication mechanism. Ø  HPCI started production level operation in Sep. 2012.

n  Issues Ø  interoperation with oversea infrastructure ü review of the operation in HPCI CA to obtain approval of International Grid Trust Federation (IGTF)

Ø  federation with other authentication system ü discussion about the federation with other web authentication systems, e.g. OpenID

Kento Aida, National Institute of Informatics

17

h=ps://www.hpci-­‐office.jp/

Kento Aida, National Institute of Informatics