Hardware

Hardware Specification

Phase2:

ToMMo Phase2 has 3 Units.
UnitA : Public (sharing)
UnitB : Information distribution (sharing)
UnitC : Database and analysis

(1)common

■Login nodes 8 nodes/Unit

  • Model: HPE ProLiant DL360 Gen10
  • O S : RHEL 7.4
  • CPU: Xeon 4116 2.1GHz (12core) x 2 socket
  • Memory: 128GB
  • HDD: 300GB (10000rpm) SAS x 3 (RAID1+ 1 spare)

写真

■Compute nodes(CPU)Ⅰ 92 nodes/Unit ※96GB memory

  • Model: HPE ProLiant XL170r Gen10
  • O S: CentOS 7.4
  • CPU: Xeon 4116 2.1GHz (12core) x 2 socket
  • Memory: 96GB
  • HDD: 300GB (10000rpm) SAS x 2 (RAID1)

写真

■Compute nodes(CPU)Ⅱ 8 nodes/Unit ※192GB memory

  • Model: HPE ProLiant XL170r Gen10
  • O S: CentOS 7.4
  • CPU: Xeon 4116 2.1GHz (12core) x 2 socket
  • Memory: 192GB
  • HDD: 300GB (10000rpm) SAS x 2 (RAID1)

写真

■Shared Memory Computer 1 nodes/Unit

  • Model: HPE ProLiant DL560 Gen10
  • O S: RHEL 7.4
  • CPU: Xeon 6130 2.1GHz (16core) x 4 socket
  • memory: 3,072GB
  • HDD: 600GB (10000rpm) SAS x 2 (RAID1)

写真

■Compute nodes(GPU) 1 nodes/Unit

  • Model:Nvidia DGX-1 (Volta)
  • O S: Ubuntu
  • CPU: Xeon E5-2698v4 2.2GHz (20core) x 2 socket
  • Memory: 512GB
  • SSD: 1.92GB x 4 (RAID0)
  • GPU:Volta V100×8/node

写真

(2)UnitC only

■Detabase nodes 24 nodes

  • Model: HPE ProLiant DL360 Gen10
  • O S: CentOS 7.4
  • CPU: Xeon 4116 2.1GHz (12core) x 2 socket
  • Memory: 128GB
  • HDD: 300GB (10000rpm) SAS x 3 (RAID1 + 1 spare)
  • SSD: 960GB

写真

(3)High Perfomance Storage

■UnitA (40G Ethernet)  

  • Lustre FileSystem:3PB

 

■UnitB (40G Ethernet)  

  • Lustre FileSystem:4PB

  • GPFS(IBM Spectrum Scale):2PB

 

■UnitC (4x FDR/EDR InfiniBand)  

  • Lustre FileSystem:12PB

 

写真

(4)FibreChannel storage (Nearline Storage)

  • Hitachi Virtual Storage Platform G400
  • <For Thin cliant>SAS1.8TB
  • <For Thin cliant>SSD3.8TB
  • <For Web server, UnitC Login nodes>SAS10TB

写真

(5)Tape Library (Backup System)

  • TS4500 Tape Library
  • TS1155×8 drive
  • JD cartridge(uncompressed 15TB)×1000

写真

 

 


Phase1:

In Bioinformatics and Medical Informatics,
next generation sequencers play a central role in processing large amounts of experimental data efficiently through the use of batch queuing systems.

ToMMo’s supercomputer processes variant calls to genome-wide association studies from whole-genome sequence data on 800 servers of PC clusters.
In addition, there is also other software in the same field that require large amounts of memory, such as denovo assembly.
Therefore, ToMMo has introduced shared-memory computers with 2TB of memory.
These shared-memory computers have a total of 16,480 CPU cores and a total theoretical performance is 401TFLOPS(teraflops).

ToMMo is developing and operating an original pipeline for high precision large-scale analysis on next-generation sequencer data and our integrated “Integration & Knowledge” database that combines the anonymized cohort information and genomic data.

This analysis platform enables over several thousand Japanese genome analysis and association studies.

1.Compute Nodes

SL230s Gen8 x672 nodes

  • Intel Xeon E5-2680v2 (2sockets, 20cores)
  • 128GB memory

SL250s Gen8 x64 nodes

  • Intel Xeon E5-2680v2 (2sockets, 20cores)
  • 128GB memory

SL250s Gen8 x64 nodes

  • Intel Xeon E5-2680v2 (2sockets, 20cores)
  • 256GB memory

Image

2. Shared Memory Computer

DL580s Gen8 x8 nodes (60cores, 2TB memory)

  • Intel Xeon E7-4870v2 (4sockets, 60cores)
  • 2TB memory

Image

3. Gateway

DL380p Gen8 x32 nodes

  • Intel Xeon E5-2680v2 (2sockets, 20cores)
  • 128GB memory

DL580_Gen8_bezel_FT2

4. High Performance Storage

Total disk capacity: 19.4PB (With a maximum of 50PB) SFA12K-40 is an active/active controller system. SFA12K-40 has 16 4xFDR connected InfiniBand ports and 32GB cache memory(Battery backed and mirrored) per controller. The Filesystem employed is a parallel distributed file system called Lustre.

Image

5.Tape Archive

Backup capacity is 12.1PB(With a maximum of 30PB) In order to protect important analysis data from future natural disasters, a Tape Archive is employed.

Image

 

 

 

gateway x32 nodes
Compute Nodes x800 nodes
Other Nodes x194 nodes
Total CPU cores 20,672core
Total theoretical performance 496TFLOPS
High Performance capacity 19.4PB
nearline Storage capacity 550TB
Tape capacity 12.1PB