#362
/capstor/store/cscs/swissai/infra01/vision-datasets/Apertus1p5_stage1_tokenized/general-mint1Tpdf_04
| kind | tokenized |
|---|---|
| status | active |
| samples | 5,437,067 |
| counted via | megatron idx |
| size | 36.0 GB |
| files | 197 |
| first seen | 2026-07-22 13:40 |
| last seen | 2026-07-22 13:40 |
| registered | 2026-07-22 13:40 |
# dataset #362 path: /capstor/store/cscs/swissai/infra01/vision-datasets/Apertus1p5_stage1_tokenized/general-mint1Tpdf_04 kind: tokenized | status: active samples: 5,437,067 (megatron idx) | size: 36.0 GB | files: 197 tree (truncated): ``` dataset_info.json rank_0_shard_2_64.bin rank_0_shard_2_64.idx rank_0_shard_34_64.bin rank_0_shard_34_64.idx rank_0_shard_50_64.bin rank_0_shard_50_64.idx rank_0_shard_61_64.bin rank_0_shard_61_64.idx rank_10_shard_18_64.bin rank_10_shard_18_64.idx rank_10_shard_21_64.bin rank_10_shard_21_64.idx rank_10_shard_40_64.bin rank_10_shard_40_64.idx rank_10_shard_62_64.bin rank_11_shard_11_64.bin rank_11_shard_11_64.idx rank_11_shard_26_64.bin rank_11_shard_26_64.idx rank_11_shard_43_64.bin rank_11_shard_43_64.idx rank_12_shard_12_64.bin rank_12_shard_12_64.idx rank_12_shard_20_64.bin ... (+172 more) ```