#42
/capstor/store/cscs/swissai/infra01/vision-datasets/Apertus1p5_stage1_tokenized/general-mint1Tpdf_07
| kind | tokenized |
|---|---|
| status | active |
| samples | 22,882,437 |
| counted via | megatron idx |
| size | 145.4 GB |
| files | 1,537 |
| first seen | 2026-07-22 13:22 |
| last seen | 2026-07-22 13:22 |
| registered | 2026-07-22 13:22 |
# dataset #42 path: /capstor/store/cscs/swissai/infra01/vision-datasets/Apertus1p5_stage1_tokenized/general-mint1Tpdf_07 kind: tokenized | status: active samples: 22,882,437 (megatron idx) | size: 145.4 GB | files: 1,537 tree (truncated): ``` 64x128_2048x2048/dataset_info.json 64x128_2048x2048/rank_0_shard_285_512.bin 64x128_2048x2048/rank_0_shard_285_512.idx 64x128_2048x2048/rank_0_shard_2_512.bin 64x128_2048x2048/rank_0_shard_2_512.idx 64x128_2048x2048/rank_100_shard_151_512.bin 64x128_2048x2048/rank_100_shard_151_512.idx 64x128_2048x2048/rank_100_shard_456_512.bin 64x128_2048x2048/rank_100_shard_456_512.idx 64x128_2048x2048/rank_101_shard_227_512.bin 64x128_2048x2048/rank_101_shard_227_512.idx 64x128_2048x2048/rank_101_shard_273_512.bin 64x128_2048x2048/rank_101_shard_273_512.idx 64x128_2048x2048/rank_102_shard_24_512.bin 64x128_2048x2048/rank_102_shard_24_512.idx 64x128_2048x2048/rank_102_shard_304_512.bin 64x128_2048x2048/rank_102_shard_304_512.idx 64x128_2048x2048/rank_103_shard_454_512.bin 64x128_2048x2048/rank_103_shard_454_512.idx 64x128_2048x2048/rank_103_shard_54_512.bin 64x128_2048x2048/rank_103_shard_54_512.idx 64x128_2048x2048/rank_104_shard_23_512.bin 64x128_2048x2048/rank_104_shard_23_512.idx 64x128_2048x2048/rank_104_shard_373_512.bin 64x128_2048x2048/rank_104_shard_373_512.idx ... (+1,512 more) ```