#344
/capstor/store/cscs/swissai/infra01/vision-datasets/tokenized/archive
| kind | tokenized |
|---|---|
| status | active |
| samples | 12,564,932 |
| counted via | megatron idx |
| size | 319.6 GB |
| files | 1,685 |
| first seen | 2026-07-22 13:39 |
| last seen | 2026-07-22 13:39 |
| registered | 2026-07-22 13:39 |
# dataset #344 path: /capstor/store/cscs/swissai/infra01/vision-datasets/tokenized/archive kind: tokenized | status: active samples: 12,564,932 (megatron idx) | size: 319.6 GB | files: 1,685 tree (truncated): ``` pd12m_full_high_res_old/dry_run_stats.json pd12m_full_high_res_old/rank_0000_checkpoint.pt pd12m_full_high_res_old/rank_0000_chunk_0000.bin pd12m_full_high_res_old/rank_0000_chunk_0000.idx pd12m_full_high_res_old/rank_0000_chunk_0001.bin pd12m_full_high_res_old/rank_0000_chunk_0001.idx pd12m_full_high_res_old/rank_0000_stats.json pd12m_full_high_res_old/rank_0001_checkpoint.pt pd12m_full_high_res_old/rank_0001_chunk_0000.bin pd12m_full_high_res_old/rank_0001_chunk_0000.idx pd12m_full_high_res_old/rank_0001_chunk_0001.bin pd12m_full_high_res_old/rank_0001_chunk_0001.idx pd12m_full_high_res_old/rank_0001_stats.json pd12m_full_high_res_old/rank_0002_checkpoint.pt pd12m_full_high_res_old/rank_0002_chunk_0000.bin pd12m_full_high_res_old/rank_0002_chunk_0000.idx pd12m_full_high_res_old/rank_0002_chunk_0001.bin pd12m_full_high_res_old/rank_0002_chunk_0001.idx pd12m_full_high_res_old/rank_0002_stats.json pd12m_full_high_res_old/rank_0003_checkpoint.pt pd12m_full_high_res_old/rank_0003_chunk_0000.bin pd12m_full_high_res_old/rank_0003_chunk_0000.idx pd12m_full_high_res_old/rank_0003_chunk_0001.bin pd12m_full_high_res_old/rank_0003_chunk_0001.idx pd12m_full_high_res_old/rank_0003_stats.json ... (+1,660 more) ```