#357
/capstor/store/cscs/swissai/infra01/vision-datasets/tokenized/bigDocs7_5m
| kind | tokenized |
|---|---|
| status | active |
| samples | 307,895 |
| counted via | megatron idx |
| size | 7.4 GB |
| files | 197 |
| first seen | 2026-07-22 13:39 |
| last seen | 2026-07-22 13:39 |
| registered | 2026-07-22 13:39 |
# dataset #357 path: /capstor/store/cscs/swissai/infra01/vision-datasets/tokenized/bigDocs7_5m kind: tokenized | status: active samples: 307,895 (megatron idx) | size: 7.4 GB | files: 197 tree (truncated): ``` tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/dataset_info.json tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_0_shard_0_24.bin tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_0_shard_0_24.idx tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_0_shard_23_24.bin tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_0_shard_23_24.idx tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_0_shard_3_24.bin tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_0_shard_3_24.idx tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_10_shard_15_24.bin tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_10_shard_15_24.idx tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_10_shard_5_24.bin tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_10_shard_5_24.idx tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_11_shard_13_24.bin tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_11_shard_13_24.idx tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_11_shard_8_24.bin tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_11_shard_8_24.idx tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_12_shard_21_24.bin tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_12_shard_2_24.bin tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_12_shard_2_24.idx tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_12_shard_9_24.bin tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_12_shard_9_24.idx tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_13_shard_19_24.bin tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_13_shard_19_24.idx tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_13_shard_6_24.bin tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_13_shard_6_24.idx tokenized_apertus_emu3_5_imgonly/ArxivOCR_image_only/64x128_2048x2048/rank_14_shard_20_24.bin ... (+172 more) ```