#355

# dataset #355
path: /capstor/store/cscs/swissai/infra01/vision-datasets/processed/sft/pangea_instruct
kind: webdataset | status: active
samples: n/a | size: 430.2 GB | files: 38

fields:
- key: int
- 0.jpg: image
- 0.png: image
- 0.webp: image

tree (truncated):
```
PangeaIns_train.jsonl
PangeaIns_train_decontaminated.jsonl
PangeaIns_train_normalized.jsonl
dropped.jsonl
subset_strip_map.json
tars/cultural/laion-multi-1M/images_part_00_00.tar
tars/cultural/laion-multi-1M/images_part_00_01.tar
tars/cultural/laion-multi-1M/images_part_01_00.tar
tars/cultural/laion-multi-1M/images_part_01_01.tar
tars/cultural/laion-multi-1M/images_part_02_00.tar
tars/cultural/laion-multi-1M/images_part_02_01_00.tar
tars/cultural/laion-multi-1M/images_part_02_01_01.tar
tars/cultural/laion-multi-1M/images_part_03_00.tar
tars/cultural/laion-multi-1M/images_part_03_01.tar
tars/doc+chart/Viet-Doc-VQA-II/images.tar
tars/doc+chart/Viet-Doc-VQA/images.tar
tars/doc+chart/doc-vqa/images.tar
tars/doc+chart/table-vqa/images.tar
tars/general/ALLaVA-4V/allava_laion_part_00.tar
tars/general/ALLaVA-4V/allava_laion_part_01.tar
tars/general/ALLaVA-4V/allava_laion_part_02.tar
tars/general/ALLaVA-4V/allava_laion_part_03.tar
tars/general/COCO/train2017.tar
tars/general/MTVQA/images.tar
tars/general/ShareGPT-4o/images.tar
... (+13 more)
```

samples

#	key	0.jpg	0.png	0.webp
1	3358450009866
2	2843977018256
3	1562861003768
4	1194422011747
5	2626771104764
6	3162182014952
7	1107776012601
8	3449619003451
9	3142473078548
10	1095363011815
11	1960871005286
12	3131885028038
13	4178115001295
14	1053458002732
15	2563274001670
16	4247329004481
17	3107977003428
18	1324958006365
19	1435061006560
20	4200822004480
21	1294733003621
22	2165810007255
23	3798769022183
24	4135491020100
25	903140009366
26	3407379008692
27	3611407009689
28	1116818004709
29	2968550000614
30	3269779005478
31	2697598001400
32	2819663009316