#729

# dataset #729
path: /capstor/store/cscs/swissai/infra01/vision-datasets/tokenized/interleave/mit_10m_recap
kind: parquet | status: active
samples: 12,612,360 (parquet footer) | size: 15.7 GB | files: 482

fields:
- document_id: int
- component_index: int
- kind: int
- token_offset: int
- token_length: int
- token_hash: text
- resize_height: int
- resize_width: int

tree (truncated):
```
dry_run_stats.json
rank_0000/_SUCCESS
rank_0000/components.000000.parquet
rank_0000/tokens.000000.bin
rank_0000/worker_stats.json
rank_0000_checkpoint.pt
rank_0000_stats.json
rank_0001/_SUCCESS
rank_0001/components.000000.parquet
rank_0001/tokens.000000.bin
rank_0001/worker_stats.json
rank_0001_checkpoint.pt
rank_0001_stats.json
rank_0002/_SUCCESS
rank_0002/components.000000.parquet
rank_0002/tokens.000000.bin
rank_0002/worker_stats.json
rank_0002_checkpoint.pt
rank_0002_stats.json
rank_0003/_SUCCESS
rank_0003/components.000000.parquet
rank_0003/tokens.000000.bin
rank_0003/worker_stats.json
rank_0003_checkpoint.pt
rank_0003_stats.json
... (+457 more)
```

samples

1

1003

1

1

0

109

92b121a40c32de9a

0

0

2

1003

2

1

436

133

d8eb72e06356c32c

0

0

3

1003

3

1

968

231

9bd1531c206c8a8c

0

0

4

1003

4

1

1892

106

cc9b51ba8a101354

0

0

5

1003

5

1

2316

192

aa933ddae6e4eae3

0

0

6

1003

6

1

3084

123

190d216ea393c0f2

0

0

7

1003

7

1

3576

127

168de47c3ec549b9

0

0

8

1003

8

1

4084

102

7cd0ade18ff4a919

0

0

9

1003

9

1

4492

256

1b59eb8d235b087b

0

0

10

1003

10

1

5516

213

26a9c5a515825dc6

0

0