#287

# dataset #287
path: /capstor/store/cscs/swissai/infra01/vision-datasets/tokenized/sft/messytable_si
kind: manual | status: active
samples: n/a | size: 6.3 GB | files: 82

fields:
- document_id: int
- component_index: int
- kind: int
- token_offset: int
- token_length: int
- token_hash: text
- resize_height: int
- resize_width: int

tree (truncated):
```
dry_run_stats.json
rank_0000/_SUCCESS
rank_0000/components.000000.parquet
rank_0000/components.000001.parquet
rank_0000/tokens.000000.bin
rank_0000/tokens.000001.bin
rank_0000/worker_stats.json
rank_0000_checkpoint.pt
rank_0000_chunk_0000.bin
rank_0000_chunk_0000.idx
rank_0000_stats.json
rank_0001/_SUCCESS
rank_0001/components.000000.parquet
rank_0001/components.000001.parquet
rank_0001/tokens.000000.bin
rank_0001/tokens.000001.bin
rank_0001/worker_stats.json
rank_0001_checkpoint.pt
rank_0001_chunk_0000.bin
rank_0001_chunk_0000.idx
rank_0001_stats.json
rank_0002/_SUCCESS
rank_0002/components.000000.parquet
rank_0002/components.000001.parquet
rank_0002/tokens.000000.bin
... (+57 more)
```

samples

1

0

0

1

0

73

31570ab5fb00a5d7

0

0

2

0

2

1

292

3

69ff76615255f736

0

0

3

0

4

1

304

436

0946826e9d74c9e1

0

0

4

0

1

0

2048

7615

6741616fca9da36d

1040

1856

5

0

3

0

32508

7615

ab9a112a9cc99595

1040

1856

6

1

0

1

62968

71

a58245defa944cf5

0

0

7

1

2

1

63252

5

80861d5afaa9fee1

0

0

8

1

4

1

63272

632

695d93d6f2b576b1

0

0

9

1

1

0

65800

7615

19f63399c8a12bab

1040

1856

10

1

3

0

96260

7615

2dfa76fba8013e80

1040

1856