Spaces:
Runtime error
Runtime error
Janne Hellsten
commited on
Commit
·
1574d42
1
Parent(s):
f7e4867
dataset_tool.py docs update, tf32 disable for calc_metrics.py
Browse files- Add a justification for using uncompressed zip and uncompressed png (#22)
- Clarify class label usage with dataset_tool.py (#18)
- Disable tf32 in calc_metrics as this has not been tested.
- calc_metrics.py +2 -0
- dataset_tool.py +21 -1
calc_metrics.py
CHANGED
|
@@ -47,6 +47,8 @@ def subprocess_fn(rank, args, temp_dir):
|
|
| 47 |
# Print network summary.
|
| 48 |
device = torch.device('cuda', rank)
|
| 49 |
torch.backends.cudnn.benchmark = True
|
|
|
|
|
|
|
| 50 |
G = copy.deepcopy(args.G).eval().requires_grad_(False).to(device)
|
| 51 |
if rank == 0 and args.verbose:
|
| 52 |
z = torch.empty([1, G.z_dim], device=device)
|
|
|
|
| 47 |
# Print network summary.
|
| 48 |
device = torch.device('cuda', rank)
|
| 49 |
torch.backends.cudnn.benchmark = True
|
| 50 |
+
torch.backends.cuda.matmul.allow_tf32 = False
|
| 51 |
+
torch.backends.cudnn.allow_tf32 = False
|
| 52 |
G = copy.deepcopy(args.G).eval().requires_grad_(False).to(device)
|
| 53 |
if rank == 0 and args.verbose:
|
| 54 |
z = torch.empty([1, G.z_dim], device=device)
|
dataset_tool.py
CHANGED
|
@@ -331,14 +331,34 @@ def convert_dataset(
|
|
| 331 |
--source path/ Recursively load all images from path/
|
| 332 |
--source dataset.zip Recursively load all images from dataset.zip
|
| 333 |
|
| 334 |
-
The output dataset format can be either an image folder or a zip archive.
|
| 335 |
Specifying the output format and path:
|
| 336 |
|
| 337 |
\b
|
| 338 |
--dest /path/to/dir Save output files under /path/to/dir
|
| 339 |
--dest /path/to/dataset.zip Save output files into /path/to/dataset.zip
|
| 340 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 341 |
Images within the dataset archive will be stored as uncompressed PNG.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 342 |
|
| 343 |
Image scale/crop and resolution requirements:
|
| 344 |
|
|
|
|
| 331 |
--source path/ Recursively load all images from path/
|
| 332 |
--source dataset.zip Recursively load all images from dataset.zip
|
| 333 |
|
|
|
|
| 334 |
Specifying the output format and path:
|
| 335 |
|
| 336 |
\b
|
| 337 |
--dest /path/to/dir Save output files under /path/to/dir
|
| 338 |
--dest /path/to/dataset.zip Save output files into /path/to/dataset.zip
|
| 339 |
|
| 340 |
+
The output dataset format can be either an image folder or an uncompressed zip archive.
|
| 341 |
+
Zip archives makes it easier to move datasets around file servers and clusters, and may
|
| 342 |
+
offer better training performance on network file systems.
|
| 343 |
+
|
| 344 |
Images within the dataset archive will be stored as uncompressed PNG.
|
| 345 |
+
Uncompresed PNGs can be efficiently decoded in the training loop.
|
| 346 |
+
|
| 347 |
+
Class labels are stored in a file called 'dataset.json' that is stored at the
|
| 348 |
+
dataset root folder. This file has the following structure:
|
| 349 |
+
|
| 350 |
+
\b
|
| 351 |
+
{
|
| 352 |
+
"labels": [
|
| 353 |
+
["00000/img00000000.png",6],
|
| 354 |
+
["00000/img00000001.png",9],
|
| 355 |
+
... repeated for every image in the datase
|
| 356 |
+
["00049/img00049999.png",1]
|
| 357 |
+
]
|
| 358 |
+
}
|
| 359 |
+
|
| 360 |
+
If the 'dataset.json' file cannot be found, the dataset is interpreted as
|
| 361 |
+
not containing class labels.
|
| 362 |
|
| 363 |
Image scale/crop and resolution requirements:
|
| 364 |
|