Data format for text-to-image
Hi All,I’m trying to load a custom dataset for text-to-image fine-tuning but I’m not sure how the data needs to be formatted. Right now I have a folder of png files and a csv that maps png paths to captions, with column names of “image” and “text”. But it seems like it needs a different format.Any help appreciated!
2023-01-13T20:58:07Z
gulp… yeah, rtfm:Create an image datasetApologies. Maybe my post will prevent further ones like it.
2023-01-13T23:30:30Z
Actually, wait… I’ve created a file with the format given under “Image Captioning” on the doc page, but I’m hitting an error when runningtrain_text_to_image.py:Exception has occurred: ArrowInvalid
JSON parse error: Missing a name for object member. in row 0Obviously something is wrong, but I’m not sure what…I have a data folder with all my png files and mymetadata.jsonlfile, formatted as:{"file_name": "something_1.png", "text": ["caption_1", "caption_2", ..., "caption_n"]}
{"file_name": "something_2.png", "text": ["caption_1", "caption_2", ..., "caption_n"]}
...
{"file_name": "something_n.png", "text": ["caption_1", "caption_2", ..., "caption_n"]}What am I not understanding here… ?
2023-01-14T00:14:18Z
jbmaxwell:JSON parse error: Missing a name for object member. in row 0Okay, just had to ask json to dump on my lines…json_formatted = json.dumps(line_dict)