A Text2Text model for semantic generation of building layouts
The goal of the project would be to fine tune GPT-Neo J 6b on the task of semantic design generation.The model will learn to transform natural language prompts into geometric descriptions of designs. It will then be deployed in a simple interface that allows for users to immediately see their generated design, creating a semantic generation tool out of Neo.I already have a dataset of paired annotations-design pairs and have started experimentations with the smaller Neo models as we speak.Let me know if you’re interested
2021-06-25T12:33:49Z
That’s a cool idea !Could you post an example of language prompt => geometric descriptions => generated design ? Thanks.
2021-06-25T13:08:21Z
Sure! I have loads actuallyThese are some outputs from training the smallest neo (125M) on 5% of my total data.Legend: bedroom=green, bathroom=pink, the other two are kitchen (darker pink) and living roomgptlayout11978×1472 138 KBMy goal is to understand the effect of scale in this, going all the way up to Neo-J 6b model trained on all the data.But also, ideally, to incorporate more modalities such as taking advantage of relational data, adding design evaluation as rewards, and other things I can’t think of but others might!
2021-06-25T13:08:52Z
Another exampleimage825×755 27.3 KB
2021-06-25T13:09:25Z
image1212×728 26.2 KB
2021-06-25T16:36:10Z
Your experiments look amazing, would love to be a part of this.
2021-06-25T17:23:57Z
That’s a great project!
2021-06-25T22:20:33Z
This sounds like a very very cool idea!
2021-06-26T06:52:10Z
Hello, thanks for your interest!yes of course you can join, as soon as I figure out how we do thatI think we can do that on Monday, if I understood it correctly!
2021-06-26T07:04:05Z
@THEODOROSthat’s cool idea, would love to be part of this project.
2021-06-26T11:44:48Z
This sounds very cool@THEODOROS,I would love to be in a project like this.I have some good hands on experience work with transformer based models.
2021-06-26T11:46:30Z
@THEODOROSlet me know if I can be a part of this project,it would be great.
2021-06-26T11:47:38Z
The idea is very unique@THEODOROS.would love to work for this project
2021-06-26T12:44:52Z
Thanks everyone! I think joining should be alright, as soon as I figure out how to do it.I believe Patric mentioned in slack that teams will be made on Monday. The limit, if enough interest, seems to be 10 people although if we do reach that we should consider ways to make this more interesting and explore more angles.
2021-06-26T14:19:16Z
Yeah, we’ll start forming teams from Monday, but it looks like you’ve already got a team hereIf there are 10 members, then the team might get  2 TPU VMs
2021-06-28T17:00:17Z
officially defining this projectPutting everybody in the official sheethere. More people can still join! Leave a comment here or on the sheet if you want to change something.@gchhablaniI was not sure if you wanted to be part of this project or not, so I added you anyways. Let me know
2021-06-28T23:35:26Z
Thannk you@valhalla! I guess we’ll follow on the slack on how to begin.
2021-06-28T23:39:12Z
Hey everyone!It’s a hectic week for me, and I guess it will be for all of us, but it would be nice to meet and discuss ideas if you want. I’ve made our channel in the discord provided by HF here:Flax-HuggingFace-Community-WeekIt’s called gpt-building-layouts, feel free to jump in and say hi!
2021-07-08T20:36:02Z
That’s an amazing idea! If it’s still possible I’d like to join the team@THEODOROSI only able to leave comments in the google spreadsheet. Also, could you please help me to join the slack channelgpt-building-layoutsthat you mentioned above?