At the time of doing this project (2018/2019), I was very interested in shanzhai (山寨 - bootleg/counterfit/parody products); usually when thinking of shanzhai products, phones or knockoff iPods come to mind, but I was looking at a lot of places like factories that make replica/fake fashion for a few brands, who also will make "fantasy pieces", for example a fake collaboration piece with both Louis Vuitton and Gucci branding, when in reality you cannot buy an official piece like this. Mostly this takes the form of shoes; one time on Taobao I saw a Balenciaga Triple-S shoe with Balenciaga, Louis Vuitton, and Off-White branding all on the same shoe. No such shoe 'officially' exists.
I really just loved the FW18 collection that Balenciaga just released, and I wanted to make more pieces that seemed like they belonged to this collection; just bootleg/shanzhai fantasy pieces that would fit in with the FW18 show's vision.
Using a corpus of Balenciaga runway shows, catalogues, and campaigns, a Pix2PixHD network was trained to reconstruct Balenciaga outfits from Densepose silhouettes.
The results are outfits which reflect this, but also the abstraction/misinterpretation from using these types of earlier generative systems. I think it still comes through that they are heavily inspired by Balenciaga's past few years under Demna Gvasalia, even though the network lacks any contextual awareness of the non-visual functions of clothing (e.g. why people carry bags, whether or not bags are separate from pants, why people prefer symmetrical outfits) - and in turn produces more strange outfits that completely disregard these functions.
The images below this point are ordered from the beginning of the project to the more final results. If you want, you can skip to the final images.
All images may be clicked to enlarge
Balenciaga's photography followed some pretty rigid rules, making it a pretty much perfect dataset to work with. Their website was already laid out in a grid, and the filenames were just numbers, so it was about 4 or 5 lines of python to just scrape their entire site.
There were 3 views of each garment too, so I decided to use densepose and train a pix2pixHD network to go from the pose -> photo. This also helped when adding runway photos and campaigns which have unusual poses into the dataset.
Below: Very very early generations. There was some light overfitting (when the network starts to just memorize specific outfits instead of synthesizing new ones); but it was different enough to still be interesting. This is best seen below (left) a generated coat, and then below (right) the ground truth for that pose.
At this point, because of the overfitting, I made some changes to the network + expanded the dataset with more campaigns + more runway shows, and restarted training. Below are some early generations from this (more successful) training run.
I thought it was very interesting that even though pretty much all of the images used for training were relatively symmetrical, some of the outputs completely lacked symmetry.
At this point after some more restarts / changes to both the model and the dataset, I finally got a network that was outputting what I thought were interesting results. Below are some of the more "finished" outfits. The main things I was interested in were the misinterpretations - the network would output garments that appeared to be treated with bleach (although that was not in the dataset), it would half-generate bags and sometimes make images of people holding what looked more like tassels, and sometimes really nice convolutional artefacts would show up as texture.
Below: A really nice example of convolutional artifacting making what looks like a knit/woven texture.
Using vid2vid, and the FW18 runway show as a dataset, I trained a model to generate runway videos; there's not very much temporal cohesion (outfits change whenever the camera angle does, and even sometimes without).
I trained a higher resolution vid2vid model, but the results (in my opinion) weren't as interesting ...
As for physical production of these outfits, me and my really good friend Mushbuh made "shin-bag" pants based off of the below generation (which is likely a misinterpretation of a model holding a bag next to their leg), through his brand itemLabel.
