next »

[FinalSin]

Hey folks,

I'm a PhD student at Imperial College in London, working on automated game design techniques. It's a really fun area - our results are super basic (check some of them here) but the research itself is interesting and might one day have some great applications.

Last December I made some Flixel platformers using art I drew based on Oryx's delightful sprites. I'm no artist, so I did it mostly by cribbing off his structures, working out the rough space heads should occupy, and so on.

What I've been wondering lately is whether I could automatically generate a sprite from a photograph. There's two parts to this:

1. Extracting the features of the person (a Machine Intelligence task)
2. Funnelling them into a template

Assuming I could extract (1.) roughly, so I get a skin colour, hair colour and so on, how hard do you think it would be to create little sprites based on these features? Do you think sprites can follow a really rigid template, or would they all look characterless and mannequin-like?

One issue I'm really aware of is picking out important features. If you've only got 4x4 pixels for the body, then a yellow pixel placed well can convey all the detail in the world, that's his sheriff's badge, or a red hankerchief, or whatever.

I'm not really looking for solutions here, but I would love to hear from spriters on whether this might work. Alternatively: if I give you a photo of someone, what is your process for converting them into an 8x8/16x16 sprite?

Thanks!

[Scut Fabulous]

I think that you could be setting a very difficult task for yourself because spriting has a lot to do with determining the salient features of a person or thing and exaggerating those features.  You might want to try setting yourself some very tight constraints for your input data so that you limit the variables that will need tweaking.  Also, it might be worth trying higher resolutions and 8x8 or 16x16 to start, because the lower in res you go, the less room you have for 'mistakes' in a single pixel's placement.

Wish you the best with this project.  It would be interesting to see what results you come up with.

[Theophilus]

Part of why this is such an incredibly difficult, near-impossible thing to do is that pixel art is incredibly stylized. Everyone does it differently. Not everyone wants photo-realistic pixel art, but instead with special features; thin legs, curves, oddities. These things really bring the pixel art to life.

[Schoq]

One approach could be to start with a pixel "doll" based on the identified body type, and then pick from a pool of template eyes, mouth etc. that matches the identified features.
Basically make an AI to play with dress up dolls.

[FinalSin]

The style point is an interesting one, but I think it would be okay for the AI to have a 'style' that was fixed - even a bland one - as long as it was consistent.

Regarding starting smaller, that's a really interesting insight. I'd assumed the opposite would be true, that mo' pixels mo' problems. But you might be right, tackling it by going down from the highest res rather than starting at the bottom might be a better idea.

I like the doll idea. I might start with that, and have the AI design the components that plug into a model. Maybe that's a good first step.

This might not go anywhere, but I'll keep this update if it does. Thanks!

[bftd]

Of course you could just run the photo through a "pixelating" filter like in some cheap-ass iOS novelty camera app, and you can even do it smart with palette construction, edge detect and the like — it could produce something usable in, say, 128x128, but not in the restrictions that usually constitute game sprites and characters.

Besides, and this was already mentioned here, it would lack the authenticity — a character is defined by one or more features, which are nigh impossible for a machine to deduce on its own. You could, though, make a "library" of head/body/etc types, and utilizing some sort of fancy image recognition, process each photo as a set of these templates.

It could go like:

"wears a hat, dark skin, white shirt" deduced from photo -> "head with hat" template, paint skin dark, paint shirt white.

While this could work for processing mugshots or very neutral photos, making it work with dynamic images, facial expressions and the like is gonna be tough. Overall, i guess, it's limited by image recognition technology only.

[baconman]

Smallest I've ever gotten to make some sort of sense was 24x24, and even that's extremely tight for bigger/taller frames.