Mayukh Goswami

Courtesy Sean Wehrli for live action cinematography and footage and Deeganto Joardar for 3D camera work and assets.

I am experimenting with machine learning in art with two main objectives:

I - To reduce the heavy load of work in cel animation or to do more challenging cel animation work with less effort.

II - To be able to do mixed media work with less effort.

I - Machine Learning in Cel Animation

Adding texture to a 2D animation is generally quite challenging. But with Ebsynth, I only had to make 1 frame of the digitally painted texture. Ebsynth textured the rest of the frames for me. For coloring, I used Cadmium which is another machine-learning tool that colors animated frames at the press of a button. All I have to do is make 1 or a few frames of flat color.

Software: EBSynth, Cadmium, Photoshop.

This too like the previous example, used Cadmium for the flat colors and Ebsynth for the final textured version.

Software: EBSynth, Cadmium, Photoshop.

Also, EBSynth reduces the amount of time taken to generally make rotoscoping art. The objective is to have a cohesive art design while incorporating as much machine learning into the process. It’ll be cool to be able to minimize effort and maximize output.

Software: EBSynth, Photoshop, After Effects.

The animation itself is 17 total frames of which I had to draw four frames. The rest of them required some level of correction but it definitely cut down work by a lot.

II - Machine Learning in Mixed Media

Runway ML is basically offers a bunch of cool one click options that can be a force multiplier of sorts when you want to apply Ebsynth to a visually complicated scene. In the above example, I used Runway ML to extract the actor from her bg so I can more easily do rotoscopy with Ebsynth.

Software: EBSynth, Runway ML, Photoshop, After Effects.