Tutorial: Datamoshing - The Beauty of Glitch

[This page has been moved to http://bitsynthesis.com/2009/04/tutorial-datamoshing-the-beauty-of-glitch/. Please update links to the new address.]

--

Datamoshing, as it has been coined by the Providence, RI artist collective Paper Rad (warning, site may induce seizures), refers to a technique of exploiting video compression to create intentional artifacting and distortion. If you've ever skipped ahead in a poorly compressed DVD rip and seen the moving outlines of actors ghosted behind the pixels of the scene you were just watching then you know what I'm talking about.

The effect has been in use by digital media artists since at least 2005, though exactly when it was first produced intentionally is a matter of some debate. It seems likely that someone was doing this before 2005, after all, there were poorly compressed DVD rips to inspire them. Regardless, it recently hit the mainstream in the form of music videos for Chairlift and Kanye West, so there has been a lot of digital press on the subject.

To understand datamoshing, you need to know about I- and P-frames. Digital video files are made up of sequences of different types of frames. I-frames (also referred to as keyframes) are full representations of a single frame of the video. Effectively, they are still images containing all the color and luminance information of that frame. P-frames, on the other hand, are reference files that inform the video player of changes in the image since the previous frame. Thus, only areas of motion or changing luminance values are described in a P-frame, making them more like a set of instructions than an image.

I-frame	P-frame

Naturally, P-frames contain less data than I-frames, so they take up less memory. This is their attraction for video compression codecs: the higher the P-frame to I-frame ratio the smaller the file size.

(note: There are also B-frames, these are similar to P-frames but a B-frame references both the frame before AND after itself. These can be used to create datamoshing effects also, but the effects are less predictable so I'll leave that up to you to experiment with.)

Following are some examples of normal frame structures.

Examples:
(clip A) I - P - P - P - P - P - (clip B) I - P - P - P - P - P - P
(clip A) I - P - B - P - B - P - (clip B) I - P - B - P - B - P - B


So, how do we use our knowledge of I- and P-frames to datamosh? Basically, the technique is to insert P-frames from Clip B after an I-frame, or a series of I- and P-frames, from Clip A (to put it another way: remove all the I-frames from Clip B). The result is that the movement from Clip B, as described in its P-frames, is translated to the image from Clip A. Clip B's P-frames act on the pixels in Clip A according to their original instructions.


Example:
(clip A) I - P - P - P - P - P - P - (clip B) P - P - P - P - P - P - P


When you skip ahead in a poor quality DVD rip, you may jump into the middle of a bunch of P-frames. Until the video hits an I-frame you will unintentionally be datamoshing the frame you skipped FROM with the movement of the frames you skipped TO. This is a result of setting the compression codec to produce fewer I-frames (thus, more P-frames) in order to save on file size for distribution over the internet or on CDs.

Another way to get some crazy effects is to copy the same P-frame over and over again in sequence. This results in color drifts and smears as the same pixel moves and manipulations are repeated over and over on top of themselves.


Example:
I - P1 - P2 - P3 - P3 - P3 - P3 - P3 - P3 - P3 - P3 - P3 - P3 - P3 - P3


If you want to clear the mess of datamoshed nonsense partway through your clip, just insert a new I-frame. Since I-frames are effectively still images, this will snap the image back to whatever is described in that I-frame.

The technique is really quite fun to play with, certainly one of the most organic and unpredictable approaches to manipulating video I've happened upon yet (that and video feedback loops).

A great tutorial of the process and the most accessible tools to do it with can be found in this 3-part tutorial (the link is to part 1): http://www.youtube.com/watch?v=tYytVzbPky8

You can find Paper Rad's pre-packaged "One Stop Datamosh Kit" here: http://www.court13.com/datamoshkit.zip

(note: If you run into problems with avidemux crashing all the time, be patient, don't move too fast, and I've found it helps to have another clip already placed after the one you're currently editing... I don't know why that is, but when copying and pasting lots of P-frames it is less likely to crash if it has that buffer)

Finally, for some inspiration, check out this clip by Takeshi Murata. I know it's been posted alongside other datamosh articles and tutorials, but that's because it's just that awesome. I mean really, even if you've already seen it, stop complaining and watch it again. You'll be happy you did.






Finally, you can find my own first experiment with datamoshing here:
http://understolenstreetlights.blogspot.com/2009/04/supermosh.html

--

Originally posted on Backstar.com.