Diminished Reality

[Penguin]

Originally Posted by August

Your analogy makes sense but the filler has to not only be the same general size and in scale with the surroundings it also has to have a valid purpose for being there.

For example you can change the hat color but you also have to provide a plausible reason for the red hat jumping into the picture, as well as the Dictators security team and the crowd responding to the presence of the now deleted insurgent.

Now all of this could certainly be achieved given enough time for the manipulator to consider all the angles and ramifications but trying to do it all live just seems to me like a recipe for getting caught doing it.

In my example I meant that you actually paste in another person who wears a red hat. Changing jsut the hat colour is rather trivial.

Somehow your example with the security and crowd response made me laugh, imagine all these people have to be deleted/exchanged too. I see the picture of waving masses, where eventually one after another disappears - in the broadcast as well as in real life, to make sure that nobody could tell the story...

The believable filler is what I talked about when I mentioned the example of the person in front of a tree in the wind. The leaves in the background era are in constant motion, really hard to compute in real time. Remember the software only has a limited time to compute, and we however have all time in the world to analyze the footage later, frame by frame. When the filler is artificially made, you will be able to see a pattern or a discrepancy when compared to the natural background.

Then another problem comes into play: light. In a natural surrounding we have a constant change of light, whenever any pixel of the person or the background changes its exposition intensity you need a constant new calculation of the object that is replaced. That's why for example a blue box works only well with constant, defined light - on the object as well as on the background.

And this is also what makes object recognation so hard. An example would be you have a white box which you film in daylight. You have a shape and use trackers to define its dimensions for different angles. The white balance of the camera is set to daylight, so that the box appears to be white on your monitor. Then your evil little brother wants to mess with you and turns on his powerfull (conventional) flashlight. The light it emits is more yellowish than daylight so the white box suddenly appears in another colour. The box is not recognized anymore, as you have told the recognation software it should track a white box.

Another point, when we disregard webcams/cell phone cams, which run indeed on 25 fps, we are talking about a refresh rate of 50Hz or 60 Hz. This leaves us a theoretical timeframe of only 20ms or 16ms. In reality we have a timeframe of 18.4ms here, for a number of reasons, I don't know the exaxt timing for 60hz countries. With the already mentioned calculations that get dramatically more complicated, we are talking about an algorithm thats gets more complex in an exponential way. For an exponentially growing algorithm the processing time can be shortened only in a limited way by raw cpu power. But this goes however more into theoretical informatics, just think of the old story with the rice and the chessboard.