Diminished Reality

[Skybird]

Penguin,

you can see a light shadow at the edited part of the video, almost all time. It still is not perfect, and the team admit it themselves, however, it is a breakthrough to acchieve such manipulation not after seconds and minutes of editing, but in almost real time. The two guys say themselves that it needs another 2 years to be perfected. And then I doubt you see there any shadow anymore where a deleted object has been.

The news is not that videos get manipulated. And the news is not green boxes or blue boxes, and copying a video stream into a neutral background. The news is that any existing video pictures can be edited live, in almost real time, within the same split of a second they get recorded and broadcasted.

You already edit the visual reality the very same split of a second you film and transmit it - no noticable time delay.

[the_tyrant]

Guys i have a question
In the book Angels and Demons, it was said that almost no tv station does real "live" broadcast anymore. The book says that its all just a reporter in front of a screen in the studio with fake wind and rain. Is it true?

[Sailor Steve]

Here in Salt Lake they have reporters all over the place all the time. One local station even has them stand in front of the station itself, just so they will be outside. Kind of hard to fake when they're right downtown where everyone can see them.

[vienna]

There is something I have often noticed on television news programs, at least locally here in Los Angeles. Say a news event occurred at 9:00 am and it was basically (visually) over at 10:00 am. If you turn on the news at 11:00 pm, there will be a report on the earlier incident with a live reporter at the scene long after the event is over and there is nothing to be seen or shown. This sort of thing happens all the time here and I have often marvelled at the reporters who have had to stand in some adverse conditions and report on something that could have easily been reported from the studio. I also wonder about the expense incurred to send a news truck, technician, and reporter to the scene. It seems senseless to expend so much time and effort to broadcast nothing at all.

[Penguin]

ok, I am back and after a meal consisting of more than 2 chocolate bars I am able to think better and realize the essence of this news.

I made a fault in my earlier thoughts. Actually I was discussing this news with a collegue of mine and we both made the same mistake, so I will try to explain it:

I thought that you actually needed to record at least one frame of the picture which you want to manipulate. This would be easy to detect, as you would then either dump the one unmanipulated frame (the non-smart way) or show the previous frame twice. Both are not hard to detect for the trained eye.

Actually in theory the invention works like this:
Say you are a friendly dictator who wants to hold a speech in front of the masses. And say that you have a healthy hatred against persons with blue hats. So you feed the image of objects who have the size and the colour of a person with a blue hat to this software.
The camera shows the masses listening to your speech, suddenly an insurgent with a blue hat jumps into the picture. The image of the person travels with the speed of light to the lens of the camera which catches this image. Before the image gets recorded or broadcast, the software manipulates it and changes the image by cutting out the person and fills the hole with something that makes sense - maybe a person with a red hat . This all happens in the timeframe (40ms) before the next picture is catched by the cam. So the picture that gets recorded is then indeed the new, altered picture.

Hope I made it a little clear. However my main reasons why I am not really shocked stay the same. Whenever you take something from a picture or add something to it, the principles are the same and used since years. When you add something you put something in front of the unedited picture - just like adding a new layer in photoshop. When you cut something out, you fill the hole with a new background - ideally with exactly the same background. The beaming in the Enterprise series worked like this, they just used an recorded image without the protagonists, but this is an old trick, used many decades before.
If you are not able to pre-record this image, the software has to fill the gap. To fill the gap you wopuld need exactly the same camera angle and position. Even a camera just behind the object you want to delete would show a slightly different picture, so you cannot use that. Imagine now person standing in front of a tree on a windy day, I really would like to see a believable manipulation of this and be able to examine it.

With the right instruments, you are basically able to analyze every line or column or pixel of the picture. Manipulations are detectable, but it would be arrogant to say that this is always the case.

I am worried about censorship/media manipulation to at least the same degree as Sky is. But us TV-clowns manipulate reality all the time. Beginning from the angle, point of of view, image section of the camera, colour settings, the choices which images are shown, the editing of recorded material, the audio mix, etc., etc. It doesn't mean that i am desensitived to these issues, but it's my daily work.

And regarding the news I find it interesting which events are not shown and which are.

hmm, answering to Sky's threads results in the same amount of words like his posts - didn't even come to answer the delay/teletext/time issue, next post

[August]

Originally Posted by Penguin

Actually in theory the invention works like this:
Say you are a friendly dictator who wants to hold a speech in front of the masses. And say that you have a healthy hatred against persons with blue hats. So you feed the image of objects who have the size and the colour of a person with a blue hat to this software.
The camera shows the masses listening to your speech, suddenly an insurgent with a blue hat jumps into the picture. The image of the person travels with the speed of light to the lens of the camera which catches this image. Before the image gets recorded or broadcast, the software manipulates it and changes the image by cutting out the person and fills the hole with something that makes sense - maybe a person with a red hat

Your analogy makes sense but the filler has to not only be the same general size and in scale with the surroundings it also has to have a valid purpose for being there.

For example you can change the hat color but you also have to provide a plausible reason for the red hat jumping into the picture, as well as the Dictators security team and the crowd responding to the presence of the now deleted insurgent.

Now all of this could certainly be achieved given enough time for the manipulator to consider all the angles and ramifications but trying to do it all live just seems to me like a recipe for getting caught doing it.

[Penguin]

Originally Posted by August

Your analogy makes sense but the filler has to not only be the same general size and in scale with the surroundings it also has to have a valid purpose for being there.

For example you can change the hat color but you also have to provide a plausible reason for the red hat jumping into the picture, as well as the Dictators security team and the crowd responding to the presence of the now deleted insurgent.

Now all of this could certainly be achieved given enough time for the manipulator to consider all the angles and ramifications but trying to do it all live just seems to me like a recipe for getting caught doing it.

In my example I meant that you actually paste in another person who wears a red hat. Changing jsut the hat colour is rather trivial.

Somehow your example with the security and crowd response made me laugh, imagine all these people have to be deleted/exchanged too. I see the picture of waving masses, where eventually one after another disappears - in the broadcast as well as in real life, to make sure that nobody could tell the story...

The believable filler is what I talked about when I mentioned the example of the person in front of a tree in the wind. The leaves in the background era are in constant motion, really hard to compute in real time. Remember the software only has a limited time to compute, and we however have all time in the world to analyze the footage later, frame by frame. When the filler is artificially made, you will be able to see a pattern or a discrepancy when compared to the natural background.

Then another problem comes into play: light. In a natural surrounding we have a constant change of light, whenever any pixel of the person or the background changes its exposition intensity you need a constant new calculation of the object that is replaced. That's why for example a blue box works only well with constant, defined light - on the object as well as on the background.

And this is also what makes object recognation so hard. An example would be you have a white box which you film in daylight. You have a shape and use trackers to define its dimensions for different angles. The white balance of the camera is set to daylight, so that the box appears to be white on your monitor. Then your evil little brother wants to mess with you and turns on his powerfull (conventional) flashlight. The light it emits is more yellowish than daylight so the white box suddenly appears in another colour. The box is not recognized anymore, as you have told the recognation software it should track a white box.

Another point, when we disregard webcams/cell phone cams, which run indeed on 25 fps, we are talking about a refresh rate of 50Hz or 60 Hz. This leaves us a theoretical timeframe of only 20ms or 16ms. In reality we have a timeframe of 18.4ms here, for a number of reasons, I don't know the exaxt timing for 60hz countries. With the already mentioned calculations that get dramatically more complicated, we are talking about an algorithm thats gets more complex in an exponential way. For an exponentially growing algorithm the processing time can be shortened only in a limited way by raw cpu power. But this goes however more into theoretical informatics, just think of the old story with the rice and the chessboard.

[Skybird]

Originally Posted by the_tyrant

Guys i have a question
In the book Angels and Demons, it was said that almost no tv station does real "live" broadcast anymore. The book says that its all just a reporter in front of a screen in the studio with fake wind and rain. Is it true?

Moderation of news, and other studio productions, are being done in blue box or green box studios indeed. The backgrounds can be animated or still, can be real video or computer graphics.

The most modern and versatile TV studio in Europe is said to be that of the second public TV channel in Germany, ZDF (at least that's what was said when it was introduced a year ago or so). Below, you see the "green hell", as the moderators call it (because they need to orientate their mimic and gestures so that they meet the superimposed images on the final TV screen, but the moderators only see the green background in the studio), and the standard studio theme that the TV audience is seeing instead. It can be replaced with anything else.





Camera-teams out there on the street and filming on location, still are needed to get the film material, though. However, computer graphics are being used for illustration purposes as well.

[TLAM Strike]

Originally Posted by the_tyrant

Guys i have a question
In the book Angels and Demons, it was said that almost no tv station does real "live" broadcast anymore. The book says that its all just a reporter in front of a screen in the studio with fake wind and rain. Is it true?

You can try asking Bob Woodruff whether or not his reports from Iraq were live.

[Skybird]

The software here works slightly different.

It first redcues the overall detail and resolution of the material - in real time.

It then automatically fills the defined area you want to clean of a given object with an auto-generated pattern that is calculated by the non-deleted background nearby - in real time.

It then increases the resolution again. By that, the difference in detail and quality between the edited and the nonedited parts of the material get reduced even more. Again: in real time.

As far as I understood it, the software is capable to recognise all by itself the once marked object. It automatically runs the above descriobed procedure on the object in later frames, even when the camera moves and the viewing angle changes: in real time.

Real time means: a delay of not more than 40 ms.

Note that in the demonstration video, it still does not work perfect - you can see a dark round sport in the place where the computermouse used to be. These artefacts, they say, will be overcome in two years or so, they estimate.

Consider what this means for silhouette recognition, and face recognition. Combine that with thbis new technology, and you get an impression of what the future holds. For example you will film a meeting of persons with live link to TV, and the wanted person you want to hide from the audience - simply is not visible in the signal output the camera and video unit transfers to the transmitter. All this with a time delay of 40 ms only, so that nobody could get the idea that what he sees is being manipulated - it is "live", right down to the second, isn't it!?

Considering the exploision in computer potency in the past 20 years and the unbelievable detail and realism of computer graphics being enabled by this, I would not rule out anything anymore.