Diminished Reality

[Skybird]

Diminished Reality - German software engineers now were successful in manipulating live videos so that during live transmission of the moving pictures objects could be erased from the video.

I imagine the potential for abuse, and manipulation of information, news, and "live-coverage". The possibilities for live censorship without the audience realising it, are endless. Currently, so say Wolfgang Broll and Jan Herling of the university of Ilmenau who pionerred and acchieved the breakthrough, manipulations like this could be proven in according videos and life-streams. But in two years, they estimate, the technology will be so perfect that the manipulation can no longer been shown.

Philip K. Dick, anyone...?

http://bundes.blog.de/2010/10/19/made-germany-diminished-reality-video-manipulation-echtzeit-9678697/

We should make a mental note that from now on, such manipulation is possible. In other words: where in the past years we have learned not to trust the media showing photography and video records, we now need to always keep in mind that we even cannot trust live coverage anymore.

What this technology is able to do in the hands of current corrupt political establishments and lobby politics, is a nightmare, imo.

German news article (http://www.welt.de/wissenschaft/article11608744/Erstmals-Objekte-aus-Live-Videouebertragung-geloescht.html)

[August]

Anyone who ever watched the movie "The Sting" knows that "live" broadcasts could be manipulated long before this...

[the_tyrant]

The Chinese TV stations have been doing this for years
live is not live, its 5 minutes behind, so they can cut stuff out if they want

[Rockstar]

A great job so far of dividing a population into two extreme groups with great rhetoric (logos, pathos & ethos). Now to seal the coffin with 'official' video feeds a new selling point to gather audiences in addition to being fair and balanced! Is it live or is it memorex? Never mind facts, no matter who shows it, we'll be busy arguing amongst ourselves again this time trying to prove a videos authenticity.

Meanwhile a select few will enjoy their jet set lifestyle and meetings in tropical settings.

The cynic

[Oberon]

Reminds me of when the Czech weather panorama got hacked and this was played:

http://www.youtube.com/watch?v=gjsZCyNb7o0

But yeah, expect this to be abused in all kinds of merry ways before long.

[Skybird]

The Chinese TV stations have been doing this for years
live is not live, its 5 minutes behind, so they can cut stuff out if they want
"5 Minutes behind", means it is delayed, edited, then broadcasted. It is not "live".

And No, August, this has not been done "since years" - not in real time.

This now means live in almost real time. In a German essay on the two experts and their breakthrough, the calculation time to process the video images and broadcast them is mentioned to be 40 milliseconds. Granted, that is also a delayed broadcast, but it is not delayed by minutes or seconds, but just 40 ms.


Als Brolls Mitarbeiter Jan Herling diese Software kürzlich bei einem internationalen Symposium in Seoul vorstellte, war das Publikum nach seinen Worten erstaunt und zugleich skeptisch. „Es gibt weltweit nur eine Hand voll Experten für diese Technik, aber selbst für sie war es vollkommen neu und bis dato nicht vorstellbar“, sagt Diplom-Informatiker Herling.

When Broll's assistant Jan Herling demonstrated the software short time ago at a symposium in Seoul, the audience was amazed and sceptical. "Worldwide, there is only a small handful of experts for this technology, but even for them this was completely new and so far unimaginable", says Diplom-Informatiker Herling.

[August]

"5 Minutes behind", means it is delayed, edited, then broadcasted. It is not "live".

And No, August, this has not been done "since years" - not in real time.

This now means live in almost real time. In a German essay on the two experts and their breakthrough, the calculation time to process the video images and broadcast them is mentioned to be 40 milliseconds. Granted, that is also a delayed broadcast, but it is not delayed by minutes or seconds, but just 40 ms.

My point was how does an audience ever know for sure that a broadcast is in real time? I mean aside from the broadcasters claims of "live coverage", unless we happen to see a clock or something similar in the background we have no real ability to tell whether any footage "is live or is it Memorex".

[Penguin]

Due to various technical reasons live is never really live. The delay untill the broadcast reaches the tv audience takes roughly 2-several seconds, depending on the transmission, production environment, yada,yada,yada. Live broadcasts in the US have some seconds extra delay to beep out bad language - thanks to the FCC. Countries with more censorship put in some more extra delay. The GDR had for example 2 minutes delay when they transmitted Gorbatchevs visit to west germany.

40 ms is exactly one frame when we talk about 25fps. It can't become any faster, as you at least need one source image first which you want to manipulate. However be assured, broadcast priofessionals can see when a frame is skipped. :DL
While the news is exiting for me as a video nerd, (nearly) live manipulation is possible since several decades. Just think about an overlay which you blend into the picture, the station logo being the most obvious one. If you give me some seconds and a static camera I can put in a dinosaur next to Angela Merkel that looks real - a fake looking one just takes one click ;)
When the americans would have been prepared at the superbowl broadcast some years ago, they could easily put in a blur to prevent the nipplegate-scandal which turned several people blind :D

[Skybird]

I have digital cable TV. Before, I had regular (analogue) cable which did not need a set-top-box.

When I now compare my clocks (radio-controlled and thus: precise) with the TV clock before the 20:00 news, for example, I realise that digital cable TV transmits the signals with a delay of around 2 seconds. It lags behind. All programs I compare with analogue cable, lag behind by 2 seconds. With analogue cable and in the past: radiowave-transmitted TV, there was no such delay. Until today I can demonstrate that difference between digital and analogue cable.

I would not be able to notice it with a delay of not 2 seconds but just 40 ms.

The two experts say that they estimate that in 2 years, graphical artifacts that c urrently are if not visible at least can be shown by graphical analyxsis to be existent, will be gone - means they will be so minor then that manipulated images no longer can be shown by analysis to have been manipulated at all. Together with that manipulation being done in almost real time, the possibility for deforming visual reality, chnaged "live" evidence of any kind, and censorship, are almost perfect, and total.

[Skybird]

Due to various technical reasons live is never really live. The delay untill the broadcast reaches the tv audience takes roughly 2-several seconds, depending on the transmission, production environment, yada,yada,yada. Live broadcasts in the US have some seconds extra delay to beep out bad language - thanks to the FCC. Countries with more censorship put in some more extra delay. The GDR had for example 2 minutes delay when they transmitted Gorbatchevs visit to west germany.

40 ms is exactly one frame when we talk about 25fps. It can't become any faster, as you at least need one source image first which you want to manipulate. However be assured, broadcast priofessionals can see when a frame is skipped. :DL
While the news is exiting for me as a video nerd, (nearly) live manipulation is possible since several decades. Just think about an overlay which you blend into the picture, the station logo being the most obvious one. If you give me some seconds and a static camera I can put in a dinosaur next to Angela Merkel that looks real - a fake looking one just takes one click ;)
When the americans would have been prepared at the superbowl broadcast some years ago, they could easily put in a blur to prevent the nipplegate-scandal which turned several people blind :D

The trick, and danger, is to manipulate the media stream without the audience knowing it, being aware of it, or being able to prove that manipulation takes place. ;) To compoare what they do now with that dinosaur is like comparing Ray Harryhousen's wax-monsters and stop-motion technique with Lucasfilm's fully digital production of the last Star Wars movie.

[Penguin]

Well as the target audience you're alwas ****ed! :DL
That's why for example in news broadcasts the host often says that the interview was pre-recorded. Or when we see politians making a statement with the Reichtag in the background: mostly they are in a windowless studio in front of a green-/bluebox.
In the example picture from the article, the one with the desktop, you can see that the background is rather plain, this is an easy manipulation. I would really want to see a moving image with a background that is also non-static, then I would be really amazed. But anyway, I believe that software doing this manipulation on their own does a rather quick and dirty job, with the right tools you can see that there is something missing or copied in - again I am not talking about the audience.

I am manipulating (editing) moving pictures of puking people right now. I love my job, especially when I drank last night ;)

[Penguin]

one more thing: take a look at 0:35 in the video of the first linek, when the shadow of the camera guy hits the desk, you'll be able to see the missing thing. That's the problem which occurs in changing environment/changing luminance
But really, I don't want to put the invention of these guys down, it's pretty impressive, but the whole maipulation thing is, as the Krauts say: "An old hat"

[TLAM Strike]

Anyone who ever watched the movie "The Sting" knows that "live" broadcasts could be manipulated long before this...

... or has watched "Speed". ;)

[Skybird]

Penguin,

you can see a light shadow at the edited part of the video, almost all time. It still is not perfect, and the team admit it themselves, however, it is a breakthrough to acchieve such manipulation not after seconds and minutes of editing, but in almost real time. The two guys say themselves that it needs another 2 years to be perfected. And then I doubt you see there any shadow anymore where a deleted object has been.

The news is not that videos get manipulated. And the news is not green boxes or blue boxes, and copying a video stream into a neutral background. The news is that any existing video pictures can be edited live, in almost real time, within the same split of a second they get recorded and broadcasted.

You already edit the visual reality the very same split of a second you film and transmit it - no noticable time delay.

[the_tyrant]

Guys i have a question
In the book Angels and Demons, it was said that almost no tv station does real "live" broadcast anymore. The book says that its all just a reporter in front of a screen in the studio with fake wind and rain. Is it true?

[Sailor Steve]

Here in Salt Lake they have reporters all over the place all the time. One local station even has them stand in front of the station itself, just so they will be outside. Kind of hard to fake when they're right downtown where everyone can see them.

[vienna]

There is something I have often noticed on television news programs, at least locally here in Los Angeles. Say a news event occurred at 9:00 am and it was basically (visually) over at 10:00 am. If you turn on the news at 11:00 pm, there will be a report on the earlier incident with a live reporter at the scene long after the event is over and there is nothing to be seen or shown. This sort of thing happens all the time here and I have often marvelled at the reporters who have had to stand in some adverse conditions and report on something that could have easily been reported from the studio. I also wonder about the expense incurred to send a news truck, technician, and reporter to the scene. It seems senseless to expend so much time and effort to broadcast nothing at all.

[Skybird]

Guys i have a question
In the book Angels and Demons, it was said that almost no tv station does real "live" broadcast anymore. The book says that its all just a reporter in front of a screen in the studio with fake wind and rain. Is it true?

Moderation of news, and other studio productions, are being done in blue box or green box studios indeed. The backgrounds can be animated or still, can be real video or computer graphics.

The most modern and versatile TV studio in Europe is said to be that of the second public TV channel in Germany, ZDF (at least that's what was said when it was introduced a year ago or so). Below, you see the "green hell", as the moderators call it (because they need to orientate their mimic and gestures so that they meet the superimposed images on the final TV screen, but the moderators only see the green background in the studio), and the standard studio theme that the TV audience is seeing instead. It can be replaced with anything else.

http://d1.stern.de/bilder/stern_5/rueckblick09/Hypes/zdf_maxsize_330_330.jpg

http://www.netzeitung.de/articleimages//24/24228602458957133620.jpg

Camera-teams out there on the street and filming on location, still are needed to get the film material, though. However, computer graphics are being used for illustration purposes as well.

[Penguin]

ok, I am back and after a meal consisting of more than 2 chocolate bars I am able to think better and realize the essence of this news.

I made a fault in my earlier thoughts. Actually I was discussing this news with a collegue of mine and we both made the same mistake, so I will try to explain it:

I thought that you actually needed to record at least one frame of the picture which you want to manipulate. This would be easy to detect, as you would then either dump the one unmanipulated frame (the non-smart way) or show the previous frame twice. Both are not hard to detect for the trained eye.

Actually in theory the invention works like this:
Say you are a friendly dictator who wants to hold a speech in front of the masses. And say that you have a healthy hatred against persons with blue hats. So you feed the image of objects who have the size and the colour of a person with a blue hat to this software.
The camera shows the masses listening to your speech, suddenly an insurgent with a blue hat jumps into the picture. The image of the person travels with the speed of light to the lens of the camera which catches this image. Before the image gets recorded or broadcast, the software manipulates it and changes the image by cutting out the person and fills the hole with something that makes sense - maybe a person with a red hat ;). This all happens in the timeframe (40ms) before the next picture is catched by the cam. So the picture that gets recorded is then indeed the new, altered picture.

Hope I made it a little clear. However my main reasons why I am not really shocked stay the same. Whenever you take something from a picture or add something to it, the principles are the same and used since years. When you add something you put something in front of the unedited picture - just like adding a new layer in photoshop. When you cut something out, you fill the hole with a new background - ideally with exactly the same background. The beaming in the Enterprise series worked like this, they just used an recorded image without the protagonists, but this is an old trick, used many decades before.
If you are not able to pre-record this image, the software has to fill the gap. To fill the gap you wopuld need exactly the same camera angle and position. Even a camera just behind the object you want to delete would show a slightly different picture, so you cannot use that. Imagine now person standing in front of a tree on a windy day, I really would like to see a believable manipulation of this and be able to examine it.

With the right instruments, you are basically able to analyze every line or column or pixel of the picture. Manipulations are detectable, but it would be arrogant to say that this is always the case.

I am worried about censorship/media manipulation to at least the same degree as Sky is. But us TV-clowns manipulate reality all the time. Beginning from the angle, point of of view, image section of the camera, colour settings, the choices which images are shown, the editing of recorded material, the audio mix, etc., etc. It doesn't mean that i am desensitived to these issues, but it's my daily work.

And regarding the news I find it interesting which events are not shown and which are.

hmm, answering to Sky's threads results in the same amount of words like his posts :03: - didn't even come to answer the delay/teletext/time issue, next post

[Penguin]

I have digital cable TV. Before, I had regular (analogue) cable which did not need a set-top-box.

When I now compare my clocks (radio-controlled and thus: precise) with the TV clock before the 20:00 news, for example, I realise that digital cable TV transmits the signals with a delay of around 2 seconds. It lags behind. All programs I compare with analogue cable, lag behind by 2 seconds. With analogue cable and in the past: radiowave-transmitted TV, there was no such delay. Until today I can demonstrate that difference between digital and analogue cable.

I would not be able to notice it with a delay of not 2 seconds but just 40 ms.



The time differences between analogue & digi cable are mainly the result of various de- and encoding procecces of the didital stream. However usually also the stream which will be transmitted to you analogue is usually digital before and gets converted before by your cable provider.

But you raised an interesting question for me: At which time the teletext gets embedded into the different transmission signals and if the embedded time signal gets an slightly "advantage" so that it reaches the consumer at the right time. Have to check it out with the people who work in the control room of the channel.

PS: The camera zoom at the beginning of the ZDF news is one of the worst which I have ever seen. The travel speed of the camera, the desk and the background differ totally! Maybe they changed it meanwhile, but in the first weeks it looked awful! Bah! :x

[August]

When the americans would have been prepared at the superbowl broadcast some years ago, they could easily put in a blur to prevent the nipplegate-scandal which turned several people blind :D

A Jackson nipple once seen, cannot be unseen! :dead: :DL

[August]

Actually in theory the invention works like this:
Say you are a friendly dictator who wants to hold a speech in front of the masses. And say that you have a healthy hatred against persons with blue hats. So you feed the image of objects who have the size and the colour of a person with a blue hat to this software.
The camera shows the masses listening to your speech, suddenly an insurgent with a blue hat jumps into the picture. The image of the person travels with the speed of light to the lens of the camera which catches this image. Before the image gets recorded or broadcast, the software manipulates it and changes the image by cutting out the person and fills the hole with something that makes sense - maybe a person with a red hat

Your analogy makes sense but the filler has to not only be the same general size and in scale with the surroundings it also has to have a valid purpose for being there.

For example you can change the hat color but you also have to provide a plausible reason for the red hat jumping into the picture, as well as the Dictators security team and the crowd responding to the presence of the now deleted insurgent.

Now all of this could certainly be achieved given enough time for the manipulator to consider all the angles and ramifications but trying to do it all live just seems to me like a recipe for getting caught doing it.

[TLAM Strike]

Guys i have a question
In the book Angels and Demons, it was said that almost no tv station does real "live" broadcast anymore. The book says that its all just a reporter in front of a screen in the studio with fake wind and rain. Is it true?

You can try asking Bob Woodruff whether or not his reports from Iraq were live. :03:

[Skybird]

The software here works slightly different.

It first redcues the overall detail and resolution of the material - in real time.

It then automatically fills the defined area you want to clean of a given object with an auto-generated pattern that is calculated by the non-deleted background nearby - in real time.

It then increases the resolution again. By that, the difference in detail and quality between the edited and the nonedited parts of the material get reduced even more. Again: in real time.

As far as I understood it, the software is capable to recognise all by itself the once marked object. It automatically runs the above descriobed procedure on the object in later frames, even when the camera moves and the viewing angle changes: in real time.

Real time means: a delay of not more than 40 ms.

Note that in the demonstration video, it still does not work perfect - you can see a dark round sport in the place where the computermouse used to be. These artefacts, they say, will be overcome in two years or so, they estimate.

Consider what this means for silhouette recognition, and face recognition. Combine that with thbis new technology, and you get an impression of what the future holds. For example you will film a meeting of persons with live link to TV, and the wanted person you want to hide from the audience - simply is not visible in the signal output the camera and video unit transfers to the transmitter. All this with a time delay of 40 ms only, so that nobody could get the idea that what he sees is being manipulated - it is "live", right down to the second, isn't it!?

Considering the exploision in computer potency in the past 20 years and the unbelievable detail and realism of computer graphics being enabled by this, I would not rule out anything anymore.

[Penguin]

Your analogy makes sense but the filler has to not only be the same general size and in scale with the surroundings it also has to have a valid purpose for being there.

For example you can change the hat color but you also have to provide a plausible reason for the red hat jumping into the picture, as well as the Dictators security team and the crowd responding to the presence of the now deleted insurgent.

Now all of this could certainly be achieved given enough time for the manipulator to consider all the angles and ramifications but trying to do it all live just seems to me like a recipe for getting caught doing it.

In my example I meant that you actually paste in another person who wears a red hat. Changing jsut the hat colour is rather trivial.

Somehow your example with the security and crowd response made me laugh, imagine all these people have to be deleted/exchanged too. I see the picture of waving masses, where eventually one after another disappears - in the broadcast as well as in real life, to make sure that nobody could tell the story... :DL

The believable filler is what I talked about when I mentioned the example of the person in front of a tree in the wind. The leaves in the background era are in constant motion, really hard to compute in real time. Remember the software only has a limited time to compute, and we however have all time in the world to analyze the footage later, frame by frame. When the filler is artificially made, you will be able to see a pattern or a discrepancy when compared to the natural background.

Then another problem comes into play: light. In a natural surrounding we have a constant change of light, whenever any pixel of the person or the background changes its exposition intensity you need a constant new calculation of the object that is replaced. That's why for example a blue box works only well with constant, defined light - on the object as well as on the background.

And this is also what makes object recognation so hard. An example would be you have a white box which you film in daylight. You have a shape and use trackers to define its dimensions for different angles. The white balance of the camera is set to daylight, so that the box appears to be white on your monitor. Then your evil little brother wants to mess with you and turns on his powerfull (conventional) flashlight. The light it emits is more yellowish than daylight so the white box suddenly appears in another colour. The box is not recognized anymore, as you have told the recognation software it should track a white box.

Another point, when we disregard webcams/cell phone cams, which run indeed on 25 fps, we are talking about a refresh rate of 50Hz or 60 Hz. This leaves us a theoretical timeframe of only 20ms or 16ms. In reality we have a timeframe of 18.4ms here, for a number of reasons, I don't know the exaxt timing for 60hz countries. With the already mentioned calculations that get dramatically more complicated, we are talking about an algorithm thats gets more complex in an exponential way. For an exponentially growing algorithm the processing time can be shortened only in a limited way by raw cpu power. But this goes however more into theoretical informatics, just think of the old story with the rice and the chessboard.

[August]

In my example I meant that you actually paste in another person who wears a red hat. Changing jsut the hat colour is rather trivial.

Well I know that. Sorry if I made you think otherwise. I was just trying to be as brief as possible since the Walls of Text some folks like to post here are often used as a way of making their arguments too much work to respond to.

Somehow your example with the security and crowd response made me laugh, imagine all these people have to be deleted/exchanged too. I see the picture of waving masses, where eventually one after another disappears - in the broadcast as well as in real life, to make sure that nobody could tell the story... :DL

Is there any real difference if the modifier does it on a "live" broadcast or a recorded one? In your scenario what extra value is gained by being able to doing the modification immediately instead of the few seconds/minutes it takes now?

Only a fool puts his complete faith in a recorded image, "live" or not. This has been true since images were recorded on stone and tapestry. I don't see how anything is changed.

[Penguin]

Well I know that. Sorry if I made you think otherwise. I was just trying to be as brief as possible since the Walls of Text some folks like to post here are often used as a way of making their arguments too much work to respond to.

Is there any real difference if the modifier does it on a "live" broadcast or a recorded one? In your scenario what extra value is gained by being able to doing the modification immediately instead of the few seconds/minutes it takes now?


No need to apologize! Usually my responses are shorter, regarding this topic I had the need to explain it a little further.

Nah, when we speak about manipulation it doesn't make a difference if the pics are edited live or not, but this thread revolves around an invention that claims it could manipulate live footage.



Only a fool puts his complete faith in a recorded image, "live" or not. This has been true since images were recorded on stone and tapestry. I don't see how anything is changed.

this man speaks the truth, listen to him, folks! :up: