Former SH5 Producer Alexandru Gris talks about Software-Quality

[Seeadler]

Former SH5 Producer "Alexandru Gris" writes on his blog about Software-Quality:

Software-Quality (1)
Software-Quality (2)
Software-Quality (3)

[CCIP]

Very thoughtful posts, but it does make me wonder what the underlying conclusion about his own experience with SH5 is. I mean, it's pretty darn clear that somewhere along the lines, despite both experience and good thinking by the developers, something went very wrong with the project. The quality level we'd expect (especially given his obvious understanding of the notion of software quality) just didn't come together. So what did go wrong and why? Hmm

[Seeadler]

Originally Posted by CCIP

Very thoughtful posts, but it does make me wonder what the underlying conclusion about his own experience with SH5 is.

Perhaps these findings are the summary/results of the project.
For SH5 they come unfortunately too late.

[CCIP]

Yeah. I get the impression that on the development side, the problem could basically be summed up as feature creep at the expense of core focus and quality. Underlying that is the conflicting demands for both new in-depth 'die hard' features and features that appealed to a broader market. The development got too ambitious and started going in too many directions at once, losing focus and sacrificing quality when deadlines started pushing. It's a trap that many game projects, and not just in simulations, have fallen victim to, and I guess for SH5 that's most unfortunate...

[janh]

Originally Posted by CCIP

Yeah. I get the impression that on the development side, the problem could basically be summed up as feature creep at the expense of core focus and quality. Underlying that is the conflicting demands for both new in-depth 'die hard' features and features that appealed to a broader market. The development got too ambitious and started going in too many directions at once, losing focus and sacrificing quality when deadlines started pushing. It's a trap that many game projects, and not just in simulations, have fallen victim to, and I guess for SH5 that's most unfortunate...

Yes, I recall Elenaiba hinted in a post towards that direction.

But it is probably a very difficult balance to strike between ambitiousness, novelty, and focus on old core game concepts. If you focus too much on getting the "old core" right, and update basically only a very few features like the outward appearance, then you basically only recreated an old game, and will likely not sell a lot (unless the predecessor is 20 years old).

If you become too ambitious and try to go well beyond the previous technological stage, you run the risks mentioned above. You'll need a lot of standing power and investment to pull ambitious projects off, and a lot of time (I get the impression Oleg just pulls that off with the new BoB).

If I compare games in the past 3-5 years to most typical games in the 90s, like TF1942, Silent Hunter 2, Gunship 2000, Pacific War, etc., then I find that many games roughly starting with the generation Falcon 4.0/Flashpoint/Silent Hunter II have evolved into such complex simulations, that topping that is hard. Especially if companies like Ubi decide on a strategy with more frequent releases of a franchise.

If less frequent, you could sell a new "Silent Hunter" (III) just with updated new flashy graphics maybe every 7 years. And even that will be hard since I find graphics sort of have converged, I can hardly see the tiny different between SHIV and SHV, nor would I care in a simulation about minuscule eye-candy differences. But coming up with new features and functionality at a higher frequency in such complex games like the SH series, War in the Pacific (AE) or anything close to that level, is going to be more and more investment and time intensive. I suppose we are up for some stagnation with hard-core simulations and "superdetailed" games in particular, since there seems to be a discrepancy: obviously it will be hard to get a pay back with a small customer ship but high release frequency for high investment costs.

I think SHV was one of victims of Ubi's new strategy. High release frequency, and limiting time and investment with such an ambitious project cannot go along very well -- and being less ambitious doesn't move it very far either, and will with the 5th title in a series (yawn...) not gain you too many new sales.

[Arclight]

From what I gather from the posts, feature creep was more of a symptom. The real problem was losing track of actual quality; they believed they had a higher quality than they did. So they started building on this (believed) stable core, only to find out at the end the quality was lacking.

Imho at that point they should have decided to delay the release, but that didn't happen.

One can have a standard they believe in.
Be it Software, Construction, or what have you.
If one works for someone else who signs the pay checks?
You do what you are told and damned your ideals unless you go elsewhere.

If higher ups demanded release of SH5 as it was released?
What could any of the Dev's have done?
Find a new job?
(Which would not have stopped release)

If jobs for them were that easy to come by I'm pretty sure they would have moved on.

We all sell our souls for the all mighty dollar at times.
And right now?
Times are tougher then in the past few years.
I know this for a fact being self employed in the Construction business.

I had to layoff all my Guys and desolve my business.

I now work for someone else until things pick up.
I don't agree with him all the time on somethings.
But he writes my check so I do what he wants.

[skookum]

An important point: (I paraphrase) Self discipline exercised by all members of the project is necessary for quality.

This concept goes beyond software development. It's the foundation of all consistently successful ventures undertaken by humanity. If we are committed not only to the goal as a concept, but also to honing the necessary skills and mental sharpness to actually realize it, then the results speak for themselves. Teams that either don't understand the importance of, or are to lazy to pursue self discipline are doomed to achieve mediocrity at best.

[Zedi]

This is when young people without any serious practical background are assigned to big projects. They are good in theory and speeches, but mediocre in the practical field. The end result: SH5.

What is going on in the SH5 moding section is not moding by any means, but a serious repair shop where people spend a lot of their lifetime working free to try fixing a game that was sold on the international market as a finished product. Every [REL] start with "this a fix for... ". Is just 2 bad that we don't have a lawyer among us, so he can sum up all these critical errors in the code and game design and sue Ubi. I seriously hope this was last time when Ubi makes a sim, as it was last time when I bought one of their products.

[BigBANGtheory]

I liked his footnote on page 1:
** Management of expectations and of communities is more important than ever in the days of the Internet.

How many developers and publishers still fall foul of this?

[janh]

Originally Posted by BigBANGtheory

I liked his footnote on page 1:
** Management of expectations and of communities is more important than ever in the days of the Internet.

How many developers and publishers still fall foul of this?

This really still reads weird to me. Does it mean that companies believe that should have people that "tell customers what they ought to desire and what their wishes and expectation should be"? Technically, I would think they should have a good public-relations staff that communicates with fans and customers -- both ways. News and infos on lasted developments to the public, and talking to fans to gather wishes, improvement suggestions, trends in the communities etc that should influence forthcoming developments.

[RobP]

Interesting article.

What it does make clear is that money always corrupts software projects - usually detrimentally unless there is lots of it.

The problem stems from two issues:
1. It is generally acknowledged that the best way to produce high quality software these days is to do it iteratively. The only problem with iterative proceses is that the end date moves(!) This is one concept a lot of commercial people just don't get.

The reason for this is that unlike the requirements for building a bridge, software requirements are a lot more volatile. It is this volatility that needs to be managed and contained - but even when managed properly end dates tend to move!

Why?

Estimates - our point number 2.

All estimates are just that - estimates. They are not predictions! However, many commercial people lack a fundamental understanding of this and indeed go on to enshrine these estimates as fixed project end dates in stone.

At the beginning of a software project it has been proven that many initial estimates are as far out as +-40%!!!!

In iterative development you keep reestimating throughout the project lifecycle to continually refine your estimates - again many commercial people don't get this.

So, the first big problem with most commercial projects is that you are running against milestones that could be +-40% out (in most surveys projects were underestimated) - however, the publisher doesn't care - you signed a contract - they want X delivered on date Y or else!!!

Stacked odds? Now consider the other problem...

Development Lead to Project manager - we estimate that this game is estimated to take 2 years with a team of 10. Project Manager to Development Lead - you got to be kidding right? - Take 6 months off it and I will consider the plan.

Said manager then goes off to sell his modified and much more attractive package to the publishers.... (This is hypothetical btw - I'm not saying this happend in SH5's case... )

The above is pretty normal practice. The only way out of this is to work stupendiously long hours or cut quality/features. Many software houses are famous for the former. They lure 'green' students in with the 'coolness' of games development then screw them. I have never worked in the commercial games industry, but I know people that did and to be honest, they are completely taken advantage of and ripped off.

Some software houses/publishers are so rich, they can adopt iterative development and deliver only when they feel the game is ready. These games are invariably high quality or vapourware(!) If the former they normally sell for bucket loads and make the publishers extremely rich.

When this happens, other publishers see the publisher's success and try to emulate it but aren't willing to spend the same kind of money. So a good game is invariably followed by many poor clones. These clones do make money, but not on the scale of the original software.

I sometimes wish that games publishers knew a little less about bean counting and a little more about games development and games in general - but alas....