Theory or fact? Streaming the world and beyond.. Thread

So, another question, another answer sought.

Streaming: theoretically speaking, isn't it possible to have an "infinite" level?

For example: GTA 4

There are 3 islands streamed together.. in theory, if I or anyone else wanted, would it be possible to stream the entirety of new york if done correctly? with the only down-side being an amazingly huge file?

Or am I wrong?

Another example: would it be possible to take Fallout 3s' map, copy it 3 times, and stream all four maps as one giant map?

Another thing about streaming: would it be possible to stream a galaxy?

Example: 30 planets, three solar systems; each planet is one giant map and each solar system a map as well. The only "rough" transfer would be traveling from solar system to solar system (system->system would NOT be streamed).

Is that possible? In theory?.. because I realize making anything of that scale is facking HUGE. Each solar system or planet would have to be a release in themselves if something of this scale were to be done.. correct?

I ask this streaming question in regards to my own idea. I just want to know my limits.. if any, lol

one thing, please remember before you answer: forget practicality; I'm assuming that the person(s) doing this/these size map(s) had the corresponding script events, story elements etc to make use of these huge lands/maps (IE - everything is of importance to the story and gameplay)

thanks for any input

Edited by: SeCALPHA1

It's purely a time and effort problem. You could do it, but I can't see anyone having enough time to actually do so...

(unless they use procedural generation - check out Infinity QFE here on moddb for wip screenshots and videos of a whole streaming galaxy )

You would need a custom engine to handle maths precision issues in a world of such a large scale. It's something a coder can solve, but it has to be done right down to the very lowest levels of the engine design. To go back to Infinity, note that it is a custom, standalone engine - they designed it in from day one.

Edited by: Gibberstein

So, essentially, you're saying I can? (lol)

Thanks for the input; I was, more specifically, trying to find this out in part of one of my ideas. I had always thought that something like a city with a free-way going through it just wouldn't make sense if "for some strange reason" it were blocked off and why you can just JUMP OVER a road block.. so I wanted to stream two cities together instead.. and I never liked the idea that, coincidentally, the "world" could exist on three happy islands that just so happen to be perfectly existent in the middle of the ocean (*coughgtacough*)

Thanks Gibbs

Well you would not even need a single large file, but many smaller ones, each holding a piece of the map that youre currently in. You could make the planet earth and cut it up into 10,000 maps if you really wanted to...but as Gibberstein said, it would just take a lot of time and effort to do, and do right. Games like Gun, and Shadow of the Colossus use streaming very effectively.

Edited by: SolarNimion

gun not realy
if you where fast enough you would enter a part where the ground texture aint streamed
if you are REALY fast you might end up with some missing objects
both of them dont happen alot tough

Lineage 2 uses UE2.5 and streamming to create its huge worldmap, however, as they add more map blocks and more stuff to the game, more and more ram is needed. I guess, there is some overhead that prevents games from being infinitely big.

Not really. What you need is simply keeping only the X times X blocks ( I call them world sectors in my game implementation ) in memory and load the next blocks once the player moves into their area of influence. And those moving out you persist to file. Granted if you want to have an entire galaxy continue working with AI and alike it's going to be a RAM problem ( and first a CPU problem ) but what goes for geometry, rendering and close to player active AI this is no real problem.

What goes for precision you can simulate ( using doubles as in my engine ) up to 1 micro meter precision an area of 1 million km. With 1mm precision even 1000 million km. If required you can also move the blocks around hence make the world and everything in it move instead of the player. Not sure though is a game used that approach but I would bet on it somebody did.

The only limits are the hard disk space (and DVD space if players buy it the old fashioned way, or download bandwidth if delivered from an online store) to store stuff, and artist time to produce stuff. Everything else is (relatively speaking) simple as long as you design it into the engine from the beginning.

Or you go the procedural route, which cuts down on both storage requirements and artist time, but needs a very specialized game design and engine.

carmack @ id is already working on something like this, or at least planning. Similar to how google earth/nasa worldwind works. A more complex mega-texture type system (which works that way on a limited scale).

I wouldn't be suprised if it can be don with modern engines & clever techniques, but like said, it may not be worth the time at the momemt.

You may not realise it, but Gears of War is an example of a game that streams a lot. We're using a similar technique in DataRun - but at some point in time, we always have a loading screen here or there and break things up a bit. There's only so much you can stream, before you start compromising the visual quality and object density of your world.

Those limits are limits of the UE3 implementation of streaming, not streaming in general. However for gameplay purposes loading screens often make sense (cos they move you from one environment to another really quickly), so there's little incentive for engineers or artists to work on the problem. For example, Far Cry 2 would not be a better game with even more uneventful open space to traverse through. Likewise the other big Ubisoft game of recent-ish times I've played, Assassins Creed. Note that in Far Cry 2, they added the bus stops, which are essentially a loading screen that takes you from one place to another which you could have travelled to without a loading screen. Loading screens aren't always a technical limitation, sometimes they are a deliberate tool to skip something that would have been boring. Often they are both

Engineering-wise, we can do very large streaming spaces. Engineers rarely do engines that do this though, and artists don't spend time making all those assets, as the game design people haven't found a good purpose for all that space yet.

That's not what I mean - it is a general note with streaming - that you can only stream so much in the time it takes for a player to move from one space to another. A world that is very densely populated with art assets is one that is going to require more time to load and swap out required material from memory. Thus, your upper limit is always going to be defined by how quickly your player can traverse your world, and how well populated and detailed your environments need to be.

Apologies, I misinterpreted. You are absolutely spot on there - while size is mainly a design and art problem, movement speed is definitely subject to technical constraints.

Gibberstein wrote: ... For example, Far Cry 2 would not be a better game with even more uneventful open space ... Note that .. added the bus stops, .. are essentially a loading screen that takes you from one place to another ... Loading screens aren't always a technical limitation, sometimes they are a deliberate tool to skip something that would have been boring. Often they are both

Engineering-wise, we can do very large streaming spaces. Engineers rarely do engines that do this though, and artists don't spend time making all those assets, as the game design people haven't found a good purpose for all that space yet.

I've found ideas. Indeed, I truly believe I have =] I also have the time. Drawings. Everything.

I suppose another question is in order: what are the technical limitations of Source?

I've recently come to believe that the Source engine is somewhat buggy and therefore unstable to enable a user to create specific parameters pertaining to overall scale (read this in a forum somewhere.. i think possibly here on ModDB). What exactly are those limitations though?

I don't know the exact figure, but Eternal Silence pretty much represents the most you can push out of Source without a loading zone somewhere, and that's a simple environment with lots of empty space.

Just how big are you thinking about? You may have to scale it back to what the common engines are designed for. Otherwise, you'll have to ask DragonLord if DragENgine is ready for general use yet.

Final option is Morrowind/Oblivion. They have the scale, but put very inflexible limits on the gameplay design.

well I'm starting a mod project with L4D (Left 4 Dead) and I had ideas about more open maps. I would've liked to have implemented a lot more indoor areas with a much more expanded free-roam campaign instead of the more linear vanilla maps as of now.

I was preferring something along the lines, in respect to size, maps roughly 4 times bigger than standard. More or less a campaign spanning the entirety of a city rather a particular section. This, to me, would make a great addition to the survival aspect of the game, considering I have plans for added weapons, items, enemies etc while still keeping the arcade-y feel and overall sense of coop satisfaction.