Making an engine? Thread

To cut a long story short, I have been thinking about making my own engine and tools for a while. The reasons vary, but 2 of the main reasons are I'm getting sick of doing work-a-rounds to get my games to work on existing engines (and not in the way I'd like), and I'm sick of not knowing how or why something does or doesn't work.

The problem is, the idea is kind of scary. I did make a 2D game in XNA, but aside from that my programming experience is mostly in other engines and mostly 2D or demo kind of stuff.

I have little to no preference in terms of language, but I have no idea where to start.

Edited by: SabreXT

I'm in a similar situation as you, but in terms of theory I've read a lot, so I can give some suggestions what you can do get started. Be warned though: if you want to do this properly, you'll need to do a stupid amount of reading and understanding of various topics to grasp the task of creating a game with a new engine.

Besides that, prepare to filter what you read very often, you won't need everything. Focus as best as you can on things that your gut feeling tells you is important for your project. To be able to do that, outline what exactly you want to do. What will the project need and such.

Theres tons of information and many ways to do it, and it will take a lot time to swim through all that to get where you want to.

I recommend starting with a few books:
-Game Coding Complete (Tells you everything about what goes into coding a full game, very good for grasping the big picture and high concepts)
-Head First C# (How to code in C#, excellent book with a very personal style of teaching, you'll love it)
-Game Engine Architecture (Similar to Game Coding Complete, but focuses on game engines specifically, lot of people have recommended this)

Good luck man!

Edited by: Reactorcore

SabreXT wrote:
The problem is, the idea is kind of scary. I did make a 2D game in XNA, but aside from that my programming experience is mostly in other engines and mostly 2D or demo kind of stuff.

that's actually the most important part when you want to create your engine, you can just read so much theory, but when it comes to writing your own engine, experience with other engines is priceless.

I would suggest you to get some open source engine and modify it until it's the way you need. you'd anyway need to 'refactor' your own code, it would be an illusion to think you can write software in the first go perfectly fine (and if you think that afterwards, you probably reached your skill-saturation quite early). having a basic framework to work with is very useful, there are tons of tiny issues that you'll have in your source, but the worst is usually to get anything running, you cannot output anything to validate if parts are ok, you see a black screen, could it be wrong API usage? some math lib has somewhere a typo? matrix multiplication in wrong order?.... ?even those great big engines usually build on top of some other engine (e.g. source engine has quake 2 sources). it's actually good practice to not waste time by starting by 0.
as of the language, it depends on what platform you target. if android only, java is fine. if XBLIG, c# is your only choice, if Windows, C++ and C# is good. if you want iOS, objective-c might have the best integration into xcode. other platforms, especially multi platform, c++ seems to be best portable, using opengl es. (I guess we can exclude web games?).
good luck!

Wouldn't building on another engine cause all kinds of technical, moral, and legal complications?

If the engine is free software / open source and you stick to the license terms ( GPL, GPL-L or whatever ) then it's not a problem. But fragmenting free software projects is not the best idea. The basic idea is to make something good in one project and improve it instead of fragmenting it all over the place confusing the later users. That's why I'm not fond of the existing free software engines as they are fragmenting around instead of making one engine containing all the goodies in one place.

If you're wanting to create a 2D game engine, you should have a look at my engine. It's built in SDL & Open GL. It's very fast and component based. The full source-code is available for free, aswell as an .NET Frontend similar to UDK.

Might be useful for you to check it out to give you an understanding of what goes into making the engine. The next version will feature A* pathfinding.
Marty-green.co.uk

Edited by: Kamikazi[Uk]

SabreXT wrote: Wouldn't building on another engine cause all kinds of technical, moral, and legal complications?

not really.

you have technical complications also if you write it all on your own, it's smarter to read up some engine sources and learn possible ways to solve something, as there is a high chance, other have solved technical complications you might face and there might be some really good solution. reinventing your own wheel is fun, but might not lead to success.
I don't see moral issues either, a lot of open source projects are meant for education. ID software releases their engines so they know how it was written for real games. I'd say it's morally good to use that and maybe write a "thank you" mail to them.
I also don't see legal complications, not more than with your own source. you know you can e.g. use 'zip'/deflate for your archives, it's not incredibly awesome and just some 'out of the box' lib, not your own, yet a lot of people use it. you can also try to run your own implementaions e.g. of lzw or some aritchmetical encoding, but those two are patent protected and you might really end up in legal complications (although the lzw patent expried, afaik). open source is more often than it's not, a legal protection.

@Kamikazi
very fast and .NET you say?

naret wrote:

SabreXT wrote: Wouldn't building on another engine cause all kinds of technical, moral, and legal complications?

not really.

you have technical complications also if you write it all on your own, it's smarter to read up some engine sources and learn possible ways to solve something, as there is a high chance, other have solved technical complications you might face and there might be some really good solution. reinventing your own wheel is fun, but might not lead to success.
I don't see moral issues either, a lot of open source projects are meant for education. ID software releases their engines so they know how it was written for real games. I'd say it's morally good to use that and maybe write a "thank you" mail to them.
I also don't see legal complications, not more than with your own source. you know you can e.g. use 'zip'/deflate for your archives, it's not incredibly awesome and just some 'out of the box' lib, not your own, yet a lot of people use it. you can also try to run your own implementaions e.g. of lzw or some aritchmetical encoding, but those two are patent protected and you might really end up in legal complications (although the lzw patent expried, afaik). open source is more often than it's not, a legal protection.

This is not quite correct. Existing engines tend to solve a single use-case with a single algorithm that behaves bad in other cases. Hence existing engines work only in one use-case but not the other and to make things worse one engines use-cases conflicts with the use-cases of another. That's why building on existing engines (currently released engines) ends in a knit-work and dropping features left and right because the use-cases just don't line up or are horribly slow because the algorithm used is too specific to be of use for anything else. Furthermore with free software you don't get into legal problems because the copyright is clip and clear. With GPL you can only use GPL or more free. You only get in troubles if you try to breach the GPL for example as some try at times. Then you get a shitstorm otherwise not.

naret wrote: you have technical complications also if you write it all on your own, it's smarter to read up some engine sources and learn possible ways to solve something, as there is a high chance, other have solved technical complications you might face and there might be some really good solution. reinventing your own wheel is fun, but might not lead to success.

I get that, but the difference is I'm not looking to reinvent the wheel. Rather, I'm sick of fitting a wheel, then learning the tyres I wanted to use don't fit, so I have to go without, and the bolts are the wrong kind and require the use of a special tool which I hate. So I want to make a wheel that fits the tyre and uses the bolts I like.

I wrote my own game engine in C++ when I programmed the game at Dominoze.com by myself. It was built on top of the OpenGL, OpenAL, and the Newton physics engines. You'll surely be building on some kind of framework or API; maybe it's just a matter of finding the one that does what you really need it to do, but little or nothing more.

Can you tell us more about what the next couple of games are that you want to write? Are they all going to be 2D? Side scrollers? Shootem'ups? Is net play involved or all single player (or multi-player with USB controllers)?

Dragonlord wrote: This is not quite correct. Existing engines tend to solve a single use-case with a single algorithm that behaves bad in other cases.

sure, and if you write your first own engine, it's a real time saver, to not implement the same algorithm yourself, just to figure out what you can check with an existing engine in 1% of the time. It's also not wise to implement some mind blowing algorithm 'just in case' without any profiling. that's another reason to look into professional engine source. if you for example imagine "how do I figure out what objects is affected by which light source for rendering?" and even if you ask in some forums, you might end up creating some kd-tree or octree or... but if you look in one of the fastest engines out there, Quake3, you'll see they just check every object against a simple array of lights and it's fast enough. (was the first thing I've checked when it came out, as I always wondered). if in your case it becomes slow, you can still improve it.
I agree that most algorithms have good and bad use cases, but nothing prevents you from trying some engines for the use cases you'd like to support and learn from their success and failures, instead of experiencing it the hard way. (learning by your own failures is the most primitive way of learning).

Hence existing engines work only in one use-case but not the other and to make things worse one engines use-cases conflicts with the use-cases of another.

the question would be, if you write your first engine, how would you know where to spent your time, what algorithms to use? if you try to do brain surgery, or defense in curt etc. you usually learn by previous cases, that's what other source-codes are for programmer.
of course, it's important to know the theory, but seeing the practice isn't minor important.

That's why building on existing engines (currently released engines) ends in a knit-work and dropping features left and right because the use-cases just don't line up or are horribly slow because the algorithm used is too specific to be of use for anything else.

well, if that would be the cases, there would be no engine licensing, cause regarding that logic, it's impossible to reuse engine. the question would be, why is anyone trying to write reusable libs like that at all?
I guess that proves that most of the industry disagrees with that point of view. Some just take engines 1:1, e.g. a lot of Unit3D games. You don't have to do that if you are a programmer, but remove unused parts, replace parts that are not suitable for your case or you write new parts that haven't been there, still even the Source Engine has Quake1 parts in it, so I have a strong believe, if "Pros" are doing it all the time. Why not learn from someone who wrote a whole engine, if you write your first?

Furthermore with free software you don't get into legal problems because the copyright is clip and clear. With GPL you can only use GPL or more free. You only get in troubles if you try to breach the GPL for example as some try at times. Then you get a shitstorm otherwise not.

sounds like you had bad experience with that particular one. I hope you don't try to imply there is just GPL or die.
Garage games just released their Torque 3D open source, with MIT license:
Garagegames.com
really great source for learning!

SabreXT wrote:

naret wrote: you have technical complications also if you write it all on your own, it's smarter to read up some engine sources and learn possible ways to solve something, as there is a high chance, other have solved technical complications you might face and there might be some really good solution. reinventing your own wheel is fun, but might not lead to success.

I get that, but the difference is I'm not looking to reinvent the wheel. Rather, I'm sick of fitting a wheel, then learning the tyres I wanted to use don't fit, so I have to go without, and the bolts are the wrong kind and require the use of a special tool which I hate. So I want to make a wheel that fits the tyre and uses the bolts I like.

I really understand that it can make you sick fitting the wheel, but I didn't try want you to use an engine and fit it to your case, but instead use it as a starting point to write your engine, replacing it all, but just step by step so you have always something to test and try out.
you know, inventing a wheel, does imply, you have the whole car also. building the whole car on your own from scratch and trying to have a ride the first time, will nearly always fail. replacing part by part might be simpler.
that's all I'm saying.
I'd also always go for your own engine

Gamieon wrote: I wrote my own game engine in C++ when I programmed the game at Dominoze.com by myself. It was built on top of the OpenGL, OpenAL, and the Newton physics engines. You'll surely be building on some kind of framework or API; maybe it's just a matter of finding the one that does what you really need it to do, but little or nothing more.

Can you tell us more about what the next couple of games are that you want to write? Are they all going to be 2D? Side scrollers? Shootem'ups? Is net play involved or all single player (or multi-player with USB controllers)?

Sure, my current plans are as follows.
-An old school horror game. Resident Evil 1-3, Silent Hill 1-3, Dino Crisis 1, Alone in the Dark 1, that kind of thing.
-A Die Hard style action game with maybe some minor adventure and stealth stuff, but mostly punching dudes in the face. Batman would be the closest example.

Both single player. Both third person. Both 3D. Both with controllers if I can work that out.

The horror game is a problem though due to most engines being really picky about shadows and/or lights.

Edited by: SabreXT

naret wrote: sure, and if you write your first own engine, it's a real time saver, to not implement the same algorithm yourself, just to figure out what you can check with an existing engine in 1% of the time. It's also not wise to implement some mind blowing algorithm 'just in case' without any profiling. that's another reason to look into professional engine source. if you for example imagine "how do I figure out what objects is affected by which light source for rendering?" and even if you ask in some forums, you might end up creating some kd-tree or octree or... but if you look in one of the fastest engines out there, Quake3, you'll see they just check every object against a simple array of lights and it's fast enough. (was the first thing I've checked when it came out, as I always wondered). if in your case it becomes slow, you can still improve it.
I agree that most algorithms have good and bad use cases, but nothing prevents you from trying some engines for the use cases you'd like to support and learn from their success and failures, instead of experiencing it the hard way. (learning by your own failures is the most primitive way of learning).

I had to invent most of the stuff I use in my engine as the way existing engines use them is so tightly limited to a single use-case it breaks when you try to do something creative with it... and I need full creativity support as I hate dancing around random limitations put up because of a single use case algorithm. Existing engine source code is way too optimized for a single use case. This is no suited for learning as you end up learning a complicated algorithm you can use in only one case... and this is to 99.9% not the case in your project for sure.

the question would be, if you write your first engine, how would you know where to spent your time, what algorithms to use? if you try to do brain surgery, or defense in curt etc. you usually learn by previous cases, that's what other source-codes are for programmer.
of course, it's important to know the theory, but seeing the practice isn't minor important.

As mentioned above the code is tailored to a single use case. The only thing you can learn from that is how somebody tried to solve a very narrow use case. That's not really useful for somebody new to the track. I've got a master in computer science. As such my strength lies in examining algorithms and techniques to figure out what can actually help to reach a goal. If you just pick something others picked without proper analysis you get nowhere as you don't understand what you knit together.

well, if that would be the cases, there would be no engine licensing, cause regarding that logic, it's impossible to reuse engine. the question would be, why is anyone trying to write reusable libs like that at all?

Engines are picked with the question "what existing game is close to my game?". It's about trying to find an engine causing you the least headache to get the job done. In the end you see only the product but not how much one had to skrew with the engine to get there and how much had to be cut because the engine can't handle it well (or not at all). Licensing is cheaper than making your own engine. The resulting games are though unfortunately most of the time poor as most energy is wasted fighting against the engine than making a good game.

sounds like you had bad experience with that particular one. I hope you don't try to imply there is just GPL or die.

I hope you don't try to put words into other people's mouth? I used GPL as an example. The problematic is the same with others no matter if more or less restrictive.

SabreXT wrote:

Gamieon wrote: I wrote my own game engine in C++ when I programmed the game at Dominoze.com by myself. It was built on top of the OpenGL, OpenAL, and the Newton physics engines. You'll surely be building on some kind of framework or API; maybe it's just a matter of finding the one that does what you really need it to do, but little or nothing more.

Can you tell us more about what the next couple of games are that you want to write? Are they all going to be 2D? Side scrollers? Shootem'ups? Is net play involved or all single player (or multi-player with USB controllers)?

Sure, my current plans are as follows.
-An old school horror game. Resident Evil 1-3, Silent Hill 1-3, Dino Crisis 1, Alone in the Dark 1, that kind of thing.
-A Die Hard style action game with maybe some minor adventure and stealth stuff, but mostly punching dudes in the face. Batman would be the closest example.

Both single player. Both third person. Both 3D. Both with controllers if I can work that out.

The horror game is a problem though due to most engines being really picky about shadows and/or lights.

Go for Unity3D, it'll provide everything you need.
One big challange with that Unity has is how to figure out the "serious game architecture", because most if not all tutorials out there focus on gameplay prototype/just for fun, than how to properly make a proper, serious game that you could actually sell. Once you figure that out, Unity3D can be seriously powerful and useful.

I haven't yet, so if you find out, please tell me too.

Edited by: Reactorcore

I tried unity, good for demos, but I didn't like it. Also, it doesn't have real time shadows unless you pay through the nose or use some kind messy tricks. I could play with it more (I never worked out animation blending for example) or just cut the shadows out and use blobs since they aren't important from a gameplay point of view, but it falls into my original point of constantly wrestling with the engine and tools rather than making the game. Making it unsuited for the horror game.

SabreXT wrote: Also, it doesn't have real time shadows unless you pay through the nose

You mention that shadows are not important to get the game working, so don't worry about them from the start.

The thing that Unity offers is by giving you the basic package to make a game and even be able to sell it. Then, using a part of the revenue, you buy the pro version. They designed it this way to begin with, so its a great opportunity.

EDIT: Also if something is unclear how something works, check out Unity Answers or their forums and ask about it. People will help you if you ask. Like I am doing now.

Edited by: Reactorcore

With other words they want you to make crap first...

How comes I never liked Unity to begin with

Dragonlord wrote: With other words they want you to make crap first...

How comes I never liked Unity to begin with

Thats a wrong attitude to look at it. I mean come on, you don't build the Rome in one day. Its just a matter of implementing certain feature later on. Get the basics down first, use it to fund the pro, then add to it to finish it. Modular design is something Unity supports, so use it to your advantage.

Reactorcore wrote:

Dragonlord wrote: With other words they want you to make crap first...

How comes I never liked Unity to begin with

Thats a wrong attitude to look at it. I mean come on, you don't build the Rome in one day. Its just a matter of implementing certain feature later on. Get the basics down first, use it to fund the pro, then add to it to finish it. Modular design is something Unity supports, so use it to your advantage.

Modular doesn't necessarily equal good design, easy to use or useful. I've seen quite the cluster-fuck already with so called modular code.

Dragonlord wrote: Modular doesn't necessarily equal good design, easy to use or useful. I've seen quite the cluster-fuck already with so called modular code.

What would you recommend then? Not being sarcastic here btw, I'm open for any alternatives you might offer.

EDIT: Its funny you imply that modular design sucks, when your own engine in your signature seems to be the very definition of modular. What do you have to say about that?

Edited by: Reactorcore

Reactorcore wrote: What would you recommend then? Not being sarcastic here btw, I'm open for any alternatives you might offer.

Nothing which exists yet. Make your own if you want to learn or otherwise take an existing engine if you can cope with the problems you noticed yourself. If you decide to use an existing engine you have to look for one which causes you the least head-ache hence matching your game the most closely. This requires reading up on feature pages and looking on forums to see what kind of problems people typically ran into. Later one is the most important for your mental sanity later on

EDIT: Its funny you imply that modular design sucks, when your own engine in your signature seems to be the very definition of modular. What do you have to say about that?

That many think doing something modular makes their design automagically intelligent or good. Problem is that a modular design is easy to fuck up ending up in a crappy cluster-fuck design which most of the time happens. With my engine the engine design and the game development process is the key priority. Modularity is one of the tools required to reach the goal but the primary goal is the design and the way game development is done not modularity just so it looks good on the features page.

Someone wrote: Nothing which exists yet.

Shit. I was hoping you woudn't say that. Back to square one then, once again.

Dragonlord wrote:

Reactorcore wrote:
EDIT: Its funny you imply that modular design sucks, when your own engine in your signature seems to be the very definition of modular. What do you have to say about that?

That many think doing something modular makes their design automagically intelligent or good. Problem is that a modular design is easy to fuck up ending up in a crappy cluster-fuck design which most of the time happens. With my engine the engine design and the game development process is the key priority. Modularity is one of the tools required to reach the goal but the primary goal is the design and the way game development is done not modularity just so it looks good on the features page.

Many things are easy to screw up, but that doesn't make it a bad way of doing things. Its just a matter of getting it right then.

Reactorcore wrote:

Someone wrote: Nothing which exists yet.

Shit. I was hoping you woudn't say that. Back to square one then, once again.

Just be patient if you don't want to do it on your own

Many things are easy to screw up, but that doesn't make it a bad way of doing things. Its just a matter of getting it right then.

That's correct. I didn't see an engine yet though that put it to full use. Compile time modularity is nice but it doesn't go all the way you need and thus doesn't solve the basic problem to begin with.