Welcome, Guest. Please login or register.

Login with username, password and session length

 
Advanced search

1411276 Posts in 69323 Topics- by 58380 Members - Latest Member: bob1029

March 28, 2024, 11:32:08 AM

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsDeveloperTechnical (Moderator: ThemsAllTook)Best cross-compilation setup?
Pages: [1] 2
Print
Author Topic: Best cross-compilation setup?  (Read 1694 times)
oahda
Level 10
*****



View Profile
« on: July 11, 2016, 09:34:34 AM »

Yo. I'm in the process of researching this.

My engine in its early days used to run fine on OS X, Windows, Linux, Android and iOS alike. I've basically spaghettified the code without thinking enough ahead since then that it'll probably only compile properly on a Mac ATM.

So I'm in for some refactoring! And I will do it basically by adding one feature after another back separately, little by little, while making some well-needed changes, to make sure every step compiles fine on every system along the way.

That makes one wonder what the best idea would be as to how this should be set up. Eclipse is a cross-platform IDE that can also do the Android part and so would cover four out of five target systems (iOS excluded, I think?). But Eclipse is horrible. I couldn't work in Eclipse. Xcode is my preferred IDE in general and it also makes app packaging and signing and all of the Mac specific stuff the easiest. So I want to keep working in Xcode primarily.

And even if I were to use the same IDE everywhere, I'd have to set up different projects anyway since each system would need specific dependencies and different types of library files and so on.

Managing an IDE project for each platform seems messy. Especially considering I won't actually be working in the different systems, only compiling: Mac and Xcode will remain my dev environment. So my idea is to do something like what I did when I made a C++ game for both iOS and Android using SDL2 once: I wrote a bash script that generated the makefiles for the Android project and ran it through the Android compiler whenever I compiled for iOS. All I needed to do then was to go inside Eclipse and click the play button and it magically worked on my connected Android device as well. I did have to keep a barebones Android project, tho, but that'd be necessary for the same reasons as the Xcode project(s) for Mac/iOS anyway: easy setup of proper signing, app name, icons, resource packaging and so on.

So I could do something similar, only I would also generate makefiles for Windows and Linux compilation? But would I need to think about similar things (the signing, packaging, icons and so on) on those systems in order to make a serious game/application?

Plus I still would have to keep separate projects going in Xcode for iOS and Mac but there at least I think I'd be able to make both projects load the same hierarchy with some nesting feature or whatever, so I don't thiiiiink that would require additional work beyond initial setup.

What are your thoughts / how do/would you solve this?
Logged

djr
Level 0
***


Smith and Winston Coder


View Profile WWW
« Reply #1 on: July 11, 2016, 09:41:38 AM »

You should probably look at using CMake to manage the generation of project files. It's become the defacto standard for C/C++ projects. It's got a steep learning curve.

Combine that with BuildBot and you'll have your own build farm churning out executables to test constantly.

https://cmake.org/

http://buildbot.net/
Logged

oahda
Level 10
*****



View Profile
« Reply #2 on: July 11, 2016, 09:42:50 AM »

You should probably look at using CMake to manage the generation of project files. It's become the defacto standard for C/C++ projects. It's got a steep learning curve.

Combine that with BuildBot and you'll have your own build farm churning out executables to test constantly.

https://cmake.org/

http://buildbot.net/
Yeah, I was wondering about CMake, if I could make the bash script call that and do something. I'll look into BB too. Thanks!
Logged

oahda
Level 10
*****



View Profile
« Reply #3 on: July 11, 2016, 10:11:05 AM »

So, basically, for my use case, perhaps something like this...

1. Add build step to Xcode that runs a bash script or terminal command when the program is compiled.
2. This bash script or terminal command runs CMake to generate stuff that can cover Windows, Linux and Android.
(3. Possibly runs Android compiler right away.)
4. OS X and iOS are managed as Xcode projects somehow loading the same files without need for individual updates if possible. No makefiles for those two.

I'd only enable this build step when necessary sometimes so that it doesn't slow down regular work in OS X if I don't want to compile for other systems at that moment.

Unless, again, I'm missing something here. Again, wondering if signing, packaging and so on will be subpar/non-standard for Linux and Windows unless I filter stuff through an IDE first (obviously the IDE will be using command-line stuff under the hood, but maybe it's way too much work to do manually?)..?

Then again I think it would be cool to release the engine if people want it in which case a setup that doesn't depend on anything and that uses a more standard procedure might be better. But it seems so much more difficult to set up and get right, again, when it comes to signing and packaging and all of that stuff...

Yeah, so, MAIN QUESTION: how to deal with platform-specific app signing, packaging, icons, resource packing and so on if compiling using make instead of an IDE like Xcode for Mac/iOS or Eclipse/ADT for Android in a way that's not incredibly difficult to set up and not worth the effort of setting up this completely automated, generalised build system? There's also the fact that I want it to be just as easily extensible to any future systems (consoles, for example).
« Last Edit: July 11, 2016, 10:30:58 AM by Prinsessa » Logged

djr
Level 0
***


Smith and Winston Coder


View Profile WWW
« Reply #4 on: July 11, 2016, 10:32:25 AM »

You need to think of CMake as a meta-make system (a make system for make systems). So what does this mean in practice?

You write series of CMake scripts that define how to build a library, executable or data bundle. These are similar to Makefiles but at a slightly more abstract level.

You then run CMake and tell it what generator to use (XCode generator, MSVC generator, UNIX Makefile generator). CMake then makes an xcode project, SLN or Makefile hierarchy for you and you use that.

The benefits are that you only need to define things once and it magically happens on all targets. The downside is that CMake files can be cryptic and platform specific things like signing authorities are not well documented and temperamental to set up.

For Smith and Winston I compile code across OSX, Windows (32+64bit), Linux, FreeBSD and various consoles and I wouldn't want to manage that many targets without CMake. It's worth the cost.

Hope that helps a bit.
Logged

oahda
Level 10
*****



View Profile
« Reply #5 on: July 11, 2016, 10:34:59 AM »

You then run CMake and tell it what generator to use (XCode generator, MSVC generator, UNIX Makefile generator). CMake then makes an xcode project, SLN or Makefile hierarchy for you and you use that.
Yeah, I'm watching a presentation now, and it just got to this part. That's great! Gomez

The benefits are that you only need to define things once and it magically happens on all targets. The downside is that CMake files can be cryptic and platform specific things like signing authorities are not well documented and temperamental to set up.
Well, I can live with going into Xcode to click a button manually if CMake generates the project for me. Do see my edit which I made while you were writing this, tho: how easily would this extend to other systems that I might want to support in the future, like consoles?

Hope that helps a bit.
Definitely! Thanks a lot. c:
Logged

djr
Level 0
***


Smith and Winston Coder


View Profile WWW
« Reply #6 on: July 11, 2016, 10:48:23 AM »

Quote
Yeah, so, MAIN QUESTION: how to deal with platform-specific app signing, packaging, icons, resource packing and so on if compiling using make instead of an IDE like Xcode for Mac/iOS or Eclipse/ADT for Android in a way that's not incredibly difficult to set up and not worth the effort of setting up this completely automated, generalised build system? There's also the fact that I want it to be just as easily extensible to any future systems (consoles, for example).

I'll be honest, CMake is weak at signing and this mostly because they have to reverse engineer how to create the SLN/XCodeProj to reliably get this to work.

For platforms like iOS people have written CMake extension scripts to make it easier: https://github.com/cristeab/ios-cmake (for example). Console aren't really a problem in my experience. I'm not too sure about Android as I've not done any android dev in years.

Since CMake generates platform specific build files it is always possible to edit the XCode Project (after generation) and add in your certificates but that can be overwitten when you next generate the project.

For me, given that signing is something I do infrequently, where as adding files, changing build options and compiling on multiple platforms is a daily chore; it makes sense. Maybe not for you.
Logged

oahda
Level 10
*****



View Profile
« Reply #7 on: July 11, 2016, 10:59:22 AM »

Console aren't really a problem in my experience.
Could you talk a bit more about this or are you tied down by silly NDA's? :c

For platforms like iOS people have written CMake extension scripts to make it easier: https://github.com/cristeab/ios-cmake (for example). Console aren't really a problem in my experience. I'm not too sure about Android as I've not done any android dev in years.
I'll have a peek at that one.

For me, given that signing is something I do infrequently, where as adding files, changing build options and compiling on multiple platforms is a daily chore; it makes sense. Maybe not for you.
I could probably live with that too.

Next issue becomes... My stuff is separated. Engine is one project and game is another using that. So I'd have to make it work with both when I want to compile the game, because I'm making the engine alongside the game and have to recompile the engine basically as often as the game. Working exclusively on the Mac on this project for a long time now, I've simply nested the two Xcode projects inside each other and made the engine compile to a library which the game then links to. Building this double project in Xcode compiles the targets of both and in practice it works as if it was just one project. Really handy. Might be a lot more difficult to get working as nicely in tandem for all platforms using CMake?
Logged

qMopey
Level 6
*


View Profile WWW
« Reply #8 on: July 11, 2016, 11:19:14 AM »

There's nothing wrong with hand-written build files. Some people don't bother with cmake, premake, msbuild, make, or any other build system/build system generator. For me specifically I've tried all of the above and ended up settling on hand-written, small scripts that directly run the compiler.

If using xcode I would probably just setup a hand-written script to run clang, or whatever compiler you're using on OSX. Then I'd have xcode launch this script to build. For other platforms, like Windows, I'd use a batch or makefile to directly call appropriate compilers.

Depending on how you code the actual engine itself determines how simple or monstrous your build requirements will be. Cutting back on dependencies, writing easily portable code, abstracting away OS-specific bits as necessary all make cross platform builds simpler. For example I do my dev work on Windows and make use of dll hotloading for fast iteration time. I don't really care about this feature on any other platform since my work machine is all that matters for iteration time. I don't compile shared libraries at all on other systems and instead just use static linking and one compiler invocation.

Since each platform relies on platform-specific hand-written build scripts whenever something custom comes up, or if a new addition is necessary, it's just a matter of adding exactly what you need. No messing with generalized build systems, referring to build system documentation, fussing around with build system errors.
Logged
djr
Level 0
***


Smith and Winston Coder


View Profile WWW
« Reply #9 on: July 11, 2016, 11:20:27 AM »


Next issue becomes... My stuff is separated. Engine is one project and game is another using that. So I'd have to make it work with both when I want to compile the game, because I'm making the engine alongside the game and have to recompile the engine basically as often as the game. Working exclusively on the Mac on this project for a long time now, I've simply nested the two Xcode projects inside each other and made the engine compile to a library which the game then links to. Building this double project in Xcode compiles the targets of both and in practice it works as if it was just one project. Really handy. Might be a lot more difficult to get working as nicely in tandem for all platforms using CMake?

That's basically what I do. I have an engine made from about 10 static libraries, 5 or so external libraries (Bullet physics, AssimpLib etc) and a few shared libraries (Qt etc). I generate a single XCode project that has two Apps (runtime and editor) and a shed load of libs. Works well.
Logged

oahda
Level 10
*****



View Profile
« Reply #10 on: July 11, 2016, 11:29:22 AM »

In response to qMopey and revisiting my original considerations, if I could somehow get an IDE for each platform to automatically load new source files as they are added so that I can just go in there and click a button every time I want a fresh build, I could probably do that if it's easier than CMake.

CMake does seem incredibly powerful if I can actually get it to obey my needs, tho. As you say, qMopey, every platform might rely on specific stuff, but CMake seems to be able to handle that.

That's basically what I do. I have an engine made from about 10 static libraries, 5 or so external libraries (Bullet physics, AssimpLib etc) and a few shared libraries (Qt etc). I generate a single XCode project that has two Apps (runtime and editor) and a shed load of libs. Works well.
So are you getting the XCode project to contain different targets for each engine library and yet again separate targets for those two apps? Could you share (at least partially, with relevant sections) your CMake setup file(s) on a pastebin or something so that I can get an idea of how that works?
Logged

djr
Level 0
***


Smith and Winston Coder


View Profile WWW
« Reply #11 on: July 11, 2016, 11:50:41 AM »

So are you getting the XCode project to contain different targets for each engine library and yet again separate targets for those two apps? Could you share (at least partially, with relevant sections) your CMake setup file(s) on a pastebin or something so that I can get an idea of how that works?

My project is literally thousands of files but I've made a VERY simple project with two apps and a single library and shared it here: https://dl.dropboxusercontent.com/u/19345/ExCMake.zip

Open that in the CMake GUI, set a build directory (outside of the source tree), click configure and generate and then open it up in XCode.
Logged

oahda
Level 10
*****



View Profile
« Reply #12 on: July 11, 2016, 12:15:35 PM »

So are you getting the XCode project to contain different targets for each engine library and yet again separate targets for those two apps? Could you share (at least partially, with relevant sections) your CMake setup file(s) on a pastebin or something so that I can get an idea of how that works?

My project is literally thousands of files but I've made a VERY simple project with two apps and a single library and shared it here: https://dl.dropboxusercontent.com/u/19345/ExCMake.zip

Open that in the CMake GUI, set a build directory (outside of the source tree), click configure and generate and then open it up in XCode.
That's great! Got it working. CMake ran fine with no manual work, XCode project built stuff fine with no manual work. Nice to see how simple it is at least fundamentally. Any info on how to add frameworks to the project, and how to make sure those do not get added on other platforms? Including non-standard ones like SDL2.

And of course feel free to stop answering all my questions if you want to. You've been very helpful and I appreciate it a lot. But so long as you're willing to respond, I'll keep asking since you seem to know a bunch about this. Will be googling as well, but the nice thing about asking here is that if I return from a failed session of googling, you might have replied with the info I need here instead. c;
Logged

djr
Level 0
***


Smith and Winston Coder


View Profile WWW
« Reply #13 on: July 11, 2016, 12:40:51 PM »

CMake has a command called find_package. So add

find_package(SDL2 REQUIRED)

to the root CMakesList.txt file. If CMake finds SDL2 it sets up some variables ${SDL2_INCLUDE_DIR} ${SDL2_LIBRARY}. If not they are blank and there is a configuration error (if REQUIRED is passed to find_package).

If you only want to include SDL2 on apple platforms I do something like:

Code:
if(APPLE)
    set(MY_TARGET_LIBS ${SDL2_LIBRARY} <other apple libs>)
else()
    set(MY_TARGET_LIBS <non-apple-libs>)
endif()

target_link_libraries(Editor
    ....
    ${MY_TARGET_LIBS}
)

I use this to include the Qt libraries in my editor but not in my game executable.

find_package is usually effective if you've installed the package in a standard place. Look at the docs to see how you can pass it hints to find it in non standard places.

Hope that help, CMake is very powerful and complicated so don't get discouraged when it all goes wrong. Just keep plugging away.
Logged

djr
Level 0
***


Smith and Winston Coder


View Profile WWW
« Reply #14 on: July 11, 2016, 01:01:16 PM »

That's quite a contrived example of course. Say you wanted to link SDL with your game but NOT your editor, you would be better of calling find_package in your CMakeLists.txt file in the game folder and adding it to the target_link_libraries command in there. Hope that makes sense?

Logged

oahda
Level 10
*****



View Profile
« Reply #15 on: July 11, 2016, 01:02:51 PM »

Makes perfect sense! I found this too, basically confirming what you said. I shall be off to bed—many thanks again! Will be looking into this and trying to set it up. Might return with more questions~ 
Logged

BorisTheBrave
Level 10
*****


View Profile WWW
« Reply #16 on: July 11, 2016, 02:00:40 PM »

Quote
IDE for each platform to automatically load new source files as they are added so that I can just go in there and click a button every time I want a fresh build, I could probably do that if it's easier than CMake

CMake should be doing this for you automatically. It creates a build dependency on the original cmake files, so that when you change them, the next time you compile it regens the whole set of build files. Usually you then have to build a second time, but it's nice that you can forget about cmake after you first invoked it.
Logged
oahda
Level 10
*****



View Profile
« Reply #17 on: July 11, 2016, 08:41:01 PM »

Quote
IDE for each platform to automatically load new source files as they are added so that I can just go in there and click a button every time I want a fresh build, I could probably do that if it's easier than CMake

CMake should be doing this for you automatically. It creates a build dependency on the original cmake files, so that when you change them, the next time you compile it regens the whole set of build files. Usually you then have to build a second time, but it's nice that you can forget about cmake after you first invoked it.
Yeah, I'm all set on CMake now thanks to djr's great help, so I'll be experimenting in the upcoming days.
Logged

Cheezmeister
Level 3
***



View Profile
« Reply #18 on: July 16, 2016, 12:35:58 AM »

CMake is pretty nifty, especially the ability to generate project files for Eclipse OR Visual Studio OR Xcode etc. etc.

It comes with solid support for detecting SDL and its companion libs out of the box. Though last I checked it wasn't hip to SDL2 and I had to grab a 3rd-party FindSDL2.cmake. Other libraries can be hit or miss; it's not hard to write your own FindXXX script, but it can be a pain in the butt. As well, CMake likes to break compatibility from one version to the next, and there are subtle differences between platforms that occasionally make for nasty surprises. IME, it almost never "just works" on Windows...but it eventually does work.

I've been meaning to mess with gyp and Premake, which scratch similar itches in a more general way, but haven't had an oportunity yet.
Logged

෴Me෴ @chzmstr | www.luchenlabs.com ቒMadeቓ RA | Nextris | Chromathud   ᙍMakingᙌCheezus II (Devlog)
oahda
Level 10
*****



View Profile
« Reply #19 on: July 16, 2016, 02:11:39 AM »

It comes with solid support for detecting SDL and its companion libs out of the box. Though last I checked it wasn't hip to SDL2 and I had to grab a 3rd-party FindSDL2.cmake.
Aaah, good to know since SDL2 is what I'm using. Thanks!

Other libraries can be hit or miss; it's not hard to write your own FindXXX script, but it can be a pain in the butt. As well, CMake likes to break compatibility from one version to the next, and there are subtle differences between platforms that occasionally make for nasty surprises. IME, it almost never "just works" on Windows...but it eventually does work.
Oh... This sounds very different from what everyone else has been saying now... :/

Going to try to get it to run on web too using Emscripten BTW. Exciting stuff.
Logged

Pages: [1] 2
Print
Jump to:  

Theme orange-lt created by panic