Discussion: Windows MKXP #35
Labels
No labels
RGSS accuracy
bug
compilation
discussion
documentation
duplicate
enhancement
invalid
performance issue
port request
question
ruby incompatibility
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: MapleShrine/mkxp#35
Loading…
Add table
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
been fiddling with building mkxp on windows, after catching a stray uint and trying again, it seems the main issue regarding building on windows is a
gmtime_rconflict that occurs upon inclusion of ruby'swin32.hThe exact error generated can be found in this gist:
https://gist.github.com/ntzrmtthihu777/e81aaa9cd95277bf9991#file-gistfile1-txt
Note: compiled with clang/clang++ for more verbose error output.
found the smallest test program one could write to trigger this output,
compiling with g++ test.cpp -Iruby-2.1.0/ -Iruby-2.1.0/x64-mingw32/ results in the above error
Could you try compiling with
-std=c++98? Maybe there is a reason why this isn't explicitly set. AFAIK mkxp itself doesn't use any GNU extensions.But even if mkxp compiles fine with this workaround, CRuby's header shouldn't unconditionally declare
gmtime_r.ok, will give it a shot and report back. Note, reversing the #incldue directives fixes the issue in the test.cpp program, but its probably too minimal for application in mkxp itself.
Nope, same error as before... this is rather annoying, windows, y u no posix!?
I don't suppose you have access to a windows machine to test with? this is really irksome :/
This is yours yes? https://www.ruby-forum.com/topic/4939821
Also this could be related: Also this could be related: http://sourceforge.net/p/mingw/bugs/1625/ (also this)
Yes, that is mine, and I've seen both of those pages, lol.
gmtime_ris defined inpthreads.handtime.hin the msys2 packagemingw-w64-{i686,x86_64}-winpthreads-gitI'm going to bug the mingw-w64 folks as well
Hmm... I've reported it to mingw-w64 and they're gonna convert
gmtime_rto an inline from a macro, but my ruby bug report has been updated with including<ctime>as fixing the test program.Yep, the patch fixes the gmtime_r issue but raises a new one.
What new issue?
rand(), but including
<stdlib.h>in pthread.h fixes that, now I just have to fix up this sdl_sound issue :pNow the current issue is:
https://gist.github.com/ntzrmtthihu777/3b754e8aea64ad73141f
a whole bunch of undefined reference to ModPlug_* basically.
Oh, you're statically linking. Yeah, the problem is very likely my limited pkgconfig file for SDL_sound which doesn't pass on the correct linker flags in case of static linking. You could compile SDL_sound with only WAV support as a temporary workaround for now, or compile it into a dll instead.
Alternatively, since all this needs is a
-lmodplug, you could either add that manually in the Makefile, or try adding the lineRequires.private: libmodplugto your SDL_sound.pc.yeah. I don't really want to link static, but I can't seem to get sdl_sound's hg version with the patch to compile into a shared library; its driving me crazy, lol.
Success! well, to a degree... It compiles and 'runs', but though I do hear the music and it does accept options from mkxp.conf and opens a window, the window is nothing but solid black...
On another note, setting
-Wl,-subsystem,windowsto stop cmd.exe from opening works to a degree, but unless you invoke mkxp.exe from the msys shell it won't work (eg, double clicking it or running it via rpg maker). In addition, setting FORCE32 won't compile either, resulting in this error log: https://gist.github.com/ntzrmtthihu777/79eaefcb75c9ce551522I was worried that the gmtime_r issue came up again, but it seems you edited the gist and now the errors are gone?
Anyway, I have a slight idea about the first three errors. Can you apply this patch and try again?
Yeah, the gmtime_r issue was there again, but that was because I hadn't patched win32.h for ruby on the 32 bit side.
Ok, will check it out when I get back on my main machine; note, those compilation errors on occur when compiling 32bit.
Nope, exact same output. I would like to mention that appending
-fpermissiveto the arguments does allow it to compile and 'play' (same results as the 64-bit compile) but this is akin to sweeping the problem under the rug and not really fixing anything.If possible, I'd like to work on actually getting a playable game, though. I'll mention that mkxp.conf settings do get applied.
Also, I do get the OpenGL info in the console as well:
Hm. It looks like glew.h correctly defines
APIENTRYto__stdcall, but then unconditionally undefs it again, wtf? Anyway..Yes, it looks like everything is working, but for some reason the result of the OpenGL rendering isn't being swapped to the screen. In mkxp.conf, have you turned
debugModeon? If you turn that on, check the console output again for any messages.Assuming the proper syntax is
debugMode=true, then yes. But, it doesn't do a thing, lol. Other mkxp.conf options work, however; to test that I set the screen to 800x600Can you try to compile and run this test program: https://gist.github.com/Ancurio/001987a2e0b172a036b0 (only depends on sdl2)
hrm... being a pita on me.
gcc -c -I /mingw64/include/SDL2with and without-Dmain=SDL_mainis giving me some nice headaches :/https://gist.github.com/ntzrmtthihu777/e1ec2c096910099b6652
setting main to
int main(int argc, char *argv[])fixed that up though... apparently sdl needs this exact entry point structure, strange no?My bad, I have updated the gist with an improved version that should compile on windows now. Can you try again?
Yes, this is because on windows your main function isn't the actual main function =) SDL2 wraps around this and renames your main function, so the types have to match exactly. C++ only allows one signature, while C let's me do silly things like
void main(void).There we go.
gcc -o non_mainthread_gl.exe non_mainthread_gl.c -I /mingw64/include/SDL2 -lmingw32 -lSDL2main -lSDL2gives me a nice executable that does nothing other than to display a 640 480 red window, is this correct?Does it switch to blue after a while? Also try resizing the window.
Sorry, didn't wait for it to change, lol. and yes, as I type this its shifting through purple into blue, and I can resize it down to just a bit larger than the window manager keys. (Just tested resizing with mkxp as well, I can resize it as much as I need, including the alt+enter fullscreen mode)
Actually I just noticed something. I think its an issue with event polling/passing. Even if the display isn't updating, you'd think that I could at least hear myself move the cursor up/down on the title screen of a default RMXP project, but nope. To further support this idea, that little SDL2 application you had me compile, I could close that either via ctrl+c in my console or by hitting the standard x button in the corner; I couldn't close the mkxp window with the window manager button, which makes me think the window destroy event never got passed along.
Ah, yes, that's what I was going to ask about next. You said that the intro music plays fine though?
If you hit the x button, and wait a while, does a dialog come up saying something like "RGSS script is stuck"?
Yep,
The RGSS Script seems to be stuck and mkxp will now force quit.Good, that means the main thread is doing ok and receiving events just fine. Somehow the rgss thread must have gotten stuck somewhere (possibly before the first
Graphics.updatecall).Do you have gdb at your disposal? We need to find out where the thread is stuck. It's usually enough to pause execution via
Ctrl Cand then typethread apply all bt.Yes, I have gdb :P and I have a large amount of *nix programs available on windows, in a normal filesystem hierarchy standard for arch linux.
urm... lets see. How would I go about getting gdb linked into the mkxp process?
gdb mkxp.exefollowed byrunjust quits when I hit ctrl c.Ok, according to this there's a couple workarounds for CtrlC not working on windows. I don't know which one is simpler, but I think just running mkxp and then attaching gdb with
gdb <pid of mkxp>avoids compiling a helper program. Can you try one of them?Edit: The correct command might actually be
gdb mkxp.exe <pid>hrm... I don't have pidof available... lemme query my package manager... shit.
Got it working, just used a normal cmd.exe window (my gdb is compiled with mingw-w64, so it is native), now what info do you need?
I need a full backtrace of all threads, via
thread apply all bt.Ok, learned how to create a logfile of gdb ouput, so here you go: https://gist.github.com/ntzrmtthihu777/8403b107ffa2f252287d
Cool. Can you give me another dump, this time with
thread apply all bt full? Afterwards, try settingfixedFramerate=-1in the conf and run mkxp normally.holy shit! one, updated the gist with the log from
thread apply all bt full, and then tested withfixedFramerate=-1and it worked! Heh, I get to the title screen and it says its running at about 3k fps, and I can start the game and move around and such, but its all very very fast. But it is 'working' better than it was, at the very least.Yes, setting
fixedFramerate=-1skips execution of where the thread appears to hang =) (It also means that, without vsync turned on, the game will render as fast as possible).I first suspected that FPSLimiter::delayTicks gets a too big number, but looking again I think this segment might be the problem:
Sweet man, hope this gets fixed up soon :D On another note, from your vsync comment, I set that to true in mkxp.conf and it yealded a perfectly playable game running at 60fps, though I think the original RPG Maker XP Game.exe ran at 40fps or so. But I suspect that's another of those 'sweeping it under the rug' things. In any case, if you need testers on windows I'm more than willing to help, as I have shown, lol. I'm actually primarily a linux guy, but hey, I'd like good features on the windows version of a game I make as well :P
hrm. On another note, setting
-Wl,-subsystem,windowsto get rid of the console window popping up still fails when you double-click execute, but works fine from the shell :/What about
-mwindowsinstead?one moment :P ... nope, same result. I'll see about editing the
sdl2.pcfile, I know the packager for the project I'm using does some derpy stuff with them sometimes :/nanosleepmight be returning -1, but not because it was interrupted but due to another error. When mkxp hangs without the fixedFramerate setting, does Task Manager show a high CPU usage for it?guess that's a relative term. Without the fixedFramerate, cpu shows 17. With it, it shows 16/17. But, then again I do have a pretty beefy cpu, AMD FX-6300 hex core @ 3.5ghz
100% / 6 cores = ~17%, so it might indeed be spinning inside the while.
Can you apply this patch and recompile: https://gist.github.com/Ancurio/ae83d3bacc7991e5ca5b
Yep, recompiled and I get debug output of
nanosleep failed. errno: 22 ticks: **********and the super zoomed up speed issue.Here's a patch ontop of the previous one that just disables the nanosleep and uses the fallback, should work decently well: https://gist.github.com/Ancurio/238eeddf89d058947eb1
I'll have to investigate why nanosleep is failing another time. Also,
ticks: **********are those really *s or is it printing actual numbers?Actual numbers, counting up it seems. Also, I'm not on my main machine again so I can't test yet, sorry, but thanks for helping me out all this way.
seems to actually do pretty nicely, considering. Its an even 40-ish fps and not freezing up.
still doesn't launch outside of a msys shell though :/
Recompiled with debugging symbols, thought it may be of more help: https://gist.github.com/ntzrmtthihu777/723ff275bd37e6b88d47
Thanks, but to fix this I'd have to look at the whole stackframe of that function to see which variables have out of place values =P
If you want to give it a try: select the frame with
thread 9, then select the::delayTicksframe withframe 2, and finally dump the variables withinfo locals.Edit: Oh, a
print *thiswould also be helpful!Btw. I have pushed some commits that do more or less what the patches I posted earlier; if you want to pull these new commits, you'll have clear the temp changes with
git checkout srcfirst. You can do that later though.You might get spammed with "nanosleep failed" messages in the console, but mkxp should still continue to run at 40FPS.
gladly, but I'm not sure how to do that
thread 9and so on :Pthose are just commands into gdb, you input them the same as the
thread apply all btcall.hrm. On that note, do I need something in mkxp.conf for this exercise?
Hope I did that right, lol.
Yep, that's exactly what I needed. Can you do a
print *thisas well?This?
Yeah, perfect. I pushed a potential fix for this entire issue, can you pull that and try again?
will do. Seems to have done the trick regarding the whole shebang; I can run fine without any mkxp.conf tricks, though I still can't make it run with a double-click using the
-Wl,-subsystem,windowsor-mwindowsflags... that's a bit annoying, lol.Great to hear that this annoying issue is finally gone =P
I'm not really a MinGW expert, so I can't help you much with the shell window issue though..
yeah, thanks though. Walking through this bit has given me a bit more info to work with as far as how to go about debugging something :P
Think I may try to build via code::blocks... this cmake stuff is killing me, what ever happened to good ol'
./configure && make && make install:/Did you get this to build correctly in the end?
I'm currently trying to cross-compile for windows also, but I'm not having much luck yet (moreso because pkg-config is not finding my libs compiled with mingw)
Are you using the cmake build or the ./configure based build?
I'm using cmake
CMake only uses pkg-config in certain cases to help "guess" where the library is located. You can provide these hints yourself with, e.g. for Boost, BOOST_INCLUDEDIR and BOOST_LIBRARYDIR. If your goal is simply to get this compiled on Windows, setting these variables for each library manually (to point to your MinGW install directory) is probably the easiest way to get it working.
Edit: When you say "cross-compile", do you mean compiling on Linux for Windows, or compiling on Windows for Windows? My comment was for the latter; if it's the former, there's a different way of doing that with cmake.
@khkramer I got it to work, I have a working windows binary in both 32 and 64 bit.
You can pass
PKG_CONFIG_PATHto point at the mingw pkgconfig folder in lib, but that's location depends on what distro you are using.@sorlok
I tried compiling on Linux for Windows first, I'm now trying on windows with cygwin.
The only problem now is getting it to properly detect boost, I'm pointing it towards the right directory but it says It can't read version.hpp.
@ntzrmtthihu777
Would you mind sending me your binaries?
They might work for my project, if they do it would save some time.
@khkramer
You don't want the binaries I currently possess, they're still a bit buggy. I'll recompile once I resetup msys2 (you should really use this instead of cygwin/msys :P) on my winxp virtual machine.
I actually got it to correctly find boost now with -DBoost_USE_STATIC_LIBS=ON
However It's still missing OpenAL and I don't know which version of OpenAL it wants, I've googled and I've find varying implementations of OpenAL, which one should I build?
And I'll try out Msys2, thanks for the suggestion.
the one from http://www.openal.org
OpenAL Soft
Got it.
My next question might be a slightly stupid one but would I be able to build with Ruby 1.9 instead of 2.0?
I haven't worked at all with 1.9 before, so I can't say. At a minimum, it would have to produce a library-fied version (libruby.so), I'm not sure when they added that. If errors you encounter are sufficiently trivial, we can patch mkxp to work with 1.9 as well.
CMake finally made me some makefiles.
When I try building I get this error:
At first I thought I had to install xxd but that didn't fix it.
xxd is a (more or less) standard unix tool; I think any mingw/msys installation should ship with it. Now, as to how cmake finds it on windows.. I don't know (I didn't write the CMake script).
You probably have to explicitly pass the path to your xxd binary to cmake, @ntzrmtthihu777 should know how to do this.
Edit: Nvm my first sentence,
xxdis actually part of vim, so you will need some form of vim package for mingw to get it.I was able to set it with -DXXD_EXE.
Here comes another error C:/CygWinDeluxe/home/Klaas-Hessel/mkxp/binding-mri/binding-mri.cpp: In function ‘void showExc(VALUE)’:
C:/CygWinDeluxe/home/Klaas-Hessel/mkxp/binding-mri/binding-mri.cpp:377:28: error: expected ‘)’ before ‘PRIsVALUE’
VALUE ds = rb_sprintf("%" PRIsVALUE ": %" PRIsVALUE " (%" PRIsVALUE ")",
^
C:/CygWinDeluxe/home/Klaas-Hessel/mkxp/binding-mri/binding-mri.cpp:381:32: error: expected ‘)’ before ‘PRIsVALUE’
rb_str_catf(ds, "\n\tfrom %" PRIsVALUE, rb_ary_entry(bt, i));
^
C:/CygWinDeluxe/home/Klaas-Hessel/mkxp/binding-mri/binding-mri.cpp:393:36: error: expected ‘)’ before ‘PRIsVALUE’
VALUE ms = rb_sprintf("Script '%" PRIsVALUE "' line %" PRIsVALUE
^
C:/CygWinDeluxe/home/Klaas-Hessel/mkxp/binding-mri/binding-mri.cpp: In function ‘void mriBindingExecute()’:
C:/CygWinDeluxe/home/Klaas-Hessel/mkxp/binding-mri/binding-mri.cpp:401:13: error: ‘ruby_setup’ was not declared in this scope
ruby_setup();
^
PRIsVALUEandruby_setupwere added with 2.0.0.Fixing the first one is possible, but I don't like the required changes.
Using
ruby_initinstead ofruby_setupis okay. But itexits the program without any message in case of an error. Also, as of now mkxp would most likely crash, because it doesn't check the return value ofruby_setup.Do you really want to see 1.9.3 supported? There are only a few incompatibilities between 1.9.3 and 2.0.0 (or 2.1).
Ah, thanks for chiming in cremno, I was going to ask you :D
Yeah, this mindset of ruby 1.9 that it's the main executable code and can just
exit()on error is really problematic. I mean, you could probably still run games with it, but getting any decent error messages would be impossible, not to even speak of all the unreleased resources at the end.If you still want to go with 1.9, you might want to try commenting out the entire body of
showExc()and replacingruby_setupwithruby_initas cremno suggested. I'm not sure as to how far I could integrate such changes upstream though..@Ancurio Not really, I'm rather unexperienced with using cmake (I'm an autotools guy myself); its simply the fact that msys2 simply 'just works' regarding stuff packaged for it. Sets up a standard FSH in duplication of archlinux, good buildscripts for things that dont yet exist, its the closest thing I know of you can get in windows to using actual *nix without a virtualmachine/reboot.
The main issue I had getting mkxp compiled against ruby-2.1.2 is a conflict with some posix macros conflicting from winpthreads and ruby; a patch to the former and commenting out of the latter and all was well.
If it's working, could someone put up a windows binary pack? It would be especially useful for packing games with a custom soundfont and fluidsynth for a similar MIDI experience across all systems.
I'd have to recompile, the version I worked together was prior to the MIDI inclusion. Currently I'm working 6-7 12 hour shifts a night and studying for my CompTIA A+ cert exam, so my spare time for this sort of thing is rather scarce.
@ntzrmtthihu777
In case you do find the time, it shouldn't be very difficult as the dependencies between back then and now didn't change at all I believe.
@Ancurio sup man. Yeah, I suppose so, though I do recall having severe issue getting fluidsynth to compile on windows (needs something from the direct* sdk, and I really don't wan't to download the entire thing [smartphone internet, 500mb 4g/month] for one header file...). Also, I never solved the issue of compiling it for windows without it opening a cmd.exe window when ran (did all the -Wl,-subsystem,windows crap in various forms. It 'runs' without opening the window, but immediatly goes into not responding mode). I assume it has something to do with the fuckery that SDL does to int main.
@ntzrmtthihu777 I see, no need to sweat it. I think in theory one could just download a random prebuilt fluidsynth.dll from the interwebs and it should work ok (you don't need the headers).
And yeah, that console window stuff was really unfortunate. I actually tried a cross compile with mingw today just for funzies, got all dependencies compiled pretty effortlessly except for boost which is a really strange beast with the weirdest build system on earth.
What distro are you on? Boost was not an issue on my end at all (BTW, I know you use it for Program Options, have you considered instead GNU GetOPT? What else is boost doing for you?)
I'm on Fedora (19). I think I'm just too stupid to read the boost documentation, no matter how I setup the config file it won't even start the build, maybe I'll get back to it some later time.
I actually also use the unordered hash and set containers from boost, but they're header only. Program options is kinda nice because it parses both config files (mkxp.conf) and command line arguments, and transparently merges them. There's a few games using mkxp on Steam right now, and it's really neat when players don't have to navigate folders and edit mkxp.conf (which might be overwritten by Steam) because I can just tell them to set a few launch parameters for the executable in the Steam settings instead.
I took a quick look at getopt, but it seems to do cmd line args parsing only?
Edit: Just noticed that my distro has prebuilt mingw packages for boost, awesome! Gonna give them a try.
AFAIK yes, getopt is command line only. I'm an Arch user myself, fully updated (even using the linux-git kernel 3.18-rc4) as of a few days ago.
So, using the nice precompiled libs from the Fedora repos, I managed to link a mkxp.exe in the end. There's one problem (potential bug) in SDL2 that I have worked around for the time being, and for some reason jpeg decoding fails every time, but other than that, most things seems to work nicely (as far as I can tell). Testing is just really annoying because my WinXP VM doesn't do OpenGL 2.0, so I have to reboot into my Vista partition to actually run anything graphical.
Once I get the jpeg problem sorted out, I'll upload the binary if you want to play with it @The-J-Person, although I give no guarantees as to bugs, and probably won't be able to debug any further (and more intricate) problems ;D
Thanks a lot :D I actually use Debian and Mint myself, and normally use WINE to see if windows executables work rather than a VM - close enough methinks.
Really though, thank you for your hard work and responsiveness - it is much appreciated!
@The-J-Person @Ancurio yeah, generally opengl based games tend to work very well under wine, my own mkxp.exe ran under wine fine, but upon using it in windows I still get the cmd.exe box... irksome.
@ntzrmtthihu777 I think that's because wine just pipes through the raw OpenGL driver it finds on LInux (eg. GL_VENDOR and such still show Mesa even under wine). Btw. I can link my exe with -mwindows and it hides the cmd window just fine.. I'm using mingw-w64 btw., not sure if that's related.
Got the SDL2 and libjpeg issues sorted out properly, but now OpenAL is giving me huge troubles with its mmdevapi backend, hopefully I can make it always default to DirectSound because I have no clue what's going on haha. I really hope it's not related to me linking libOpenAL statically though.
@Ancurio what steps did you take to compile and link mkxp for windows?
@ntzrmtthihu777 Hmm, I'm not exactly sure what you mean? I compiled all .cpp's into .o's, then linked them with the libraries into an .exe. Here's my ad-hoc build script, maybe that will make more sense.
@The-J-Person Unfortunately I hit a wall that I couldn't further debug. To be more specific, I always compile two executables, one that opens a console window for output and one that doesn't, but for some reason the non-console-window one keeps segfaulting over and over without visible cause (console version works completely fine) on my Windows Vista system. Using wine or running it in my Windows XP WM, it works perfectly fine however, so I have no idea what's going wrong.
Here is the last build I made, please try it for yourself: https://www.dropbox.com/s/ugpkcz83zxebamh/mkxp-mingw32.zip?dl=0
@Ancurio ah, static linked it all? sounds like a plan, considering the tendancy of windows to get stuck in dll hell.
@ntzrmtthihu777 It's really just because Windows has no clean rpath equivalent. I usually prefer to dynamically link all my dependencies, and on Linux/OSX you can neatly stow them away into a lib subfolder and still have everything work, whereas windows leaves you no choice but to splatter a giant blob of dll's into the folder which just makes it very confusing for users to "find the executable to double click", that's why I prefer static linking there.
With a bit of initial work it would be possible to do that on windows, too. This method is similar to how isolated applications work.
Basically you need two manifest files, one will be embedded in the
mkxp.exeor can stay in the same directory asmkxp.exe.manifest. This file references a local assembly, let's call itmkxplibs.Then put all dlls (ruby, sdl,...) in a subfolder named
mkxplibsand the second manifest asmkxplibs.manifestthere, too. This file simply lists all needed dll names.When windows loads the executable, it will first reference the assemblies and therefore will have all listed dlls available.
Reference, with example: http://sourceforge.net/p/mingw-w64/mailman/message/30019971/
How to add the manifest to the exe: http://cournape.wordpress.com/2008/09/02/how-to-embed-a-manifest-into-a-dll-with-mingw-tools-only/
@carstene1ns Very interesting. I've heard about this mechanism but always thought it was only available to C# applications. Thanks for sharing!
I'm actually getting the same problem--console version works, -mwindows causes instant crash. I've tried both using the binaries posted here and compiling it myself with shared libraries, same exact symptoms. If it's any help, using shared libraries revealed that the crash originates in the Ruby library.
@mathewv Yes, I tried making a debug build and using gdb on windows first. The crash happened in io.c when manipulating stdio fds, specifically fstat was returning a EBADF errno and then ruby crashed in
rb_bug()although that call meant it would have terminated anyway. So I tried commenting out that check and recompiling ruby. The crash there stopped, but it continued to crash somewhere else,st_foreachor something really basic like that, and that's where I threw the towel in because I assumed something must be randomly overwriting and corrupting memory. And I really don't have the setup to run valgrind under windows.Very interesting, if disheartening. @Ancurio I had been rubber-duckying with a co-worker and thought I had an idea, but I suppose in light of the above revelation that the bug was originating in ruby seems to kill that idea... BUT. Just in case, lemme express it. I'm currently cloning mkxp on my windows partition again, and gonna try something. The idea is mayhaps SDL + -mwindows does not play well with your printing of the various opengl infos onto a nonexistant console. Hope I'm not sounding too foolish suggesting that.
@Ancurio valgrind is a code checker of sorts, yes? Well I may have the setup to be able to do it, if you can guide me.
@ntzrmtthihu777 It's not confirmed that the bug originates in ruby per se, just the symptoms always show up there, but who knows. When the first stdio issue I explained above came up, I did consider the possibility of the mwindows switch somehow causing the default file streams to be closed, but a. that doesn't make any sense and b. even then writing to a closed stream is an error condition and doesn't cause memory corruption, so that's out for me. Googling this also didn't reveal anything like a common problem.
When I said valgrind (which is a collection of tools) I specifically meant
valgrind --tool=memcheck, which will print any use of unallocated memory. It's not like this would immediately reveal the error, but it's often the only way to even get close to it. Actually, reading the homepage, valgrind doesn't support windows at all haha, so nevermind that. There might be tools that do a similar job though.Debugging problems like these can be arduous and take quite a bit of time, so I'd suggest to only do this if you have plenty of time and patience. I appreciate your will to help though.
Patience I have in abundance, time not so much. So in what directory/etc should one use valgrind? oh, no windows support.... would loading it in a debugger meant for windows help? and should this be on a static-compiled or shared version of mkxp?
I already ran it in gdb which showed me where the segfaults happened, but a debugger will not help you track down memory corruptions.
hrm... what a pain this has come to be. I'd rather avoid it, but I've been considering doing a rewrite with sfml (even if it does not feature all that mkxp on linux provides.)
Eh, it's not really much of a pain :D I went into the mingw compile thinking "how far can I get without any effort" and this is the line (for now). If someone is serious about running mkxp on Windows, they will have to do the hard work.
Interesting, but that's not really related to this issue is it?
Fun fact: I originally started out writing mkxp using SFML in 2012, and already had a lot of stuff done, but I got so annoyed with SFML's limitations and inefficient code that I completely rewrote the graphics part in SDL+OpenGL (best decision ever). I'd be interested to hear what advantages you're hoping to get out of SFML =)
Much to my surprise, I figured out a solution. It's not exactly pretty, but it stops the crashes on my Windows 7 VM.
Basically I redirected the standard input/output streams like this:
I put the code before the
ruby_setup()call inmriBindingExecute(). Then in mkxp.pro,I don't necessarily like it, but I like it a lot better than the program not working at all.
It's crashing, because (at least on Windows)
ruby_sysinit()needs to be called. Which does something similar to @mathewv's code. Also, see https://github.com/Ancurio/mkxp/issues/45#issuecomment-48457900. But I think my code is slightly wrong (argvneeds to be initialized). It's a weird function anyway.Thanks for chiming in @cremno. I feel a bit stupid actually, because my first idea when I saw the fstat problem too was "well, let's just reopen all the streams to null!", and already had the code written, but then deleted it instead thinking it wouldn't work if ruby used the fd values (0, 1, 2) directly.. oh well. And closing stdout before ruby_init on Linux also leads to a crash in the exact same location, so that's definitely the cause.
Do you know if it's safe to just init the argc and argv arguments to
0or do I actually have to pass the real stuff along? (At least on windows, it looks like it just ignores them and does its own parsing viaGetCommandLine())Is passing the arguments from
main()not an option? Theargcargument doesn't seem to be used at the moment, but theargvone is used when amalloc()call fails (name isvec):4cb3f72fdd/win32/win32.c (L1775-L1776)Eh, there's like two million places that will crash / throw uncaught exceptions if malloc fails, so that's fine. I'm just trying to reduce the needed changes to as little as possible (passing the real args would cause line changes in a few unrelated places). Although I think argv[0] is available in shState because PhysFS needs it, so we could provide that if absolutely necessary (don't know why ruby code would want to see anything else).
Well, then the code I've posted in the other issue should be fine.
Yup, just tested this, works fine under my Vista system too. Please try these new binaries @The-J-Person : https://www.dropbox.com/s/ugpkcz83zxebamh/mkxp-mingw32.zip?dl=0
I've also added in the random fluidsynth.dll I found via google.
mainly that its pretty straight-forward with the console/no console bit, no wierd entry point manipulation. Plus, lot less libs on the windows side (though I suppose static linking them as you were would actually remove that concern.
More straightforward than a single linker flag? I don't know what's going wrong for you, but as you saw you're the only one having this problem
I don't know what this refers to.
SFML bundles like half of its dependencies as static libraries, so you're already unknowingly statically compiling away half of them without realizing it.
entry point manipulation; using sdl you have to use
int main(int argc, char** argv)which gets overwritten with SDLmain.Console/no-console; actually we've discovered the issue was a ruby/windows thing, correct? so I guess this is moot point. My thought was that SDLmain for windows did that funky WinMain stuff which mayhaps caused issue with the console output, but seems to not be the issue, correct? I'm honestly not terribly familiar with windows-specific coding, so when something that works on linux doesn't on windows I tend to just go
damn you windows!and procede to gripe the rest of the day.Hah, is that so. Once again, more of a linux guy, lol.
Oh, I see. I think that's because SDL wants to be called first even for console apps so it can verify correct initialization (grep for
SDL_SetMainReadyfor details..), otherwise it wouldn't have to. Oh, and SFML does "that funky WinMain stuff" too. From my understanding, I think you have to have aWinMainentrypoint which serves the purpose ofmainon Windows if you want a real GUI app without console opening itself.I think I misunderstood what you meant then. I thought you were referring to your problems in hiding the console window. The crashing on the non-console build was indeed a ruby/win32 interaction (although I'm still not sure who exactly closed the stdio streams O.o)
I know exactly what you mean, I had the same reaction when I gave up on the crashes earlier haha. I have close to zero experience with windows programming myself.
Nah. I've done a standard
int mainentry point with sfml on windows before,-mwindows'd it (or was it-Wl,-subsystem,windowsor whatever it was?) and no problem, mayhaps the above flag does the WinMain bit.Either way, I think we can say we've got this thing mostly filed away :) tried both of your new binaries on my windows partition and via wine, all is well.
Pushed the crash fix:
33c571e691Excellent, now we just need a good buildscript for the windows side (either using mingw-w64 on linux or msys/mingw on windows) and we're golden.
@Ancurio hey, found your issue https://www.ruby-forum.com/topic/6182196 while googling about for info on how one would overcome some issues in cross-compiling ruby for windows on linux (the last supported version of
mingw-w64-rubyon arch linux was 2.0.0_p247, and that simply was not good enough for me). I've since fixed up a PKGBUILD for mingw-w64-ruby 2.1.5, soon to be posted to the AUR for others to use. I don't know about how Fedora does its packaging, but all a PKGBUILD is is a collection of functions that automates the./configure && make && make installor the equivalent, so you could manually use it as a recipe of sorts if you need :)So how did you tell configure to use your system installed ruby for generating the build files?
I didn't, I don't think. seems it all went very smoothly with no issue. All the fiddling I had to do was to deal with a bit of preprocessor macros
Here's the goods, see what you can see :) https://gist.github.com/ntzrmtthihu777/52423fb0691de108b093
one problem I'm having using your win32_build.sh is I need a static gmp lib, but I'm having issue generating one...
Well, you're not really supposed to use that script, I only posted it to clarify how I'm building, but otherwise it's just a 10 minute hack job to get this done on my computer :D Of course if it works for you, thats cool.
I just downloaded the latest source code of GMP and compiled a static library without any problems. What is going wrong for you?
probably something funky in the PKGBUILD for mingw-w64-gmp from the AUR, it just doesn't seem to create any static libs. Could you post a directory tree of what is produced on your end, please?
The configure script mentioned that it cannot build both static and dynamic libraries at the same time, because the
gmp.hfor both is different.yeah, I noticed that when I tried both at once XD
lemme check that out in a bit, I think I'll do some bull-hockery to make gmp.h an #ifdef switch between gmp_static.h and gmp_shared.h
Hi all,
I've got most of the cross-compile working, but I'm having some trouble building both zlib and ruby. Does anyone have a set of commands handy that they used for these libraries?
I always use mxe (M Cross Environment) to cross-compile for Windows. It has a built-in script for compiling zlib, and it should make building Ruby easier (I think you might need to pass "--enable-shared --disable-static" or "--enable-static --disable-shared" to configure to get it to build a library, but I forget)
@mathewv: Thanks for your advice, but after doing a clean build through MXE, I am running into the exact same kinds of errors when compiling ruby (zlib builds, but ruby doesn't find its symbols, and there are lots of scary-looking warnings.)
Does anyone who actually cross-compiled MKXP for Windows have time to do a quick overview of how they built ruby (with the zlib patches, of course)?
@sorlok I haven't been able to convince the configure script to add
-lzwith mingw yet either. For now maybe just built Ruby without the zlib patch? Or does your game definitely require the module?@Ancurio Ah, ok, at least it's not just me. I think in the long run I'll just add the zlib files manually to the ruby source directory. For now, I'll try disabling zlib and seeing if I can work around the breakage in my game.
Last year, my colleague and friend ntzrmtthihu777 stated both:
But, then again I do have a pretty beefy cpu, AMD FX-6300 hex core @ 3.5ghz.
and
On another note, from your vsync comment, I set that to true in mkxp.conf and it yealded a perfectly playable game running at 60fps, though I think the original RPG Maker XP Game.exe ran at 40fps or so. But I suspect that's another of those 'sweeping it under the rug' things. In any case.
Running on Windows 7, I too had it running on 60FPS.However, upgrading my PC to a beefy i5 with Windows 8, the FPS was cut down to nearly half. It's now running at 32FPS at the most. Any clues? Oh, its the same .exe we used this past year when you last talked to ntz about the framerate.
@DerVVulfman The default framerate is defined by the RGSS version, but RPG Maker XP has no problem running at 60FPS if it's instructed to do so.
Your PC changed. The executable stayed the same. Update your drivers.
Hey, @Ancurio its me again. Was hoping to discuss some things with you regarding this again, do you have somewhere we can talk live-ish? Basically looking to patch mkxp up a bit for some additional features a friend needs and build a windows version. I see you're hosting precompiled builds, was hoping you'd share the build setup.
I would be interested in that as well, I'm currently using Msys2 and it's working just fine however I have to put the dll of every single dependency in the folder as well (which are a lot) and haven't succeeded at a static build yet.
@khkramer I'm currently working on setting up a gentoo/crossdev toolchain
for this purpose. Certain
things are fighting me, but I think I've almost got it to a point of being
workable for general usage.
Most of the problems are from packages that honestly are probably not
needed within a given sysroot
for cross-compilation (no need to install, say, coreutils to
/usr/i686-w64-mingw32for example).On Tue, Feb 28, 2017 at 5:48 AM, khkramer notifications@github.com wrote:
--
If you're looking towards the Son, all your shadows are behind you.
@ntzrmtthihu777 I'll also share my build steps, which use a MXE cross-compilation setup on Linux:
https://docs.google.com/document/d/1G2SgTy2Ff76B6IO-w6mBLvO1NyThqmvM8TkTdl71JAQ/edit?usp=sharing
Warning: notes are very scatter-brained. I'm happy to answer questions as needed.
@sorlok does this result in a single static binary on windows? Or at least
close to it?
@ntzrmtthihu777 I use Telegram for instant messaging; we can talk over IRC too but then we'd have to arrange a date and time.
My build setup is very bare-bones; a lot of libraries I don't even have to compile myself because Fedora's repos are so rich in mingw64 prebuilts. This is the script I previously used for building the statically linked binaries: http://pastebin.com/6Vc2KVbm
"sources" is just a bash file filling some variables with the filenames of all source files.
Some notes I kept on compiling dependencies with mingw: http://pastebin.com/aMB1rinV
I also had a couple auxiliary files, like "build_env":
Note that it's been a loooong time since I last worked with this so my memory is a bit hazy, and I'm really short on time. If you need more infos / details, we can chat in a few weeks (assuming I get done with my projects til then).
@ntzrmtthihu777: It results in everything static except OpenAL (and libgcc_s_sjlj on 32-bit for some reason). (Ancurio has managed to link OpenAL statically, but I never was able to do it.)
@sorlok I may have misinterpreted your instructions on patching openal, as it would not build for me (x86_64, trying i686 now).
@ntzrmtthihu777 Yeah, I was never able to get OpenAL to build statically. It should build just fine as a .dll though.
@sorlok ah, I get it.
${CHOST}.sharedYou may want to consider making some patches that can be easily applied to the various tarballs and such :)EDIT: just failed x86_64-w64-mingw32.shared openal. Do you think you could share your CMakeLists.txt for openal? I'm pretty sure I'm interpreting what you wrote wrong.
@sorlok I'll try adapting your build process soon.
I don't mind having a single DLL for OpenAL but at the moment I have like 20 Dll's, since SDL image also needs dll's for every single image format atm.
@sorlok seriously, if you could share your openal cmakelists.txt file I'd really appreciate it.
Actually I got it to compile... now I'm stuck on SDL2. If you could have a chat with me at some point I'd be very grateful.
@Ancurio so, using your ad-hoc build script, I managed to get most everything to work. Problem is, src/sprite.cpp doesn't know what
M_PIis... I could maybe define it or sommat, but I'm wondering if there is a cleaner way to do it.still stuck. passing -DM_PI=3.14..... lets build progress to the point of linking, but fails due to undefined references to modplug stuffs... I've even recompiled SDL_sound with --disable-modplug and --enable-modplug=no, really stuck here :/
and I'm a dummy. good now, compiles, but still requires a few dlls.
@ntzrmtthihu777 My apologies; I dropped out for a day and missed these messages. I'll copy my CMakeLists.txt file when I get back from work today.
@sorlok while I appreciate the offer, I just got a static compile to work using @Ancurio 's ad-hoc script.
Took a bit of debugging, but it all worked out in the end.
Ended up just setting up an archlinux chroot and using the mingw-w64 repo on sourceforge (some dude called ant32) and a few self-compiled PKGBUILDS.
@ntzrmtthihu777 That's great! Are you still having the issue with SDL2?
Nope, got it all nice now. 64-bit mkxp that actually runs (in a vm, kinda fucky display [due to fake gpu I bet] but it runs). Will be setting up my laptop as a test bed for this stuff. My friend's gonna be so stoked.
@ntzrmtthihu777 That's great, glad to hear it!
@Ancurio I have some fresh builds for windows. 32 and 64 bit, iff'n you wanna stick them up somewhere.
@ntzrmtthihu777, I want to try something on Windows, can you please kindly make a new Windows build from the mkxp-abs repo?
https://github.com/Ancurio/mkxp-abs
Sure, you just need a binary, right?
On Fri, May 12, 2017 at 3:02 PM, Ali Rastegar notifications@github.com
wrote:
--
If you're looking towards the Son, all your shadows are behind you.
Thanks!
If the binary has the libs in a static form then yes, but if the libs are dynamically loaded I would be grateful if you include them too.
Sure, no problem. I can't make a dynamic loaded mkxp with my current setup
(openal only builds shared or static, not both)
so it will be one static binary. Is there any major differences between it
and the mainline mkxp repo?
On Fri, May 12, 2017 at 3:10 PM, Ali Rastegar notifications@github.com
wrote:
--
If you're looking towards the Son, all your shadows are behind you.
Static will be great actually.
Hmm, steamshim support should be one of the most noticeable differences.
Ah, cool. I'm currently at work away from my box, so it will be a bit
before I can do so. Do I need anything extra for steamshim
compilation, or is all that's needed in your repo?
On Fri, May 12, 2017 at 3:47 PM, Ali Rastegar notifications@github.com
wrote:
--
If you're looking towards the Son, all your shadows are behind you.
Thanks, I really appreciate it.
The repo only includes the steamshim support for mkxp and in my experience with building on OS X it didn't need any other lib (it does not compile the steamshim_parent itself, I have to cross compile that one using the Steam SDK later).
@openmac 32/64 bit?
Also, it may be useful for you to merge in the changes from the main repo for the windows icon stuff.
Something seems to be wrong with mkxp-abs, in that the same makefile I use for the mainline mkxp repo
can't find SDL_main when linking SDLmain.a into it.
Either of 32 or 64 bit is fine, I want to test it a while before figuring out how to make my own builds.
As for SDL_main, that's strange, if it is giving you too much trouble I still can benefit from testing with a build from a mainline mkxp repo too if that one builds as expected for you.
@openmac no, seems like my arch chroot is busted for mingw-w64. pulling head on both repos results in the same error. I apologize for the delay, but I'm gonna go ahead and setup gentoo's crossdev and use that, which could take a while, but its becoming too much of a pain to maintain two distros.
Sorry for all the trouble, there is absolutely no rush.
@Ancurio yo, been working on tweaking cmake to be able to build for windows (not specifically on windows, mind you. I don't have the willpower to sort out that mess) and I was wondering if you had any particular preference as to how its done.
Considering its relatively easier to produce a fat, static linked mkxp.exe for windows rather than managing that whole side-by-side assemblies thing mentioned above, I'd basically have to replace
target_link_libraries ${LIBRARY}_LIBRARIESwith${LIBRARY}_STATIC_LIBRARIESin anif(WIN32)block of some sort.Barring that approach, I could look into making something like
PostBuildMacBundlewhich would copy needed libraries into a subdir along with the manifest needed to use them (I've mostly figured out using windres from binutils with cmake, actually fairly simple).Suggestions? Ideas?
@ntzrmtthihu777 I don't have a preference; if the OSX path in CMakeLists.txt copies over the libs to form the bundle, you can do the same for windows. A fat statically linked mkxp.exe is fine too, but in that case everything should be linked (except maybe fluidsynth).
@Ancurio actually I had a question regarding the whole fluidsynth/fluidlite
thing; think it would be too much of a problem to add in some #ifdef's to
easily allow someone to build against either instead.
On Tue, May 23, 2017 at 1:01 PM, Jonas Kulla notifications@github.com
wrote:
--
If you're looking towards the Son, all your shadows are behind you.
@ntzrmtthihu777 They have identical APIs.
@Ancurio yes, identical apis, but not identical header/pkgconfig files.
On Wed, May 24, 2017 at 12:30 AM, Jonas Kulla notifications@github.com
wrote:
--
If you're looking towards the Son, all your shadows are behind you.
I've managed to get cmake to build it, but there are some adjustments that I think are a bit unfortunate and very hackish that are needed. for whatever reason I need to
#undef mainto link, and add flac, ogg, and speex to the libraries (which I think is due to the lack of being able to make a shared SDL_sound.dll build on mingw-w64). Obviously this should not be part of the upstream repo, as it is a massive hack.@ntzrmtthihu777 Regarding fluidsynth/fluidlite; this only concerns the build setting where you hard-link (instead of dlsym'ing) against the library, which is off by default. I feel ambivalent about this, because the normal fluidsynth dev packages you can download from every repository, whereas I'm not so sure how well fluidlite has been taken up by packagers. But then again, lots of other things don't make sense in mkxp either.
Feel free to send a PR that switches the respective pkg-config names to target fluidlite.
Agreeable in general, for sure, but if one is going to use fluidlite instead of fluidsynth, it makes more sense to do a hard-link, otherwise you need vorbis/others in the same dir as fluidlite.dll (and the side-dir, if you use that side-by-side assemblies technique discussed way back in the thread). On another note, with regards to the whole winmain issue with a cmake build, how would you feel about an ifdef'd undef main for windows? Yeah, its an ugly hack, but, you do get all the niceness of using cmake to build instead of an ugly script/makefile that got threw together, and its 'good enough' for now...
Also, I'm working on packaging fluidlite for gentoo at least, and possibly arch.
@Ancurio I've figured a bit more out. I think my SDL2 install is busted on the windows side, however, so more work needs to be done there. But, I've discovered a bit of a hack that at least lets mkxp compile and link via cmake cross-compilation.
Basically, running
make VERBOSE=1will reveal the final link command to bewith linklibs.rsp containing the following
Copying that file, editing the first bit to say
-L/usr/x86_64-w64-mingw32/usr/liband rerunning the g++ command with@newlinklib.rspinstead does link and compile it properly.I've tried various methods of shoe-horning that directory into the link command in a more integrated fashion, such as the EXTERNAL_LIB_DIR variable you set up, to no avail. If we were able to get that to work somehow, I feel confident that this would work properly.
@Ancurio so, I've worked out a build using meson that works for both linux and mingw-w64 cross-compile (have not tested running it yet, setting up a windows vm) but it does compile and link without error. I know you're a bit averse to the idea of adding new build systems, but this works very well, is very fast at building, and is relatively small (176 lines for meson.build, 5 lines for meson_options.txt, 15 lines each for x86_64 and i686 mingw-w64 cross-files) compared to the cmake build (443 lines).
What do you think? I've not yet a way to test builds on OSX, but meson apparently does handle/have instructions for it.
@Ancurio good news! I figured out the error I was having (I don't think I discussed it with you, basically my toolchain was build with pie enabled and apparently that's a a bit fucky on mingw-w64, so even the simplest hello.exe wouldn't work unless it was compiled with -static), got a working meson.build setup for mkxp (tested in linux and mingw-w64, both amd64 build) which for windows uses the above listed side-by-side assemblies technique, putting the needed dlls into a runtime subdir of the game folder.
@ntzrmtthihu777 Can you open a new issue for this? This one is bursting at the seams already.