Compiling .vm files gives bizarre results on mod; .dll works fine

1

Hello again! I’m almost ready to release the next version of my mod - thanks to the help of Chomenor and ToKu I was able to resolve a problem with entity flag transmission. On my compiled .dlls the code runs perfectly fine with no issues. However, I went to do the .qvm compiling as I usually do, and for the previous three versions of my mod it has worked without problem. This time, however, the output .vm files are exhibiting very strange anomalies when I run them. This isn’t a comprehensive list, but among those problems:

  • A green-colored aura I use for a power-up shader never shows up in the .vm game, but works fine with the .dll’s. I checked my files and can confirm that it is indeed not due to missing shaders.

  • Homing rockets travel normally when other players aren’t present with the .dll game. With the .vm compiled version, they disappear out of existence if they can’t find someone.

  • A sound file that plays just fine on the menu with the .dll’s does not play with the .vm’s. Again, the sound file is present and in the right directory, the game simply just doesn’t play it.

  • Storage Capsule client-side messages are played out-of-order in the .vm, but in the correct order in the .dll.

As you can see from the examples, the problem affects all three components of the source code (server, client, and the UI)

I’m not looking to specifically fix the above issues as they are symptoms of a much bigger problem, which is that the .vm compilers / makefile, for whatever reason, are seemingly modifying the behavior of the code. Whatever the .vm’s / makefile put out should logically work exactly the same as the .dll’s, and up to this point they have without fail. I have not made any changes to the makefile / lcc since the last release.

I cloned the last GitHub commit I performed right before version 0.3’s release and compiled the .vm’s on it, and it exhibits the same issue, so I can confirm that it isn’t because of code I put in since then. I also made sure to exclude my directories from antivirus tracking so they don’t get flagged as false positives. I had all three “vm_” cvars set to 2, and I changed them to 1 temporarily to see if that would fix it, but it doesn’t change anything. I have tried making both 32-bit and 64-bit .vm’s; both have the same problem.

Has anyone had experience with lcc or makefiles producing unexpectedly different results from the .dll’s? If so, why does it do this, and what did you do to resolve it? Thanks for any help in advance, and my apologies that this is my third thread on here - I know I’m a magnet for problems! Using Cygwin on Windows 10.

2

Well, I did a quick check, I’m a bit busy IRL, so I only have some small notes for now.

  1. I tried to compile your code, there are two errors you should fix, unfortunately NOT related to your problem. In line bg_misc-c line 881 there is a ‘m’ which prevents you from compiling: https://github.com/EmeraldTiger/Uber-Arena/blob/master/ioq3-master/code/game/bg_misc.c#L881
    If this is fixed, the compiling process complains that PW_SCOUT is not defined. This is because you broke Team Arena support! By default (unmodified MAKEFILE) ioq3 compiles TA too (for good reason). Solution: Either repair your code so TA will compile again (you shouldn’t break things), or if you really don’t care about TA, open your Makefile and set building TA to 0 (like BUILD_MISSIONPACK=0) in this case Team Arena will be excluded from compiling. https://github.com/EmeraldTiger/Uber-Arena/blob/master/ioq3-master/Makefile#L33
    I assume you will prefer this method, but please keep in mind there are still players/coders that like TA!
  2. Finally your code compiles fine for me (Win 8.1 using Cygwin as well). There are no warnings/errors anymore (with the exception of the usual ones not ‘introduced’ by you) after fixing the things from above. Probably this is the problem, silly question, are you sure you really compiled your code successfully? Eventually you load outdated/wrong qvm files.
  3. Testing Uber Arena seems okay to me, with one exception, all the media your code refers to is missing. I can’t find the shaders models etc. Probably you already know it, but anyways, keep in mind Github also ‘accepts’ media like shaders, textures, models etc. Tbh, I don’t know if there is a limit or not, but for a mod it should be okay to upload all you need. So you can/should open another directory called similar like uber-data, uber-assets, uber-media or anything like that. It’s important to keep it seperate from code repository imo, because if this repository grows (which is very likely) you don’t have to dl so many MB (ALLWAYS respect licence, best to create your own stuff). This leads to the next point…
  4. Your repository becomes quite large, this is because of the added .vs folder inside msvc12. Do you really need this files upload onto Github? I suggest to remove it. Not everyone knows how to not dl this folder. It takes time to dl 378 MB.
  5. Eventually a small readme would help a bit, Github also supports a Wiki, though you can also use the README added to your Github repro (extending the one from ioq3 for example).

So, all in all I have to admit currently I can’t say whats wrong with your code (referring to your description it looks like engine/game mismatching somehow), well, saying this is not really a help, I know.
Are you sure all the latest qvms are loaded? It does for me at least, but you know sometimes things went wrong just by not loading the correct stuff. Do you pack the qvms in a pak file etc?
Maybe I can find more infos in the next weeks…

Anyways I think it’s cool how enthusiastic you keep on working. Many start coding and lost interest soon! Keep up the good work!
BTW, g_bouncy is lot of fun!
Ah, and you know, everything that I have said, I said in good intentions and represent only my own opinion.

3

I may be misunderstanding this, so forgive me if I am looking in the wrong place - I checked the creation date of all my .qvm files (cgame.qvm, qagame.qvm, and ui.qvm) to make sure that they’re the ones I just generated. I typically delete my old vms first before creating new ones, or I add some random comment in the code to cause the makefile to recognize it needs recompiling (which is where the random ‘m’ comes from, since I disable TA building), otherwise the makefile will skip doing anything. I also package them in a .pk3 as well, though I tested with external .vm’s as well (has the same issue).

My generated .qvm files are stored in ioq3-master/build/release-mingw32-x86-64/baseq3/vm.

P.S. Glad you like g_bouncy! :slight_smile: I actually wrote that as a means of testing maps to see how well they could work with trampolines without having to take the time to decompile them first (or just compile period, in the case of open-source maps). Could make an interesting alternate physics option though for fun games, if given the ability to enable / disable it at will.

4

Just looked at another thread which mentioned something about a make clean (Compiling, but code changes not applying) command, which should rebuild everything and remove unwanted clutter / artifacts the next time make is ran. I have ran this and did another make as I usually do, and unfortunately I am still getting the same issue. I also tried copying the mod folder into another Quake 3 installation with ioquake3 rather than vanilla Q3 and it exhibits the same issues. Just random things not working like they should on the .qvm’s, yet the .dll’s work fine and as I intended.

5

Well, afaik the only difference in creating dll vs. vm is that the syscalls.asm files are used instead of the syscalls.c files. Both files match (trap calls) so I don’t know why they produce different results. What makes me wonder is what happens when you use an unmodified makefile (from ioquake3)? I’m no makefile expert, but the modified makefile is the main file that affects vm building and was changed. BTW, for what is -vq3 used in your case?

6

Just re-downloaded the original ioquake3 repository from GitHub and tested out its makefile. Still exhibits the same issues, unfortunately. I use the -vq3 flag in order to compile .qvm’s that will be compatible with both ioquake3 and vanilla Quake 3. I did remove the -vq3 settings on one run to see if that might do anything, but it creates the same results.

7

And you are sure that before ‘the previous three versions’ qvm worked without problem? So the commit/version that introduced the issue must be he last one, right? Or what do you mean by version? I ask because the only thing I can suggest is to compile the code backwards to find the commit that introduced the issue for the first time.

8

Did you make sure the game is loading the vm/dll files from the right place? It’s possible if you had one version of the mod in homepath and another in basepath, for example, you could end up with the game loading old versions of the mod by accident. Check the console log for the location of both the qvms and dlls to make sure the game is loading the ones you are expecting.

9

I went ahead and checked the console log for the .qvm’s being loaded up. Unfortunately, they do indeed point to the right location in the mod’s folder ("C:\Program Files (x86)\ioquake3\uberarena, which is where the pak0.pk3 with the vm’s is), instead of some other vm folder elsewhere on the computer. I also checked fs_basepath (C:\Program Files (x86)\ioquake3) and fs_homepath (C:\Users<username>\AppData\Roaming\Quake3); both also point to the correct location.

The conservation aura problem I mentioned in my first post was actually me just accidentally overwriting some shader code; I assumed it was the .vm’s as no missing texture was drawn in place; I managed to fix it by re-adding the missing shaders, so that one is my bad.

The sound file not playing in the menu I was able to fix by moving it down to the end of the UI_MainMenu function, instead of having it near the top. I have no clue why that fixes it though. I suppose it’s because it was being called before caching, though that makes one wonder why the .dll can play it just fine while the .vm couldn’t.

I did find out that there were several warnings being generated in the .vm compiler about undefined behavior when using == for strings, which I replaced as I figured it was causing the problem. There are no more warnings, but my homing rockets are still acting strange when I run the game using the .vm’s. In fact I just embarrassingly found out that the problem has existed in the previous versions too, and I missed it completely as I only ever tested it with the .dll’s, as I didn’t have any reason to expect it to behave differently. Oof.

Here is my homing rocket code below. Again, it works fine in the .dll version, with homing rockets going in a normal straight path if no players are around. In the .vm’s the rockets just disappear out of existence or go towards a random direction if no players are detected. I did a test where I added an else to the if statement for if (candidate) which made the game print a console message if there was no player in the rocket’s sight. The discovery was that the else code almost never executed in the .vm while it worked as expected in the .dll. I have no clue why the .vm would mangle up something like this, and Google hasn’t helped me much, sadly. Are there certain things the .vm compiler is more strict on or treats differently?

// Some code courtesy of Chris Hilton of Code3Arena
	// https://www.quakewiki.net/archives/code3arena/tutorials/tutorial35.shtml
	if (isUber(ent->parent, COUNTER_ROCKET) && ent->s.weapon == WP_ROCKET_LAUNCHER && level.numPlayingClients > 1) {
		VectorCopy(ent->s.pos.trDelta, forward);
		VectorNormalize(forward);
		// Homing rockets can "see" for 2048 units
		finallength = 2048;
		for (i = 0; i < level.maxclients; i++) {
			spotted = &g_entities[i];
			if (!spotted->client) {
				continue;
			}
			// don't track the player who fired it
			if (spotted == ent->parent) {
				continue;
			}
			// don't track corpses
			if (spotted->client->ps.pm_type == PM_DEAD) {
				continue;
			}
			// don't track players on the same team
			if (OnSameTeam(spotted, ent->parent)) {
				continue;
			}
			// homing rockets won't track invisible players
			if (spotted->client->ps.powerups[PW_INVIS]) {
				continue;
			}
			VectorSubtract(spotted->r.currentOrigin, ent->r.currentOrigin, dir);
			rocketlength = VectorLength(dir);
			if (rocketlength > finallength) {
				continue;
			}
			VectorNormalize(dir);
			if (DotProduct(forward, dir) < 0.9) continue;
			trap_Trace(&vtr, ent->r.currentOrigin, NULL, NULL, spotted->r.currentOrigin, ENTITYNUM_NONE, MASK_SHOT);
			// don't track if the target is not within rocket's line-of-sight (behind walls, etc.)
			if (spotted != &g_entities[vtr.entityNum]) {
				continue;
			}
			// we've found our candidate
			candidate = spotted;
			finallength = rocketlength;
			VectorCopy(dir, finaldir);
		}
		if (candidate) {
			VectorNormalize(ent->s.pos.trDelta);
			VectorAdd(finaldir, ent->s.pos.trDelta, finaldir);
			VectorMA(forward, 0.3, finaldir, finaldir);
			VectorNormalize(finaldir);
			vectoangles(finaldir, ent->r.currentAngles);
			VectorScale(finaldir, 800, ent->s.pos.trDelta);
		}
	}
10

It looks like rocketlength is defined as an int. It should probably be changed to a float.

The float/int casting behavior can be wonky in vms. For example rocketlength = VectorLength(dir); is potentially risky because it relies on automatic casting from VectorLength (float) to rocketlength (int), which is normally valid C but I’ve experienced this kind of thing being buggy in the vm compiler.

I haven’t actually tested your code yet to see if this fixes the problem, this is just a possible issue I noticed looking at the code.

11

I think I may have figured out what was going on. I added a “candidate = NULL” statement right before the for loop since it was never initialized. I didn’t think I needed to at first because the compiler would assume it was null before assignment anyway. Turns out the .vm compiler doesn’t think like that, and yet amazingly it never reported it as a warning / error so I didn’t think anything was wrong (thanks lcc.exe, so helpful /s). The rockets seemed to start working after I put this statement in.

If I had to hazard a guess, I think the .dll compiler assumes uninitialized structs like gentity_t are false upon declaration, while the .vm is the opposite and assumes true. So in the .vm it was calling the code in the if (candidate) statement all the time even if it was never assigned an entity. (Don’t quote me on this, just trying to make sense of this weird issue lol)

If anyone struggling with similar issues happens to come across this post: if you have an gentity_t struct that is assigned an entity only under certain conditions, be sure to initialize it NULL before doing so. The .dll and .vm compilers will not treat it the same way.

Thanks so much for your assistance!

12

In C any uninitialized local variable is undefined, and will often be some random value left in memory, at the mercy of both the compiler and runtime environment. This is the case for any variable, regardless of whether it is struct, pointer, or any other type, so it’s not just specific to the gentity_t type or any particular compiler.

The compiler can warn on this type of situation but I guess the loop in front confused it in this case.

Anyway glad you got it working!