Current status of iqm/md5 models?

[ToKu]

Hello,

Ioquake3 added support for a model format called inter-quake-model (iqm). Another impressive feature of ioquake3, unfortunately not much used as well. Afaik, most of the work was done by zturtleman, again. Thanks for that!
Well, I want to change the player/models animation system from a key/frame based system to a skeletal anaimation system (a physics engine needs skeletal models). In other words I want to change from the current .md3 model format to iqm, or md5., and that results in my question:

Is it worth to implement the md5 model format, or is this format too outdated? Here: http://wiki.ioquake3.org/Modular_Rendering_System, the md5 format is praised as “great starting Point”.
As far as I know iqm models use .md5anim files for animations, so it’s more a question if it’s worth to load the md5 meshes, or prefer the iqm format but add support for reading the md5anim files?

What do you developers think?

BTW, has anyone managed to get animated iqm models to work, in ioquake3?

Thanks in advance!

[quirkychirps]

Didn’t xreal manage to get md5 models integrated?

[zturtleman]

Actually most of the IQM support for ioquake3 was done by gimhael (bugzilla #4965).

Yes quirkychirps, Xreal supports MD5 models.

Viability of IQM in ioquake3

Last month I made a IQM exporter for my fork of Misfit Model 3D. (I haven’t released or fully completed the exporter yet. I haven’t managed to build Misfit Model 3D for Windows yet either.) It allowed me to export my Turtle Arena player model as IQM which renders and animates correctly in ioquake3.

Adding the turtle IQM in ioquake3 opengl1 on q3dm1 using the testModel command causes my frames per-second to drop from 550 to 450. About 30% of ioquake3 CPU time is in RB_IQMSurfaceAnim(), the next highest function uses only 5%. I would not recommend releasing a game using animated IQM models until ioquake3 IQM performance is improved.

---

Screenshot of Turtle Arena (Spearmint engine) with IQM bone drawing (not committed yet) and player drawing hacked up to use a IQM model for whole body (missing torso animation blending).

Here is the turtle IQM test model I used (missing tag_torso, etc). Add to baseq3 and use ioquake3 commands testmodel models/players/raph/raph.iqm, prevframe, and nextframe to view it in-game.

If you have MD5 models you want to test use iqm.exe in command prompt from IQM development kit to convert .md5 and .md5anim files to a single IQM file.

Advantages of IQM

I think the only advantage the MD5 format has over IQM is that MD5 supports more than 4 bone weights per-vertex (unlimited?). However I think IQM has many advantages over MD5.

1. IQM is faster to load because it’s a binary format.

2. There is less processing to do when loading the model because IQM usually contains vertex normals, vertex tangents (useful for normal maps), and triangle adjacency (useful for shadows, but ioq3 doesn’t use it). Most of the data in IQM format is optional. If storing normals, tangents, and triangle adjacency seems like a waste of space, you can modify the exporter to leave them out or make a program to remove them. (Though ioq3 assumes that normals and tangents always exist in IQM.)

3. IQM stores the bone joint quaternion W component to minimize animation jitter.

4. IQM may have better vertex lighting in-game due to using the vertex normals created in the 3D model software instead of auto-generated smoothed normals.

5. IQM supports animating the scale of bones in addition to position and rotation.

IQM Animation

Animations can be stored in the IQM file. There is no need to use .md5anim. Using IQM in ioquake3 animation frames are accessed by frame number like MD3. IQM can contain a list of animations similar to the information listed in quake3 player animations.cfg (animation name, start frame, number of frames, frame rate, and whether the animation loops). There isn’t a way to access list of animations in ioquake3 though.

It’s possible to use separate IQM models for meshes and animations. This allows sharing the animations with multiple models (if they have the same skeleton). Though I wouldn’t recommend putting each animation in a separate file like .md5anim. (If you’re going to load all the animations anyway, using a single file is less file system access.) Separate IQM meshes and animations are support by Spearmint but not ioquake3 because it requires changing the CGame/UI API.

ioquake3 IQM implementation needs to be improved

The major issue with ioquake3’s IQM support is that calculating the vertex positions for rendering animated IQM models is abysmally slow. Though I’m not sure if MD5 would be any faster. Doom 3 uses SIMD optimization for better performance calculating joints and vertex positions on CPU (see Fast Skinning and other Doom 3 docs by Mr. Elusive, includes code licensed as LGPL v2.1 or later which can be used with ioq3).

Performance could be improved in the opengl2 renderer by calculating the vertex positions on the GPU (using GLSL vertex skinning) instead of the CPU. GLSL vertex skinning is probably fairly easy to add and a huge performance improvement. Would still benefit from using SIMD for converting relative-to-parent joints to absolute joints.

[ToKu]

WOW! This is the best summarisation I’ve ever read so far concerning the models!
Thanks alot for the clarification, zturtleman! That was exactly what I had hoped for!
And yeah, right, I totally forgot gimhael’s patch on bugzilla/icculus.org, shame on me! Excuse me gimhael!

First of all, you are absolutely right, GPU-Vertex-Skinning is the most essential part to get high frame rates! We can test this in ETXreal with the r_vboVertexSkinning cvar.
If I remember correctly, in Xreal it was hard-coded. Btw, the Qio engine is not using GPU vertex-skinning, there the framerate is really awful! This makes a big difference!

You know I mainly use(d) Spearmint a lot for my personal programming experience, but to be honest I even failed to animate IQMs in Spearmint as well, though, this was a long time ago. I used a model called Mr.Fixit. Wasn’t this the first ‘official’ IQM model? Anyways it didn’t work for whatever reason.
After that I tried to hack in the models from Xreal (called Xsarge and Shina) but I also failed (because of the missing programming skills).

That’s why I was a bit undecided of using IQM or not. So far I only saw IQM models in Alien Arena but not in any idtech3 game.
I didn’t know that using IQM in Turtle Arena has gone so far.

So all in all using IQM seems okay, that will save lot of writing code implementing the md5 format.

I absolutely have no idea how the IQMs are working with the VAOs of OpenGL2, Xreal (md5) only uses VBOs for md5 not VAOs (for the Vertex Skinning and RT lightning?). I don’t even know if this is something of an issue at all.
It’s really not easy to ‘finalize’ the integration of Xreal into ioquake3, ioquake3’s renderer 2 differs a lot from Xreal (which makes the statement in the already mentioned ioquake3 Wiki hard to realize).

zturtleman, I really admire all the work you have done so far!
Many thanks for the help and your explanations!

[ToKu]

Okay, after reading all the articles in depth it seems iqm is the better format (imo).
So I will try to implement:

1. Ability to have animated IQM models (side-by-side with the existing animations for md3 etc.).
2. GPU-Vertex-Skinning (at least for renderer 2).

I am curious how far that will be successful!

What’s surprising to me is how fast and flawless your turtle IQM test model is loading, with other models in the past this wasn’t the case, at least not with my experimentations.

Anyways, thank you zturtleman to provide your test model here! The turtles are already flying!

[zturtleman]

Yeah, the Mr.Fixit IQM has a mesh with too many vertexes to load in ioquake3. I always had to increase SHADER_MAX_VERTEXES to 2000 in code/qcommon/qfiles.h to test it. I guess most MD5 models converted to IQM would probably also fail to load for the same reason.

ioq3 using VAOs isn’t an issue for GPU vertex skinning. VAOs are used together with VBOs and GLSL in opengl2 renderer. As far as I know GPU vertex skinning can’t be added to opengl1 without switching from fixed function to GLSL (which is the main reason opengl2 is a separate renderer).

[ToKu]

Confirmed, all the models I wanted to load failed loading because of the SHADER_MAX_VERTEXES limit.
In my testing code (https://github.com/KuehnhammerTobias/ioqw-XREAL-SKELETAL-RAGDOLL-PHYSICS/commit/b9a9e86792f5e16fb5f61d21ea0a29b4d78a1f9c) I also increased SHADER_MAX_VERTEXES to 10000, all models will load. Xreal and most games based on it also use a value of 10000 for SHADER_MAX_VERTEXES.

It’s okay if only one renderer will using GPU vertex skinning (imo). The really ugly part is the needed API changes (needed for skeletal models), a downfall for implementing into iquake3, you already pointed that out, but I deny to accept that

BTW, the iqm compiler seems to work fine, although there is not much feedback. Well, at lest it compiled a md5 model to iqm

[zturtleman]

I improved the IQM CPU vertex skinning performance and added GPU vertex skinning to the opengl2 renderer. IQM may actually be usable now.

CPU vertex skinning now only calculates unique vertex bone indexes and weights combinations once per model surface; inspired by Cube 2: Sauerbraten documentation.

http://sauerbraten.org/docs/models.html

To optimize animation of the model on both CPU and GPU, keep the number of blend weights per vertex to a minimum. Also, similar combinations of blend weights are cached while animating, especially on the CPU, such that if two or more vertices use the same blend weights, blending calculations only have to be done once for all the vertices - so try and minimize the number of distinct combinations of blend weights if possible.

opengl1

Vertex skinning for my turtle IQM use to take 3.6 times as long as MD3. Now it takes 1.6 times as long as MD3.

Old IQM CPU vertex skinning:
(draw entity with model 100 times, time measured in nanoseconds but displayed as milliseconds)

(100x turtle)     total IQM frame time 43 msec, vertex 38 msec, skeleton 4 msec
(100x mrfixit)    total IQM frame time 17 msec, vertex 16 msec, skeleton 1 msec
(100x qshambler)  total IQM frame time 42 msec, vertex 41 msec, skeleton 1 msec
(100x qvore)      total IQM frame time 10 msec, vertex  9 msec, skeleton 1 msec

New IQM CPU vertex skinning:

(100x turtle)     total IQM frame time 20 msec, vertex 16 msec, skeleton 4 msec
(100x mrfixit)    total IQM frame time  7 msec, vertex  6 msec, skeleton 1 msec
(100x qshambler)  total IQM frame time 38 msec, vertex 37 msec, skeleton 1 msec
(100x qvore)      total IQM frame time  7 msec, vertex  6 msec, skeleton 1 msec

MD3: The performance goal

(100x turtle)     MD3 frame time 12 msec

opengl2

I didn’t know CPU vertex skinning in opengl2 renderer was so much slower than in opengl1.

Old IQM CPU vertex skinning:

(100x turtle)     total IQM frame time 127 msec, vertex 117 msec, skeleton 10 msec
(100x mrfixit)    total IQM frame time  51 msec, vertex  50 msec, skeleton  1 msec
(100x qshambler)  total IQM frame time 121 msec, vertex 120 msec, skeleton  1 msec
(100x qvore)      total IQM frame time  30 msec, vertex  29 msec, skeleton  1 msec

New IQM CPU vertex skinning:

(100x turtle)     total IQM frame time  81 msec, vertex  71 msec, skeleton 10 msec
(100x mrfixit)    total IQM frame time  30 msec, vertex  29 msec, skeleton  1 msec
(100x qshambler)  total IQM frame time 108 msec, vertex 107 msec, skeleton  1 msec
(100x qvore)      total IQM frame time  22 msec, vertex  21 msec, skeleton  1 msec

When drawing 100 turtle IQM models there are 40 msec spent in R_VaoPackNormal() / R_VaoPackTangent() called by RB_IQMSurfaceAnim() each frame. Setting r_depthPrepass to 0 cuts total time in half because r_depthPrepass 1 draws the models a second time. Switching to VBO avoids R_VaoPackNormal() slowness and reduces impact of r_depthPrepass.

opengl2 GPU vertex skinning compared to new CPU vertex skinning and opengl1 renderer:

(100x turtle)     GPU:  60fps, CPU:  7fps, opengl1: 25fps
(100x mrfixit)    GPU: 160fps, CPU: 18fps, opengl1: 60fps
(100x qshambler)  GPU: 200fps, CPU:  6fps, opengl1: 18fps
(100x qvore)      GPU: 200fps, CPU: 30fps, opengl1: 81fps

[ToKu]

Man, you are really fast! Was this already on your HDD, or was this the a new high-speed coding competition? How can you implement vertex skinning so fast?

It’s so cool to have this in ioquake3 now. Imagine all the high-poly models we can have in idtech3, I’m so excited!

What do you think, is the opengl2 renderer capable to even outperform the Xreal renderer (as far as FPS are concerned), due to it’s additional enhancements, like the VAOs? Your test results for the opengl2 renderer are impressive, to me it seems the opengl2 renderer has a lot of unused potential.

Hmm, what I do not understand is why 100 qshambler or 100 qvore are at 200 fps whereas 100 turtle models are at 60 fps on GPU vertex skinning (opengl2)?
Is there some other aspect than triangle/vertex count that influences this gap?
Howsoever, opengl2 is damned fast compared to CPU and/or opengl1!

You’re genius zturtleman, do you know that! You made my day!
So again, thank you very much zturtleman! This is definitely a milestone for the idtech3 world!

[zturtleman]

I recently implemented IQM GPU vertex skinning in my Toy Box engine. It wasn’t very hard to add to ioquake3 after that. I don’t know enough about Xreal renderer to guess how it performs compared to ioquake3/opengl2.

My turtle IQM is probably slow due to being split into 16 meshes. Fewer meshes is better. There is an OpenGL draw call for each VAO mesh which takes time and all the bones are recalculated for each mesh (ComputePoseMats()). Turtle IQM’s skeleton time is much higher than the other models because it has many meshes.

Performance with many meshes may be improved a little by making a list of bones referenced by each mesh and then only calculating bones required by the mesh when drawing. (or I could combine meshes in the turtle IQM file.) (or engine fork could store bones in trRefEntity_t/refEntity_t instead of calculating when drawing each mesh.)

turtle    has 16 meshes, 4299 vertexes, 5873 triangles, 51 bones
mrfixit   has  2 meshes, 1861 vertexes, 2988 triangles, 75 bones
qshambler has  2 meshes, 3799 vertexes, 6780 triangles, 47 bones
qvore     has  2 meshes, 1032 vertexes, 1478 triangles, 45 bones

[zturtleman]

I updated turtle IQM (version 2) and fixed axis returned by trap_R_LerpTag(). tag_wp_away_primary (belt weapon holder) had it’s (blue) Z axis pointing up when it should be pointing down.

Turtle IQM v2:

• Add bone joints to IQM for tag_torso, tag_head, tag_weapon, tag_flag, tag_wp_away_primary, tag_wp_away_secondary.
• Merge meshes to reduce number of meshes from 16 to 9.
• Improve IQM normals around eyes by increasing normal max smoothing angle from 89 to 180 in Misfit Model 3D. (Changed for all meshes, decreases number of vertexes and seems to look better. Issue existed in turtle MD3 as well.)
• Rename eye/mouth texture from default.png to models/players/raph/white.png and use shader to make it full bright.

OpenGL1 vertex skinning (decrease 2 msec)

(100x turtle-v2)   total IQM frame time 18 msec, vertex 15 msec, skeleton 3 msec

OpenGL2 vertex skinning (increase GPU 20fps, CPU 3fps, opengl1 10fps)

(100x turtle-v2)  GPU:  82fps, CPU:  9fps, opengl1: 35fps

Model change -7 meshes, -420 vertexes, +0 triangles, +6 bones.

turtle-v2 has  9 meshes, 3879 vertexes, 5873 triangles, 57 bones

[ToKu]

Did you update your turtle model by using your enhanced Misfit Model 3D program (https://clover.moe/mm3d/), or did you use a different 3D modelling program?

The last days I spent a lot of time trying out the Misfit Model 3D program to also rework a few models (I can’t use your updated version, unfortunately I was only able to try the official version 1.3.8).
The good thing is Misfit Model 3D reminds me very much of Milkshape 3D (at least compared to Blender, which i never got used with…)
So because probably it is like with any other program, if you are used to it, it is very unlikely to change to a different program. Switching from Milkshape 3D to Misfit Model 3D seems to be easy
This results in my next question: Do you plan to add md5 or even iqm support to your Misfit Model 3D project (https://github.com/zturtleman/mm3d)?
Sorry if the question is silly, as already said, I did testing the official version, not yours!
At the very least, I could help translate the menu into German, which has some problems, but that’s another topic I think …

Anyways, thanks a lot man!

[zturtleman]

Yeah, I used my Misfit Model 3D fork. I completed support for saving Inter-Quake Export (IQE) models. IQE (text format) can be converted to IQM (binary format with additional generated/sorted data) by the iqm program in IQM development kit. I plan to add support for saving/loading IQM in the future.

There is a Misfit Model 3D fork (Misfit Cubed) that added MD5 export. I may add support for saving/loading MD5 to my fork. Though I should probably figure out building and releasing MM3D on Windows first. The German translation file is here; I’ll update it tomorrow to add new messages. It’s fine if you want to wait until I actually getting it working on Windows.

I made Misfit Model 3D split the turtle model into three IQE models like Quake 3 players (not committed yet). Now my turtle IQM (version 3) is a fully usable player model in ioquake3 after fixing head model bounds. Add the pk3 and run model raph; addbot raph. I also added a MD3 player/bot (raph_md3) to compare performance / behavior. I’m not quite sure why the vertex normals/shading are so different in the MD3 version. It might be partially a issue with MM3D MD3 player export.

[ToKu]

Wow, IQE support in your Misfit Model 3D fork is fantastic!

Okay, please feel free to contact me if you need any help in testing or whatsoever! You know I’m a lousy programer, but a good tester (cough…)
Though, at first I have to learn the Misfit Model 3D in general… even at my age this shouldn’t be too hard I hope.

Thanks for all the answers and all the work!

[zturtleman]

IQM models didn’t always render correctly between frames. This is now fixed.

Test using testModel models/players/raph/raph.iqm 0.5 and then run nextFrame or prevFrame to change test frame. This will lerp half way between current test frame and frame 0. Some frames may have weird scaling. nextFrame to frame 40 gave this result: vertical crush and flat arm.

[subterfuge]

Can you recommend any good books for building complex skeletal mesh formats?

[zturtleman]

No, I don’t have any book recommendations.

[zturtleman]

@zturtleman

I made Misfit Model 3D split the turtle model into three IQE models like Quake 3 players (not committed yet).

The Quake 3 style player export to IQE that I wrote about in 2018 is now available in Maverick Model 3D 1.3.15. The exported head.iqe, upper.iqe, lower.iqe can be converted to IQM and used for a skeletal player model in ioquake3.