Making new art assets (and WIP)

[Darklord]

Mmm that remind me of a giant rasher of Bacon.

Mmm Bacon...

Daniel.

[CaveTroll]

Thanks bitcpy for your great reverse engineering work!
I have several remarks about your script and the way it behaves (with my version of blender, at least):

• The UV map seems wrong (to get the correct UV coordinates, it seems you have to take (x, 1-y) where x and y are the UV coordinates your script currently produces)
• It seems the models from the game use the left hand convention; it is therefore more intuitive to swap the y axis and the z axis during importation; of course, the vertex order of the faces then needs to be reversed as well to keep them correctly oriented

I use blender 2.60, but I think those problems are version independent. I have made changes to your script to correct those two aspects; plus, I've tweaked it so that the specular map and the normal map are rendered as such in blender. Here's the new version:

Code: Select all

bl_info = {
    "name": "Legend of Grimrock Mesh Format (.mesh)",
    "author": "",
    "version": (1, 0, 1),
    "blender": (2, 5, 7),
    "api": 36339,
    "location": "File > Import > Legend of Grimrock Mesh (.mesh)",
    "description": "Import Legend of Grimrock Meshes (.mesh)",
    "warning": "",
    "wiki_url": "",
    "tracker_url": "",
    "category": "Import-Export"}

import os
import struct

import bpy
from mathutils import *

from bpy.props import *
from bpy_extras.io_utils import ExportHelper, ImportHelper
from bpy_extras.image_utils import load_image

class _mesh_material(object):
    __slots__ = (
        "bl_mat",
        "bl_image",
        "name",
        "index_offset",
        "num_primitives",
        )

    def __init__(self):
        self.bl_mat = None
        self.bl_image = None
        self.name = "<Empty>"
        self.index_offset = 0
        self.num_primitives = 0

# reads file magic from file
def read_magic(file_object, endian = '<'):
    data = struct.unpack(endian+'4s', file.read(4))[0]
    return data;

# read signed integer from file
def read_int(file_object, endian = '<'):
    data = struct.unpack(endian+'i', file_object.read(4))[0]
    return data

def read_int2(file_object, endian = '<'):
    data = struct.unpack(endian+'ii', file_object.read(8))
    return data

def read_int3(file_object, endian = '<'):
    data = struct.unpack(endian+'iii', file_object.read(12))
    return data

def read_int4(file_object, endian = '<'):
    data = struct.unpack(endian+'iiii', file_object.read(16))
    return data

# read floating point number from file
def read_float(file_object, endian = '<'):
    data = struct.unpack(endian+'f', file_object.read(4))[0]
    return data

def read_float2(file_object, endian = '<'):
    data = struct.unpack(endian+'ff', file_object.read(8))
    return data

def read_float3(file_object, endian = '<'):
    data = struct.unpack(endian+'fff', file_object.read(12))
    return data

def read_float4(file_object, endian = '<'):
    data = struct.unpack(endian+'ffff', file_object.read(16))
    return data

def read_string(file_object, num, endian = '<'):
    raw_string = struct.unpack(endian+str(num)+'s', file_object.read(num))[0]
    data = raw_string.decode("utf-8", "ignore")
    return data


# Build content base path
def build_assets_path(filename):
    #filename = "C:/grimrock/assets/models/wall_sets/dungeon/floor_01.mesh"
    pathname = os.path.dirname(filename)

    index = pathname.rfind("assets")
    if index < 0:
        return pathname

    return pathname[0:index]


# Parse the '/assets/materials/default.materials' file
def parse_material_file( path_assets, find_material ):

    # diffuse, normal, specular
    texture_names = ["", "", ""]

    filename = os.path.join(path_assets, "assets/materials/default.materials")
    if os.path.exists(filename) == False:
        return texture_names

    file = open(filename, 'r')

    # Parse material (very primitive ...)
    material_scope = False
    material_found = False
    for line in file.readlines():
        tokens = line.split()
        num = len(tokens)
        if num <= 0:
            continue
        token0 = tokens[0].strip()
        if material_scope == True:
            if token0 == "}":
                if material_found == True:
                    break
                material_scope = False
                continue
            if num < 3:
                continue
            token1 = tokens[1].strip()
            token2 = tokens[2].strip()

            if material_found == True:
                if (token0 == "DiffuseMap" or token0 == "SpecularMap" or token0 == "NormalMap") and token1 == "=":
                    if token2.endswith(','):
                        token2 = token2[0:-1]
                    if token2.startswith('"') and token2.endswith('"'):
                        token2 = token2[1:-1]
                    if token0 == "DiffuseMap":
                        texture_names[0] = token2
                    elif token0 == "NormalMap":
                        texture_names[1] = token2
                    elif token0 == "SpecularMap":
                        texture_names[2] = token2
            else:
                if token0 == "Name" and token1 == "=":
                    if token2.endswith(','):
                        token2 = token2[0:-1]
                    if token2.startswith('"') and token2.endswith('"'):
                        token2 = token2[1:-1]
                    if token2 == find_material:
                        material_found = True
        else:
            if token0 == "material{":
                material_scope = True
            elif num > 1 and (token0 == "material" and tokens[1] == "{"):
                material_scope = True
    file.close()

    for i in range(len(texture_names)):
        texture_name = texture_names[i]
        if len(texture_name) <= 0:
            continue

        # Check if source exists (.tga) or whatever was given in material file
        source_name = os.path.join(path_assets, texture_name)
        if os.path.exists(source_name):
            texture_name[i] = source_name
            continue

        # Check for .d3d9_texture instead
        base_name, ext = os.path.splitext(source_name)
        dxt_name = base_name + ".d3d9_texture"

        if os.path.exists(dxt_name):
            texture_names[i] = dxt_name
            continue

        # Didn't find file, just remove it
        texture_names[i] = ""

    return texture_names


def load_mesh(filename, context):
    # Dig out file base name and extension
    name, ext = os.path.splitext(os.path.basename(filename))

    path_assets = build_assets_path(filename)

    print("Opening file: " + filename)
    file = open(filename, 'rb')
    try:
        magic = struct.unpack("<4s", file.read(4))[0]
    except:
        print("Error parsing file header!")
        file.close()
        return

    # Figure out if it's a valid mesh file
    if magic != b'MESH':
        print("Not a valid mesh model!")
        file.close()
        return

    mesh_unknown = read_int(file)
    num_vertices = read_int(file)

    # This will store all vertices and index info
    vertex_set = []
    indices = []
    materials = []

    for i in range(15):
        # Assumption that this is what it actually means ...
        data_type = read_int(file)
        num_comp = read_int(file)
        byte_width = read_int(file)

        print( "Vertex Attrib %d" % i )
        print( "\tData Type: %d" % data_type )
        print( "\tNum Components: %d" % num_comp )
        print( "\tByte Width: %d" % byte_width )

        # Skip empty
        if data_type == 0 and num_comp == 0 and byte_width == 0:
            continue

        # Report unknown data types (?)
        type_size = 0
        if data_type == 2:
            type_size = 4   # int32 ?
        elif data_type == 3:
            type_size = 4   # float
        else:
            print("Valid mesh, but not supported vertex data: data_type = %d" % data_type)
            file.close()
            return

        # byte_width should be type_size*num_comp
        if byte_width != (num_comp*type_size):
            print("Valid mesh, but not supported vertex data: data_size = %d" % data_size)
            file.close()
            return

        vertex_data = []
        for j in range(num_vertices):
            data = []
            if num_comp == 2:
                if data_type == 2:
                    data = read_int2(file)
                elif data_type == 3:
                    data = read_float2(file)
            elif num_comp == 3:
                if data_type == 2:
                    data = read_int3(file)
                elif data_type == 3:
                    data = read_float3(file)
            elif num_comp == 4:
                if data_type == 2:
                    data = read_int4(file)
                elif data_type == 3:
                    data = read_float4(file)
            vertex_data.append(data)

        vertex_set.append(vertex_data)

    num_indices = read_int(file)
    for i in range(num_indices):
        index = read_int(file)
        indices.append(index)

    num_materials = read_int(file)
    for i in range(num_materials):
        name_length = read_int(file)
        material_name = read_string(file, name_length)
        print("Material Name: " + material_name)

        material_unknown = read_int(file)
        #if pre_material != 2:
        #    print("Valid mesh, but not supported material data: pre_material = " + str(pre_material))
        #    file.close()
        #    return

        index_offset = read_int(file)
        num_primitives = read_int(file)

        material = _mesh_material()
        material.name = material_name
        material.index_offset = index_offset
        material.num_primitives = num_primitives

        materials.append(material)

    # Origin x,y,z (?)
    origin = read_float3(file)

    # ?
    read_float(file)

    # Bounds min x,y,z (?)
    bounds_min = read_float3( file )

    # Bounds max x,y,z (?)
    bounds_max = read_float3( file )

    file.close()

    # Build the blender object
    build_objects(name, vertex_set, indices, materials, path_assets)


def add_material_texture( bl_material, type, filename ):
    # Create texture
    bl_texture = bpy.data.textures.new(name=type, type="IMAGE")

    # Try to load image
    image = load_image(filename)
    if image:
        bl_texture.image = image

    # Create texture slot and link to 'uvset1'
    mtex = bl_material.texture_slots.add()
    mtex.texture = bl_texture
    mtex.texture_coords = "UV"

    #XXX
    if type=="specular":
        mtex.use_map_color_diffuse = False;
        mtex.use_map_color_spec = True;
    if type=="normal":
        # Bump method: default; space: texture space
        mtex.use_map_color_diffuse = False;
        mtex.use_map_normal = True;
    #/XXX

    return image


def create_material( path_assets, material ):
    # Parse material file for the actual texture images
    texture_names = parse_material_file(path_assets, material.name)

    # Create blender material
    bl_material = bpy.data.materials.new(material.name)

    # Create textures and texture slots on material
    image_diffuse = add_material_texture(bl_material, "diffuse", texture_names[0])
    image_normal = add_material_texture(bl_material, "normal", texture_names[1])
    image_specular = add_material_texture(bl_material, "specular", texture_names[2])

    # Assign material
    material.bl_mat = bl_material

    # Assign preview display image for material
    material.bl_image = image_diffuse


def build_objects(name, vertex_set, indices, materials, path_assets):
    print("Building Blender data")

    vertices = vertex_set[0]
    normals = vertex_set[1]
    tangents = vertex_set[2]
    bitangents = vertex_set[3]
    texcoords = vertex_set[4]

    num_vertices = len(vertices)
    num_indices = len(indices)
    num_primitives = int(num_indices / 3)
    num_materials = len(materials)

    print("Num vertices: " + str(num_vertices))
    print("Num indices: " + str(num_indices))
    print("Num primitives: " + str(num_primitives))
    print("Num materials: " + str(num_materials))

    # Before adding any meshes or armatures go into Object mode.
    if bpy.ops.object.mode_set.poll():
        bpy.ops.object.mode_set(mode='OBJECT')

    # Create mesh
    me = bpy.data.meshes.new(name)

    # Add vertices
    me.vertices.add(num_vertices)
    for i in range(num_vertices):
        #XXX
        me.vertices[i].co[0] = vertices[i][0]
        me.vertices[i].co[1] = vertices[i][2]
        me.vertices[i].co[2] = vertices[i][1]
        #/XXX

    # Add faces
    me.faces.add(num_primitives)
    for fi in range(num_primitives):
        idx = fi * 3
        for i in range(3):
            #XXX
            me.faces[fi].vertices_raw[2-i] = indices[idx+i]
            #/XXX

    # Add uv map
    uvmap = me.uv_textures.new("uvset1")
    for fi in range(num_primitives):
        uvf = uvmap.data[fi]
        #XXX
        uvf.uv1 = texcoords[indices[(fi*3)+2]]
        uvf.uv2 = texcoords[indices[(fi*3)+1]]
        uvf.uv3 = texcoords[indices[(fi*3)+0]]
        uvf.uv1.y = 1 - uvf.uv1.y
        uvf.uv2.y = 1 - uvf.uv2.y
        uvf.uv3.y = 1 - uvf.uv3.y
        #/XXX

    # Create object and link with scene
    ob = bpy.data.objects.new(name, me)
    bpy.context.scene.objects.link(ob)

    # Add/create all our materials
    for material in materials:
        create_material(path_assets, material)

    # Assign materials to mesh
    for mi in range(num_materials):
        material = materials[mi]
        me.materials.append(material.bl_mat)

        face_offset = int(material.index_offset / 3)
        for fi in range(material.num_primitives):
            me.faces[face_offset+fi].material_index = mi

            uvf = uvmap.data[face_offset+fi]
            uvf.image = material.bl_image

    # Update mesh and scene
    me.update()
    bpy.context.scene.update()


class IMPORT_OT_mesh(bpy.types.Operator, ImportHelper):
    # Import Mesh Operator.
    bl_idname = "import_scene.mesh"
    bl_label = "Import Mesh"
    bl_description = "Import a Legend of Grimrock mesh"
    bl_options = { 'REGISTER', 'UNDO' }

    filepath = StringProperty(name="File Path", description="Filepath used for importing the mesh file.", maxlen=1024, default="")

    def execute(self, context):
        load_mesh(self.filepath, context)
        return {'FINISHED'}

    def invoke(self, context, event):
        wm = context.window_manager
        wm.fileselect_add(self)
        return {'RUNNING_MODAL'}


def menu_func(self, context):
    self.layout.operator(IMPORT_OT_mesh.bl_idname, text="Legend of Grimrock Mesh (.mesh)")


def register():
    bpy.utils.register_module(__name__)
    bpy.types.INFO_MT_file_import.append(menu_func)


def unregister():
    bpy.utils.unregister_module(__name__)
    bpy.types.INFO_MT_file_import.remove(menu_func)


if __name__ == "__main__":
    register()

The parts I changed are located between "#XXX" and "#/XXX" comments.

[Isaac]

To install the add-on in Blender just copy the Python script and place it in your add-ons folder.
Example
C:\Program Files\Blender Foundation\Blender\2.62\scripts\addons\io_import_grimrock.py

When I do this, I do not see the import option for it in the File>>Import menu.

Blender's documentation states that addons must be installed, and enabled; (as I'm sure you already know). When I try to install the addon (via user preferences), it never seems to show up in the list, so there is no option to give it an enabling 'check'.

I'm still rather new to Blender 262 (and now 263); have you any idea what I may be missing?

[CaveTroll]

You can try this: start blender, open the internal text editor (instead of the 3d view, for example), open the script file ("text" menu, "open text block") and run it ("run script" button). It it worked, it will put the "Legend of Grimrock mesh (.mesh)" item in the "Import" submenu. If it did not, it should at least print a more or less helpful message (which you can post here if needed).

[Isaac]

You can try this: start blender, open the internal text editor (instead of the 3d view, for example), open the script file ("text" menu, "open text block") and run it ("run script" button). It it worked, it will put the "Legend of Grimrock mesh (.mesh)" item in the "Import" submenu. If it did not, it should at least print a more or less helpful message (which you can post here if needed).

This works, but when I press Alt+P to run the script, I get the following error.


** after this, I selected the whole block and 'de-indented' it, just to see... It then chokes on some other unexpected indent.

[CaveTroll]

Can you make sure you've got no spaces/tabs/whatever between "bl_info" and the beginning of the first line of the script? (If you have some, erase them.)

[Isaac]

Can you make sure you've got no spaces between "bl_info" and the beginning of the first line of the script?

Seems not, but if it were, then it was in the file; I did not alter anything in it yet.

(from above: ** after this, I selected the whole block and 'de-indented' it, just to see... It then chokes on some other unexpected indent.)

I am trying this in Blender 2.62.0 [r44136], does this happen for no one else?

[CaveTroll]

Ok, I see on your screenshot your indentation is totally messed up (was it before or after you deindented the code?). Remember, indentation is how python knows where blocks start and end, so it is absolutely critical!

I recommend you copy/download the script again from bitcpy's pastebin (or my previous post), and ensure you have 4 spaces per indentation level throughout the file. Then try again!

[Isaac]

Ok, I see on your screenshot your indentation is totally messed up (was it before or after you deindented the code?). Remember, indentation is how python knows where blocks start and end, so it is absolutely critical!

I recommend you copy/download the script again from bitcpy's pastebin (or my previous post), and ensure you have 4 spaces per indentation level throughout the file. Then try again!

I see what it is. Firefox ignores the indention in the code block of your post; not so in IE.

I now have no errors, but running the script seems to have no effect on anything. No new object.

[CaveTroll]

I now have no errors, but running the script seems to have no effect on anything. No new object.

It seems some of the models do not have 2 or 3 as the value of their "data_type" field. In which case, importing them generates the message "Valid mesh, but not supported vertex data: data_type = [something]" on the standard output ("[something]" is generally 0). This message cannot be seen without a "standard output" (which is typically the case when using Windows), and the script indeed seems not to generate anything in those cases. I was hoping bitcpy would investigate and correct this problem (it affects for mostly item models).

Have you extracted all the models? Can you try and import for instance "assets/models/wall_sets/dungeon/goromorg_statue_04.mesh"? This one should work fine, normally. If it does not, you have some other problem...