Featured Post

Deus Ex - Extraction Guides (Master Post)

Deus Ex (2000) Extract Audio Extract Textures & Models Deus Ex: The Conspiracy (2002) Extract Video Deus Ex: Invisible War (2003) Extrac...

Showing posts with label Deus Ex: Mankind Divided. Show all posts
Showing posts with label Deus Ex: Mankind Divided. Show all posts

Saturday, 18 March 2023

Deus Ex - Extraction Guides (Master Post)

Deus Ex (2000)

Deus Ex: The Conspiracy (2002)

Deus Ex: Human Revolution (2011)

Deus Ex: The Fall (2013)

Deus Ex: Mankind Divided (2016)

at No comments:
Labels: , , , , ,

Deus Ex: Mankind Divided - Extract Textures & Models

Note: In this post, I will not be covering how to unpack the game content as I have already covered this before in a previous post. You must unpack the game files before starting here, so for instructions on how to do that please follow the other guide I created, and then come back to this post which is purely focusing on textures and models.

Exporting textures to .png using DEMDDatabase

  1. Run DEMDDatabase.exe and open the unpacked database from the DXMDExtract extraction
  2. In the file tree, search for the texture(s) you want, for example for character textures, they are usually found under characters > {character you want} > {character you want}_material, etc.
  3. Select the .tex file and go to File > Export as... > and then you can save the individual texture

Exporting meshes using DEMDDatabase

  1. Similar to the above, character models can be exported following the same method, just instead of going into the character materials, go back one step and find the character model file. This can be exported as an .obj file
  2. You can also export all the textures and meshes for a specific character by going to their parent node in the tree > File > Mass fast export.
    Note: This will not work unless you set your fast export savepath. To do this, go to File > Settings > Fast export tab > select the folder you wish to save to, and you can also change the automatic Image format and Models format from here too

This provides the basis of extracting meshes; it is possible to load and combine the different mesh parts and material textures into Blender (or preferred 3D software):

GUIDE UPDATE: NOV 2025

At the time I wrote this guide (Mar 2023), DEMDdb was working properly, i.e. the mesh parts would export with the file format selected.

However, lately, it has occured to me and a few others who tried following this guide, that some errors seem to occur where the mesh does not export, or the only format it will export is an .stl or a format you don't want. While you can still get at the mesh, anything more functional like the rig or weights will not be exported in that format, so it is not ideal whatsoever.

I have since found a way to do this while bypassing DEMDdb all together. Below is the workaround for when DEMDdb fails to export meshes, using DXMDmodel and PureHair, as well as a brief explanation on how to get the mesh working with XNA Lara posing studio.

Extracting Model and Rig with DXMDmodel

  1. Open NameMap.txt and search for the weightedprim (.prim) and bonerig (.borg) corresponding to a specific model (usually suffixed with .wl2). Alternatively, browse the filetree view in DEMD Database to find what you're looking for if the text file is confusing.
  2. Locate the .bin file for both of these. The .bin file will be the second string of numbers on the same line in the NameMap and also at the top of the texture/model viewer panel in DEMDdb. I've outlined the .bin search process in more detail in my other guide about extracting audio & video from Mankind Divided, check that guide out if you need more help.
  3. Drag and from them onto DXMDmodel.exe; it should generate an .ascii mesh (geometry + weights) file and an .smd file (skeleton + rig)

Importing into Blender using Blender Source Tools 

  1. In order to import these into Blender, you need to install the Blender add-on "Blender Source Tools" from Steam Review, it works here even though Mankind Divided’s engine is not the Source engine. To install add-ons in Blender, you usually just need to go into Edit > Preferences > Add-ons > Install from File > browse to the Blender Source Tools .zip you downloaded > tick the checkbox to enable
  2. Usually, to import with this add-on you need a .qc file which helps Blender locate the different parts, however I was able to import just the .smd. Go to File > Import > Source Engine (.smd, .vta, .dmx, .qc) and browse for the .smd file of the mesh you want to import; the .ascii mesh should automatically load with it.
  3. In Blender you should see a mesh, with different parts coloured various shades, the armature/posing rig. Sometimes this is all that is needed to proceed, but some characters/meshes are divided up into multiple parts, probably due to these items requiring different rendering/physics in-game, i.e. the body, the cloth, the hair. In this case, keep reading.

Extracting and Importing Clothes with XNALara i/o Tools

  1. Go back to the NameMap (or DEMDdb file tree) and search for the corresponding .pc_apex file for the model you've already converted, i.e. for Pritchard his model was missing his jacket, so I found pritchard_jacket.pc_apex
  2. Locate the .bin file and drag and drop it onto DXMDmodel.exe
  3. It should generate an .ascii mesh (geometry + weights) file
  4. In order to import/append the .ascii files into your previous Blender project with the other mesh parts, you need to install the Blender add-on "XNALara-io-Tools" from Valerie-Bosco. After install, go to File > Import > XNA Lara / XPS > XNA Lara/XPS Model (.ascii/.mesh/.xps) and browse for the .ascii file for the clothes mesh.

Extracting and Importing Hair with XNALara i/o Tools

  1. Go back to the NameMap (or DEMDdb file tree) and search for the corresponding .pc_hair file for the model you've already converted, i.e. for Pritchard his model was missing his jacket, so I found pritchard_hair.pc_hair
  2. Locate the .bin file and drag and drop it onto PureHair.exe
  3. It should generate an .ascii mesh (weights) file and an .obj (geometry) file
  4. In order to import/append the .ascii files into your previous Blender project with the other mesh parts, you need to install the Blender add-on "XNALara-io-Tools" from Valerie-Bosco. After install, go to File > Import > XNA Lara / XPS > XNA Lara/XPS Model (.ascii/.mesh/.xps) and browse for the .ascii file for the hair mesh and the .obj file should automatically load with it
Notes: 
  • In the case of duplicate armatures/posing rigs imported into Blender, choose main armature, clear parent of appended objects, set their parent to main armature, combine collections, ensure the armature modifier on each meshport is the main armature
  • The texture coordinates are sometimes flipped on the Y-axis, manually fix the UVs by selecting all the UV islands, S (scale), Y (vertically), -1 (flip vertically down)
  • The textures can be batch-exported with DEMD Database after setting the Mass Export settings

Preparing an extracted mesh for XNA Lara Posing Studio

General tips

  1. Version: Make sure the version of Blender is compatible with the XPS import/export tool you are using. The very latest version of Blender may not be compatible with the old tools. The oldest version of Blender which can open 4.x files and newer is 3.6. Some tools only work with version of Blender below 2.80 as well - it is not possible to open files created with 3.6 and above in a version this old, so you will need to change the Format
  2. Format: If you get crashes due to versioning (i.e. the Blend file you’ve been working with is too new), export into an intermediate format such as .fbx (which retains the bones, armature, materials) and import that into a previous version of Blender. Doing this will reset the Shader back to Principled BSDF for all the materials though, so you will need to change it back to XPS Shader and add the textures again manually

Bones, Armatures and Posing

  1. Armatures: Ensure all the mesh parts are linked to the same Armature via the modifier, are parented to the same Armature with Empty Groups via the Object settings in Object Mode, and they have the same Vertex Groups in the data panel
  2. Posing: To pose, click on the Armature in the Scene collection and change to Posing Mode. Select the bone nodes and move/rotate and it should move the mesh parts with the associated Vertex Groups
  3. Bones: You may find it difficult to see the mesh under the bone nodes, to change this you need to go into the Armature modifier option and change the Viewport Display to sticks and make sure there isn’t a custom mesh shape applied to the nodes

Missing textures or “missing.png” error when importing .xps

  1. Textures: Ensure all used textures are saved in the same folder as the one you export the .xps file to, if you want a portable .xps file you need to always bring the textures with it
  2. Materials: Ensure that the materials are renamed so both the image texture and the material both have the same name - leave out file type identifier suffixes like _diff or _norm
  3. UV Map: Ensure all the materials have only one UV map assigned, and rename it UVMap
  4. Shader: Ensure the textures are using XPS Shader and not Principled BSDF - if the shader is not available you need to import a pre-existing .xps model to have the option
  5. Vertex Colors: Remove any vertex colors - any being present causes the exporter to replace the materials texture with plain black
  6. Vertex Groups: Ensure that every mesh part has the same Vertex Groups - you can apply a Data Transfer modifier or run this Python script in the scripting panel:

import bpy


source = bpy.data.objects['goodmesh']  # replace with your mesh names

targets = ['badmesh1','badmesh2']  # replace with your mesh names


for target_name in targets:

    target = bpy.data.objects[target_name]

    for vg in source.vertex_groups:

        if vg.name not in target.vertex_groups:

            target.vertex_groups.new(name=vg.name)

Tools

Please view the Tools section of my other guide to see the full list of tools needed to extract and convert content from Deus Ex: Mankind Divided.
at 3 comments:
Labels:

Deus Ex: Mankind Divided - Extract Audio & Video

Note: All tools used are linked at the bottom of the post.

In this guide I will cover:
  • How to extract content from the game and locate audio and video files
  • How to convert the audio and video into viewable formats

Extracting the files from the game using DXMD Extract

  1. Unpack the DXMDExtract archive and move DXMDExtract.exe to the installation location for the game, for example, for the Steam version:

    Main game: Steam\steamapps\common\Deus Ex Mankind Divided\runtime
    DLC: Steam\steamapps\common\Deus Ex Mankind Divided\DLC\runtime

  2. In that same folder, create a new text file. Open it in a text editor, and enter the following line:

    DXMDExtract . .\extracted

    Example:
  3. Save and exit, and then rename the file to unpack.bat
  4. Double-click on it, and then the Command Prompt will appear and begin to unpack the game
  5. Once complete, the unpacked files should be in a subfolder within the runtime folder named extracted

Browsing the contents of the extracted files using DEMDDatabase

  1. Unpack the DEMDDatabase archive and run DEMD Database.exe
  2. File > Open > Open unpacked DB
  3. Navigate to the NameMap.txt file from the extracted folder created previously when unpacking with DXMDExtract
  4. After loading, the file trees are now displayed graphically and can be browsed, for example if you’re searching for conversational dialogue, you can expand the file list until you find what you need
Note: After selecting a NameMap, the software will automatically open that NameMap every time it starts, which becomes a problem if you want to load a different NameMap (either from the Runtime or DLC Runtime). To fix this, you need to either move the previous NameMap.txt to another folder or rename it to something like NameMap.old so the software can't open it anymore. It should then enable you to open a new unpacked DB. There is no way to merge the two NameMaps to generate a more complete file tree including both the base game and DLC content, unfortunately, so you'll have to switch between them when necessary.

Convert unsorted audio files with HxD

  1. Navigate to the runtime folder for either the main installation of Mankind Divided or the DLC runtime folder. Here the audio and video files appear to be unsorted and scattered. Audio files have the extension .pc_fsbm
  2. Open the file you want to convert with HxD and remove the first 24 bytes of information (the header), until it starts with FSB
  3. Save the file with HxD editor, this will automatically create a backup of the original .pc_fsbm file in the same folder
  4. Rename the file and change the file extension to .fsb
  5. As a .fsb file, it can now be converted to.wav using vgmstream or FSB Extractor - this is covered in detail in my other guide on extracting audio and video from Deus Ex: Human Revolution - the process is identical.

Extract unsorted audio files containing music layers with Game Extractor

A problem that can be encountered after extracting and browsing the file tree using DEMD DB is that there may be missing files. Files that NameMap.txt indicates should be in the path presented, i.e. Dubai music should be inside the Sound > Music part of the tree - but it doesn't show up.
The reason why this might be happening is that a lot of the game files are not actually located inside the main archive layers, but are just casually sitting around in the runtime folder.

The HxD method as demonstrated above may not work with most of the music ones because within the file are multiple layers of audio that will not magically downmix to one stereo audio file if we modify the header or file extension.

Luckily there is a tool available called Game Extractor that has recently been patched (version 3.16) to deal with these .pc_fsbm files more easily.
  1. Download and install Game Extractor, and run the tool.
  2. One open, you will see a file browser on the right panel; browse to your game runtime where all of the .pc_fsbm files are located. It may help to already copy all the .pc_fsbm files to a specific folder to work from if you don't want to clutter up the main folder.
  3. The browser should now list all the .pc_fsbm files. From this view you can double click one, and on the central panel, a new list will display of the file contents in .ogg format, which are music layers. With this information you will be able to find the missing files that were listed in the NameMap but not displayed in the DEMD DB file tree.
  4. You can extract the contents of that .pc_fsbm by clicking the icon at the top which looks like a floppy disk with a green arrow pointing at a file.
  5. After clicking the button, a section underneath the file browser on the right will appear, where you can input the extraction directory. To the right of that input field are two more buttons which look the same as before. The first one will extract the individual selected file from the archive only, and the second one will extract all files from the archive instead.
  6. Click the one you want, and the files should be saved inside the extraction directory you specified as .ogg files.
  7. Repeat the process for the other .pc_fsbm files you want to extract. As the music for some areas in the game are divided up into multiple layers (some more than others), you may have to import every layer and downmix them into one stereo file in an audio editing software like Audacity, similarly to how I demonstrated with the cutscene audio layers in this section

Extract specific audio (i.e. conversational dialogue) with DEMD Database

  1. For a more targeted approach, the NameMap.txt file that is generated from the DXMD Extraction comes in useful as it lists every file and where they can be found within the file tree
  2. Opening it with a text editor and using the find function can help narrow down the search, in this instance I was searching for a specific conversation, and in the .txt file it is called dlc_1_441_con. I was able to find this easily in the file tree
  3. It is possible to export the entire conversation in one go, but in order to do that the fast export savepath needs to be set first. To do this, go to File > Settings
  4. In the Settings window, you can set the Fast export savepath under the Fast export tab. Save to commit the changes
  5. Select the parent node for the conversation you want to export > File > Mass fast export
  6. A dialogue box will appear when the process is complete
  7. The audio files from the conversation have now been exported as .ogg can be found in the folder you added to the Fast export savepath

Fix corrupted .ogg audio files with QuickBMS

Some audio, upon extraction and conversion becomes corrupted, and sounds garbled and distorted when it is played. This appears to be some kind of problem with DEMD Database failing to properly convert the .bin files found within the .pc_headerlib files--which is where most the game audio is stored.

Why this happens is unclear, but it might have something to do with when the source file has multiple audio layers and the conversion process is unable to produce a file that contains this layer information, or is unable to produce a stereo mixdown of those layers (more on that at the end of this section).

To bypass this issue, we have to extract the audio a different way by using a QuickBMS script:

  1. Assuming you've already extracted the game files using the above instructions, and you have DEMD Database set up to explore the extracted content, you need to find the problematic audio files again in the file tree. You should have the names of the corrupted audio already from the failed conversion, so just search for the name.
  2. After you find it, check the right panel for more information. The first line should display a long string of numbers and letters, a forward slash and then another string of text followed by a .bin extension. The second line displays the name of the file. We need the first line, as these strings are pointing to a path to where the audio is actually located. The first section is the name of the .pc_headerlib file the audio was contained within, and the second section the name of the .bin file that contained the actual audio.
  3. In your game directory, go to the /extracted folder made earlier when extracting the game where the NameMap.txt is located (either the default runtime or DLC runtime, in this case I've extracted from the DLC). In this folder, you should be able to find a subfolder with a name that matches the first section of the path displayed in the panel. Within that folder, should be the .bin file with the audio. You can also use the search function and type the .bin names in manually to find the files.
  4. Depending on how many corrupted files you may need to repeat this process again, for me I had multiple corrupted audio within the same dialogue "conversation", so all the .bin files I needed were located within just one folder. I copied these .bin files to my Desktop so I could continue working outside of the game installation files.
  5. Next you need to create a script file for converting the .bin files into .fsb audio files. You can do this by opening up Notepad or any text editor of your choice, pasting this code in, and then saving it as a .bms file.
  6. Download and extract QuickBMS. In the QuickBMS folder, place the bin2fsb script and also make a subfolder called extraction.
  7. Run QuickBMS.exe - when prompted, select the script you created, the input files (.bin files), and the output destination (extraction subfolder) and continue. The progress will display for every file selected as well as any errors.
  8. Once done, various .fsb audio files should have been extracted in the output folder set.
  9. To convert the .fsb files to .wav you need to use vgmstream. Download and extract vgmstream on your computer, and either drag each of the .fsb files one-by-one onto vgmstream-cli.exe or create a batch script, making sure to set the path to match where vgmstream-cli.exe is located on your drive, like so:

    FOR /R %%a IN (*.fsb) DO "...\vgmstream-cli.exe" -l 2 -f 10 -o "%%a.wav" "%%a"

  10. To run the batch convert, first place all the .fsb files into the vgmstream folder, and then double-click the .bat file.
  11. The audio should now all be converted to playable .wav files in the same folder.
  12. This step is optional, but I found that it helped with organising, I renamed each of the .wav file to match the corresponding name in DEMD Database.
  13. I opened one of the .wav files in Audacity to compare it to the corrupted ones. The properly converted .wav files appear to have multiple layers of audio, the first two being the stereo channel for the voice, the second two appearing empty, and the third two with environmental effects such as reverb and echo. The corrupted ones are just one channel with various peaking blips and noise. Here are screenshots of the difference:

Convert unsorted video files (cinematics, loading screens) with HxD

  1. Before starting, you need RAD Video Tools (Bink Video Player) to play the videos after converting them. Download and install it, as you'll need it for the next section of this guide to convert the audio streams from the video.
  2. Navigate to the runtime folder for either the main installation of Mankind Divided or the DLC runtime folder. Here the audio and video files appear to be unsorted and scattered. Video files have the extension .pc_binkvid
  3. Open the file you want to convert with HxD and remove the first 24 bytes of information (the header), until it starts with KB2
  4. Save the file with HxD editor, this will automatically create a backup of the original .pc_binkvid file in the same folder
  5. Rename the file and change the file extension to .bk2 (the icon should change)
  6. The .bk2 files should now be able to play with RAD Video Tools (Bink Video Player)

    Note: Due to localisation, audio streams, usually the voices, will not play and need to be extracted and split if you want to access them, as covered in the next section.

Extract and split audio tracks from .bk2 files with VGMToolbox

  1. Open VGMToolbox > Misc. Tools > Stream Tools > Video Demultiplexer > Options > set the Format to BIK (Bink Video Container)  > check Split Audio Tracks > check Extract Audio and Video
  2. Drag and drop the .bk2 file you want to extract into the grey area and it will automatically run with the options set
  3. Multiple files will then be extracted into the same folder the .bk2 file was in. The extracted files will have the file extension video.split.bik or audio.split.bik
Note: In the EU version of the game, layer 00000004 contains the English voices, and layers 00000005, 00000006, 00000007, 00000008, 00000009, 0000000C, and 0000000D all have other European localisations such as Spanish, German, Russian, etc. Layer 00000000 seems to have the main background music and effects, 00000001, 00000002, and 00000003 have additional effects such as footsteps, etc.

Convert the audio streams using RAD Video Tools

  1. Open RAD Video Tools > navigate to and select every .audio.split.bik file you want to convert > Convert a File > Yes to Batch Process dialogue box if you are converting multiple files
  2. Within the Bink Converter window > change the Output Type to .wav > leave the other settings as default > click the Batch button on the right
  3. The RAD Batch Editor window will then appear > click Close (might have to click this several times until all the files appear in the Batched commands list) > Start
  4. The audio.split.bik files should now be converted to .wav files
Note: Some audio streams will not appear in the Batch command list file, and RAD will ignore them and fail to convert. They can be converted in an alternative way, covered in the next section.

Convert the audio streams using VLC Media Player

  1. Open VLC Media Player > Media > Convert/Save
  2. The Open Media window will open. Click the Add... button > select the .bik file you want > drop-down arrow to the right of Convert/Save > Convert
  3. The Convert window will open. Under Settings > Profile drop-down > Audio - FLAC > profile edition (wrench icon)
  4. The Profile edition window will open. Encapsulation tab > check WAV > Video codec tab > keep it unchecked (there is no video) > Audio codec tab > check > Codec - WAV, Bitrate - 320kb/s, Channels - 2, Sample Rate - 44100 or 48000 Hz > Save
  5. Back in the Convert window set the destination path by clicking Browse > select folder you want to save the converted file (make sure to change the file extension to .wav in the save dialog) > Start
  6. The .bik file should now be converted into a .wav file and can be found in your set destination folder

Convert the video stream using RAD Video Tools

  1. Open RAD Video Tools > navigate to and select the original .BK2 file you want to convert > Convert a File
  2. Within the Bink Converter window > change the Output Type to .MP4 > leave the other settings as default > make sure Audio is NOT selected > click the Convert button on the right
  3. A window will appear showing the progress of the conversion
  4. The .BK2 file should now be converted to an .MP4 file without audio

Combine the video stream and the audio streams using FFmpeg

Note: You can use FFmpeg (a command line interface program, instructions to install on Windows are here) to add map each audio stream as separate channels, but the problem with this is that a video player can only play one channel at a time. If all channels are to play, an amix filter would need to be applied that will combine all the channels into one track, for example:

ffmpeg -i INPUT1 -i INPUT2 -i INPUT3 -filter_complex amix=inputs=3:duration=first:dropout_transition=3 OUTPUT

But for the sake of simplicity, another way to do this without having to write up a complex command and enter all the file names is to import all of the audio streams (i.e. all the SFX ones and ONE voice one depending on the localisation you want) into Audacity, and then export all of them into one .wav file:
Then, with one video file and one audio file, you simply need to combine the two into one file using the following command:

ffmpeg -i 2B4301F00CBF6612436EB95C599AAFC3.mp4 -i 2B4301F00CBF6612436EB95C599AAFC3_00000000.audio.merge.wav -c:v copy -shortest output.mp4

To understand the command, the video and audio are set as inputs (filename will be different depending on which files you are doing this with), it will copy the same codec settings to avoid quality loss, it will cut to the shortest stream length to avoid overrunning if the lengths differ, and the output name will simply be output.mp4:
When played, the video remains with a very high quality and all the audio channels are audible in one audio track:

Results

Tools

at 13 comments:
Labels:
Subscribe to: Posts (Atom)