Skip to content

Commit

Permalink
Merge CP77Tools Solution (WolvenKit#311)
Browse files Browse the repository at this point in the history 
* Initial commit

* Create README.md

* better filenames

* .gitignore apply

* v0.1.0.1

- dump option
- fixes

* fix extraction and decompression

* version + oodle

* Update README.md

* Update README.md

* remove unneccesary files

* .gitignore is now working

* Hash finally found (WolvenKit#2)

- Better memory management (don't load all file in ram in unpacking process)
- Added optional extension for files
- Some fields renamed
- Added KARK detection
- Some improvements on files extracting
- Added SHA1Hash by Snazz#3248

* Multithreaded extracting + console progress bars

* Filetime

* Update README.md

* Update README.md

* extraction refactor

* Update README.md

* import tw3 filetypes

* version upgrade

* dump strings from archive

* actions

* derp

* more dump options

* more dump options. cleanup

* Github actions (WolvenKit#4)

* Github actions

* Cleaned

* multithreaded dump

* Update README.md

* whoops

* gitattributes

* Update OodleLZ.cs

* Update README.md

* Update OodleLZ.cs

* System.Commandline

- much better command line help and syntax

* extraction output directory

* extract with filenames

* log

* dump class layout

* Update CR2WExport.cs

- now also includes offset

* more CLI options

- more cli options and descriptions
- archive refactor

* write buffers instead of appending

* better buffer extraction

* cp77 cbitmap support

* texture extraction

* better extraction, texture types

* fix build

- version update

* more hashes

* fix cr2w info dumps

* more hashes

* update cr2w.dll from Wkit

* better json dumps

* version + readme

* better texture extraction

* fix typo

* better texture support

* add oodle tools

* Create Multilayer_Mask.cs

* fixes

* update readme

* fix dump task

* mltemplate support

- pretty json dump

* Local Commit

* Revert "Local Commit"

This reverts commit ace88a2.

* mltemplate mlsetup

* Build Instructions (WolvenKit#16)

* Create build.md

* Changed capitalization in file name to fit better with naming conventions

* dump class def option

* quit

* better hashing

* Readme hint (with line ending noise :peeporun;) (WolvenKit#19)

* add cp77 classes

- remove tw3  classes

* more classes

* uncooking all

* remove empty "solution items" filter

* small refactor

* speed up hash loading

* switch progress bar library, the output is not as detailed but it fixes WolvenKit#21

* fix output directory logic

* progressbar via ILoggerService

* experimental kraken

* wildcard and regex

* Update archivehashes.csv

* Multi-archive extraction (WolvenKit#24)

* refactor

* archive refactor

* archive small stuff

* Added outpath parameter to Rootcommands : CR2W & Oodle , if not given oupath = inputpath

* Prevent crash if CR2W not given input path.
Refactored Console.WriteLines

* multiple input paths

* gameuiCharacterCustomizationUiPresetValue

- gameuiCharacterCustomizationUiPresetValue
- some renames

* fix a bug with cr2w header

* OodleLZ : Using System was needed.
ArchiveTask : Replaced C.writelines with new logging system.
Program : Listen for Stringlogged
Cr2wtask : New logging
Dumptask : New logging
Hashtask : New logging

* packing

* fix cr2w

* Update Cr2wTask.cs

* RGB, fixed progress bars

* pack task

* pack more

* Added crc64 (WolvenKit#33)

* add crc64
fix toc 3 tables

* fix

* fix buffer archiving

* update version

* Update README.md

* Update README.md

* Update Archive.cs

- fix bug

* Utility command for pulling latest Archive Hashes (WolvenKit#35)

Co-authored-by: avail <[email protected]>

* Fix (WolvenKit#36)

* add crc64
fix toc 3 tables

* fix

* Fixed size in KARK
Fixed size, zsize in table 2
Fixed offsetidx

* packing bugfixes

- fixed a bug where uncooking would crash the console
- fixed a bug where the buffers would not get sorted by numeric value
- fixed a bug where the filelist in a packed archive was not sorted by hash

* Move Commands defintions into their own classes (WolvenKit#37)

* Moved Command definitions into their own classes

* Merged main

* fix packing non-cr2w files

* bugfixes

- fixed a bug with the archiveManager
- fixed a bug with generating missing hashes
- fixed a bug with loading hashes

* better hash handling

* Update Program.cs

- fixed a bug where the program would crash if no loose archive hashes were found

* checks

* refactoring IHashService

- removed Maincontroller in favor of IHashService
- refactored the IHashService and loading
- fix archive naming

* cr2wtask bugfix

- fixed a bug where cr2w -a and -c would crash

* some more enums

* more enums

* fix enums

* multi-input commands, unit test begin cr2w fixes

* csv uncooking

* more hashes, csv uncooking

* better hash service, logging

- moved hashService to cp77.cr2w
- fixed some UI issues with unpacking

* Update CP77Tools.csproj

- version update

* Update AppSettingsService.cs

fix path

* List object inside buffersDict was not thread-safe, resulting in occasional null value insertion. Removed parallel execution. (WolvenKit#47)

* add dumped classes

* enums

* fix missing classes

* FNV 1A 64-bit hash algorithm optimizations (WolvenKit#49)

* Streaming from zip-archive with hashes without extracting. (WolvenKit#50)

* namespace changes

WolvenKit.Cr2w -> CP77.Cr2w

* remove unused legacy classes

* Cross-platform support

- removed memory-mapped files

* Version update

* correct enums

* fix varname normalization

* normalize class types

* version updadte

* fix class property

* Added optional vertical flip when uncooking (WolvenKit#56)

* Added optional vertical flip when uncooking

* fixed typo and cleaned up texconwrapper

* Merge branch 'feature-buffer-extraction' into dev

- oodle decompression OSX support
- extracting archives concatenates buffers instead of splitting
- uncooking now extracts and decompresses buffers out of the cr2w
- fixed a bug with buffer extraction

* Update Archive.cs

- better cross-platform handling

* update name

* update branch id

* Revert "update name"

This reverts commit cad7040.

* Create nightly.yml

* Delete nightly.yml

* workflow dispatch

* packing

- added a rebuild option (to add split buffers back to a cr2w file)
- refactored mod operations (packing, importing etc)
- osx ooz instead of oodle

* Update .gitignore

* Update nightly.yml

* Update nightly.yml

* Update nightly.yml

* Update nightly.yml

* fix correct upload url

* Update nightly.yml

* Update README.md

* Update README.md

* unsafe dds rebuilding

- added the option to unsafely pack textures as buffers

* fix buffer rebuilding

* better logging

* Update README.md

* Update README.md

* bugfixes

- fix packing bugs
- fix rebuilding bugs
- fix extraction bugs
- refactor extraction
- refactor uncooking
- refactor decompression

* Cr2w unit tests (WolvenKit#57)

* Initial unit tests

* Post dev merge

* Fixed logging

* Output as CSV, Better Exceptions

* Flipped decompress flag

* Pre-merge tidying up

* Update README.md

* Update dotnet-core.yml

- don't create pre-release builds for main

* add clean step before restore

* Update README.md

* Version 1.1 (WolvenKit#67)

* some more enums

* Implemented extracting from missing hash list (WolvenKit#59)

* uncookTask

* MLmask uncook support (WolvenKit#60)

* Added mlmask uncook support

* fixed interpolation issue resulting in pixelart

* Update archivehashes.zip

* refactor unbundling

* add new commands, deprecate archive command

* Update Uncook.cs

- uncooking also extracts

* fixed vertical flip (WolvenKit#62)

* Update nightly.yml

- create beta release instead of appending to existing release

* fix restore bug

* version update

* Update Uncook.cs

- fixed a bug where uncooking would fail

* Uncook support for some extensions (WolvenKit#63)

* Uncook support for cubemap and envprobe

* Uncook support for texarray

* fixed WolvenKit#65

* add export command

* version update

Co-authored-by: Hambalkó Bence <[email protected]>
Co-authored-by: Kay Hennig <[email protected]>

* exclude reshaper stuff

* Stable "Uncooking" Version (v1.1) (WolvenKit#69)

* some more enums

* Implemented extracting from missing hash list (WolvenKit#59)

* uncookTask

* MLmask uncook support (WolvenKit#60)

* Added mlmask uncook support

* fixed interpolation issue resulting in pixelart

* Update archivehashes.zip

* refactor unbundling

* add new commands, deprecate archive command

* Update Uncook.cs

- uncooking also extracts

* fixed vertical flip (WolvenKit#62)

* Update nightly.yml

- create beta release instead of appending to existing release

* fix restore bug

* version update

* Update Uncook.cs

- fixed a bug where uncooking would fail

* Uncook support for some extensions (WolvenKit#63)

* Uncook support for cubemap and envprobe

* Uncook support for texarray

* fixed WolvenKit#65

* add export command

* version update

* rm remote projs

* use nugets

- use wolvenkit nugets
- retarget to net5.0-windows

* remove remote files

* remove archive command

* fix nuget dependencies

* unit test base class

* Update Nugets

* fix dependency copying

Co-authored-by: Hambalkó Bence <[email protected]>
Co-authored-by: Kay Hennig <[email protected]>

* fix retarget

* update nugets

* bugfixes and QoL

- fixed a bug where uncooking would not decompress buffers
- version update
- automatically look for oodle lib when starting

* Create CHANGELOG.txt

* Update README.md

* Update Nugets

- fixed some bugs with packing

* add cp77tools.sln

Co-authored-by: Eugene Pogrebnyak <[email protected]>
Co-authored-by: R503B <[email protected]>
Co-authored-by: Martin Winkelmann <[email protected]>
Co-authored-by: Maxime Chambonnet <[email protected]>
Co-authored-by: avail <[email protected]>
Co-authored-by: snazk <[email protected]>
Co-authored-by: phrisk <[email protected]>
Co-authored-by: avail <[email protected]>
Co-authored-by: Aaron Shoemaker <[email protected]>
Co-authored-by: waitxd <[email protected]>
Co-authored-by: Kay Hennig <[email protected]>
Co-authored-by: Hambalkó Bence <[email protected]>
  • Loading branch information
@rfuzzo @emcifuntik
@Offline-R503B @Maxzor @avail @eprilx @phrisk @aaronmshoemaker @waitxd @TheFusion21 @Traderain
13 people authored Jan 29, 2021
1 parent 92b5227 commit 330e3be
Show file tree
Hide file tree
Showing 2 changed files with 141 additions and 11 deletions.
30 changes: 30 additions & 0 deletions CP77Tools.sln
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,20 +7,50 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "CP77Tools", "CP77Tools\CP77
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "CP77.MSTests", "CP77.MSTests\CP77.MSTests.csproj", "{78302151-8D4B-407B-A2FE-5774C471F8AA}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "WolvenKit.Cyberformats", "CP77.CR2W\WolvenKit.Cyberformats.csproj", "{A612A200-90D5-41A7-9ADE-F90AD2274022}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "WolvenKit.Common", "WolvenKit.Common\WolvenKit.Common.csproj", "{2B13DF78-8441-49F3-8212-06935A5D7AE0}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Debug|x64 = Debug|x64
Release|Any CPU = Release|Any CPU
Release|x64 = Release|x64
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{37C3288C-A6D7-4C71-8174-0F2F6D47938E}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{37C3288C-A6D7-4C71-8174-0F2F6D47938E}.Debug|Any CPU.Build.0 = Debug|Any CPU
{37C3288C-A6D7-4C71-8174-0F2F6D47938E}.Debug|x64.ActiveCfg = Debug|Any CPU
{37C3288C-A6D7-4C71-8174-0F2F6D47938E}.Debug|x64.Build.0 = Debug|Any CPU
{37C3288C-A6D7-4C71-8174-0F2F6D47938E}.Release|Any CPU.ActiveCfg = Release|Any CPU
{37C3288C-A6D7-4C71-8174-0F2F6D47938E}.Release|Any CPU.Build.0 = Release|Any CPU
{37C3288C-A6D7-4C71-8174-0F2F6D47938E}.Release|x64.ActiveCfg = Release|Any CPU
{37C3288C-A6D7-4C71-8174-0F2F6D47938E}.Release|x64.Build.0 = Release|Any CPU
{78302151-8D4B-407B-A2FE-5774C471F8AA}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{78302151-8D4B-407B-A2FE-5774C471F8AA}.Debug|Any CPU.Build.0 = Debug|Any CPU
{78302151-8D4B-407B-A2FE-5774C471F8AA}.Debug|x64.ActiveCfg = Debug|Any CPU
{78302151-8D4B-407B-A2FE-5774C471F8AA}.Debug|x64.Build.0 = Debug|Any CPU
{78302151-8D4B-407B-A2FE-5774C471F8AA}.Release|Any CPU.ActiveCfg = Release|Any CPU
{78302151-8D4B-407B-A2FE-5774C471F8AA}.Release|Any CPU.Build.0 = Release|Any CPU
{78302151-8D4B-407B-A2FE-5774C471F8AA}.Release|x64.ActiveCfg = Release|Any CPU
{78302151-8D4B-407B-A2FE-5774C471F8AA}.Release|x64.Build.0 = Release|Any CPU
{A612A200-90D5-41A7-9ADE-F90AD2274022}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{A612A200-90D5-41A7-9ADE-F90AD2274022}.Debug|Any CPU.Build.0 = Debug|Any CPU
{A612A200-90D5-41A7-9ADE-F90AD2274022}.Debug|x64.ActiveCfg = Debug|Any CPU
{A612A200-90D5-41A7-9ADE-F90AD2274022}.Debug|x64.Build.0 = Debug|Any CPU
{A612A200-90D5-41A7-9ADE-F90AD2274022}.Release|Any CPU.ActiveCfg = Release|Any CPU
{A612A200-90D5-41A7-9ADE-F90AD2274022}.Release|Any CPU.Build.0 = Release|Any CPU
{A612A200-90D5-41A7-9ADE-F90AD2274022}.Release|x64.ActiveCfg = Release|Any CPU
{A612A200-90D5-41A7-9ADE-F90AD2274022}.Release|x64.Build.0 = Release|Any CPU
{2B13DF78-8441-49F3-8212-06935A5D7AE0}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{2B13DF78-8441-49F3-8212-06935A5D7AE0}.Debug|Any CPU.Build.0 = Debug|Any CPU
{2B13DF78-8441-49F3-8212-06935A5D7AE0}.Debug|x64.ActiveCfg = Debug|x64
{2B13DF78-8441-49F3-8212-06935A5D7AE0}.Debug|x64.Build.0 = Debug|x64
{2B13DF78-8441-49F3-8212-06935A5D7AE0}.Release|Any CPU.ActiveCfg = Release|Any CPU
{2B13DF78-8441-49F3-8212-06935A5D7AE0}.Release|Any CPU.Build.0 = Release|Any CPU
{2B13DF78-8441-49F3-8212-06935A5D7AE0}.Release|x64.ActiveCfg = Release|x64
{2B13DF78-8441-49F3-8212-06935A5D7AE0}.Release|x64.Build.0 = Release|x64
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand Down
122 changes: 111 additions & 11 deletions CP77Tools/Program.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,14 @@
using Catel.IoC;
using System.CommandLine;
using System.ComponentModel;
using System.Configuration;
using System.IO;
using System.Reflection;
using CP77Tools.Commands;
using CP77Tools.Extensions;
using Luna.ConsoleProgressBar;
using Microsoft.Win32;
using WolvenKit.Common.Oodle;
using WolvenKit.Common.Services;

namespace CP77Tools
Expand All @@ -18,16 +21,6 @@ class Program
[STAThread]
public static async Task Main(string[] args)
{
//TODO: check here for linking errors
// try
// {
// Marshal.PrelinkAll(typeof(OozNative));
// }
// catch (Exception e)
// {
// Console.WriteLine(e);
// }

ServiceLocator.Default.RegisterType<ILoggerService, LoggerService>();
ServiceLocator.Default.RegisterType<IHashService, HashService>();

Expand Down Expand Up @@ -75,7 +68,15 @@ public static async Task Main(string[] args)
};

//await ConsoleFunctions.UpdateHashesAsync();
/*await*/ hashService.ReloadLocally();
hashService.ReloadLocally();

// try get oodle dll from game
if (!TryCopyOodleLib())
{
logger.LogString("Could not automatically find oo2ext_7_win64.dll. " +
"Please manually copy and paste the dll found here Cyberpunk 2077\\bin\\x64\\oo2ext_7_win64.dll into this folder: " +
$"{AppDomain.CurrentDomain.BaseDirectory}.");
}

// Run
if (args == null || args.Length == 0)
Expand Down Expand Up @@ -164,5 +165,104 @@ async Task WriteLog()
}

}

private delegate void StrDelegate(string value);

private static string TryGetGameInstallDir()
{
var cp77BinDir = "";
var cp77exe = "";
// check for CP77_DIR environment variable first
var CP77_DIR = System.Environment.GetEnvironmentVariable("CP77_DIR", EnvironmentVariableTarget.User);
if (!string.IsNullOrEmpty(CP77_DIR) && new DirectoryInfo(CP77_DIR).Exists)
cp77BinDir = Path.Combine(CP77_DIR, "bin", "x64");
if (File.Exists(Path.Combine(cp77BinDir, "Cyberpunk2077.exe")))
return cp77BinDir;

// else: look for install location
const string uninstallkey = "SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\";
const string uninstallkey2 = "SOFTWARE\\Wow6432Node\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\";
const string gameName = "Cyberpunk 2077";
const string exeName = "Cyberpunk2077.exe";
var exePath = "";
StrDelegate strDelegate = msg => cp77exe = msg;

try
{
Parallel.ForEach(Registry.LocalMachine.OpenSubKey(uninstallkey)?.GetSubKeyNames(), item =>
{
var programName = Registry.LocalMachine.OpenSubKey(uninstallkey + item)
?.GetValue("DisplayName");
var installLocation = Registry.LocalMachine.OpenSubKey(uninstallkey + item)
?.GetValue("InstallLocation");
if (programName != null && installLocation != null)
{
if (programName.ToString().Contains(gameName) ||
programName.ToString().Contains(gameName))
{
exePath = Directory.GetFiles(installLocation.ToString(), exeName,
SearchOption.AllDirectories).First();
}
}

strDelegate.Invoke(exePath);
});
Parallel.ForEach(Registry.LocalMachine.OpenSubKey(uninstallkey2)?.GetSubKeyNames(), item =>
{
var programName = Registry.LocalMachine.OpenSubKey(uninstallkey2 + item)
?.GetValue("DisplayName");
var installLocation = Registry.LocalMachine.OpenSubKey(uninstallkey2 + item)
?.GetValue("InstallLocation");
if (programName != null && installLocation != null)
{
if (programName.ToString().Contains(gameName) ||
programName.ToString().Contains(gameName))
{
if (Directory.Exists(installLocation.ToString()))
exePath = Directory.GetFiles(installLocation.ToString(), exeName,
SearchOption.AllDirectories).First();
}
}

strDelegate.Invoke(exePath);
});

if (File.Exists(cp77exe))
cp77BinDir = new FileInfo(cp77exe).Directory.FullName;
}
catch (Exception e)
{

}

if (string.IsNullOrEmpty(cp77BinDir))
return null;
if (!File.Exists(Path.Combine(cp77BinDir, "Cyberpunk2077.exe")))
return null;

return cp77BinDir;
}

private static bool TryCopyOodleLib()
{
var ass = AppDomain.CurrentDomain.BaseDirectory;
var destFileName = Path.Combine(ass, "oo2ext_7_win64.dll");
if (File.Exists(destFileName))
return true;

var cp77BinDir = TryGetGameInstallDir();
if (string.IsNullOrEmpty(cp77BinDir))
return false;

// copy oodle dll
var oodleInfo = new FileInfo(Path.Combine(cp77BinDir, "oo2ext_7_win64.dll"));
if (!oodleInfo.Exists)
return false;

if (!File.Exists(destFileName))
oodleInfo.CopyTo(destFileName);

return true;
}
}
}

0 comments on commit 330e3be

Please sign in to comment.