Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to Trees 2.0 #826

Merged
merged 7 commits into from
Jan 11, 2024
Merged

Migrate to Trees 2.0 #826

merged 7 commits into from
Jan 11, 2024

Conversation

Sewer56
Copy link
Member

@Sewer56 Sewer56 commented Jan 8, 2024

This migrates our existing tree code to the new, efficient 'create your own tree' system.

General Changes

  • Replaced some usages of LINQ over trees with source generation.
  • Adds custom tree (ModFileTree) containing only the tree info needed by our DataModel.
  • Fixes some incorrect asserts related to use of .AllSatisfy throughout our codebase.
    • After fixing these asserts, it turns out we had broken code, even on main branch.
    • So I fixed any failing tests related to that.
  • Cleaned up some dead tree code that was unused.

Observations

  • Total lines of code in game extensions has noticeably decreased with the new APIs.
    • Note total LoC in this project increased because we now have the tree construction here instead of paths; however excluding that file, we observe around -150 LoC in this PR.
  • Each tree node now fits under a single x86 cache line (64 bytes), providing significant speedup on x86.

Other Notes

There are still opportunities for perf gains, for instance the GetFiles().Select pattern in some places can be replaced with GetChildrenRecursive with custom TFilter and TSelector, at expense of a few extra LoC. For now I've refrained from rewriting those to keep the code simple, but if additional speedup is required, the possibilities are still there.

I just didn't consider them to be significant enough perf improvements to be worth implementing in place of readability; as the impact on UX would be negligible. (Gains are limited by IEnumerable return type on ModInstallerResult )

Reviewer

@halgari - Self requested during meeting.

@Sewer56 Sewer56 requested a review from halgari January 8, 2024 17:40
@Sewer56 Sewer56 added the meta-improvement An issue that improves an existing feature label Jan 8, 2024
@Sewer56 Sewer56 self-assigned this Jan 8, 2024
@Sewer56
Copy link
Member Author

Sewer56 commented Jan 8, 2024

Note that CI might fail due to CanInstallAndApplyMostPopularMods for SkyrimLE. This is broken on main too, the failure is unrelated to this PR.

@codecov-commenter
Copy link

Codecov Report

Attention: 46 lines in your changes are missing coverage. Please review.

Comparison is base (ee141e3) 61.53% compared to head (ec24008) 61.59%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #826      +/-   ##
==========================================
+ Coverage   61.53%   61.59%   +0.05%     
==========================================
  Files         618      618              
  Lines       19083    19054      -29     
  Branches     1396     1406      +10     
==========================================
- Hits        11743    11736       -7     
+ Misses       6994     6977      -17     
+ Partials      346      341       -5     
Flag Coverage Δ
Linux 60.97% <83.91%> (+0.06%) ⬆️
Windows 60.84% <83.91%> (+0.04%) ⬆️
clean_environment_tests 61.58% <83.91%> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...ames.AdvancedInstaller.UI/Content/BodyViewModel.cs 69.12% <100.00%> (ø)
....UI/Content/Left/ModContent/ModContentViewModel.cs 87.50% <100.00%> (-0.18%) ⬇️
...Games.AdvancedInstaller/AdvancedManualInstaller.cs 0.00% <ø> (ø)
...exusMods.Games.AdvancedInstaller/DeploymentData.cs 97.43% <100.00%> (ø)
...ames.AdvancedInstaller/DeploymentDataExtensions.cs 100.00% <100.00%> (ø)
...dSorcery/Installers/BladeAndSorceryModInstaller.cs 92.45% <100.00%> (+1.22%) ⬆️
...arkestDungeon/Installers/LooseFilesModInstaller.cs 100.00% <100.00%> (ø)
...es.DarkestDungeon/Installers/NativeModInstaller.cs 87.50% <100.00%> (-0.84%) ⬇️
src/Games/NexusMods.Games.FOMOD/FomodAnalyzer.cs 92.06% <100.00%> (-4.82%) ⬇️
...c/Games/NexusMods.Games.FOMOD/FomodXmlInstaller.cs 81.72% <100.00%> (+1.72%) ⬆️
... and 29 more

... and 8 files with indirect coverage changes

@@ -70,7 +71,7 @@ public AdvancedManualInstallerUI(IServiceProvider provider)

private async Task<(bool shouldInstall, DeploymentData data)> GetDeploymentDataAsync(
GameInstallation gameInstallation, string modName,
FileTreeNode<RelativePath, ModSourceFileEntry> archiveFiles)
KeyedBox<RelativePath, ModFileTree> archiveFiles)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just had a quick look through,
since we use KeyedBox<RelativePath, ModFileTree> a lot, can we make a type alias for it that is shorter and either has Node or Tree in the name?

I think something simple like RelativeModFileTree works. To improve readability and shorten it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this, I had to dig through the path library to find out what KeyedBox was, it turns out it's a wrapper for TVal with an additional TKey typed tag, the Key isn't passed along with the box, just the type of the Key. We might want to find a better name for these in the mainline parts of the application.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm open to suggestions personally.
Heck, even ModFileTreeNode works for me.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go ahead with ModFileTreeNode. We can change it later if we come up with a better name.

@halgari
Copy link
Collaborator

halgari commented Jan 8, 2024

Aside from the KeyedBox comment this all looks solid

@halgari halgari merged commit 8ccc70a into main Jan 11, 2024
5 checks passed
@halgari halgari deleted the new-trees branch January 11, 2024 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
meta-improvement An issue that improves an existing feature
4 participants