Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] vulkan.hpp accidentally included in sources/third_party/vulkan, very broken #1974

Closed
sstarosz opened this issue Nov 26, 2023 · 17 comments
Labels

Comments

@sstarosz
Copy link

Description

I updated the NDK version of my project from version 25.2.9519653 to 26.1.10909125, after which my Vulkan project stopped compiling due to errors in with repeated definitions in the vulkan.hpp header shipped with NDK

(26.1.10909125\sources\third_party\vulkan\src\include\vulkan\)

Version 26.1.10909125 has a lot of repeated definitions in the vulkan.hpp and vulkan_enums.hpp headers. At least from these files come most of the compilation errors, I don't know if other files in the vulkan folder are not also affected.

Compilation output
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan.hpp:1161:10: error: class member cannot be redeclared
    void link( void * dstBase, void const * srcBase, VkBaseOutStructure * dst, VkBaseInStructure const * src )
         ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan.hpp:1149:10: note: previous definition is here
    void link( void * dstBase, void const * srcBase, VkBaseOutStructure * dst, VkBaseInStructure const * src )
         ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan.hpp:1342:9: error: redefinition of 'DispatchLoaderBase'
  class DispatchLoaderBase
        ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan.hpp:1318:9: note: previous definition is here
  class DispatchLoaderBase
        ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan.hpp:5881:7: error: class member cannot be redeclared
      vkSetDeviceMemoryPriorityEXT( VkDevice device, VkDeviceMemory memory, float priority ) const VULKAN_HPP_NOEXCEPT
      ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan.hpp:5552:10: note: previous definition is here
    void vkSetDeviceMemoryPriorityEXT( VkDevice device, VkDeviceMemory memory, float priority ) const VULKAN_HPP_NOEXCEPT
         ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan.hpp:5888:10: error: class member cannot be redeclared
    void vkGetDeviceBufferMemoryRequirementsKHR( VkDevice                                 device,
         ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan.hpp:5559:10: note: previous definition is here
    void vkGetDeviceBufferMemoryRequirementsKHR( VkDevice                                 device,
         ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan.hpp:5895:10: error: class member cannot be redeclared
    void vkGetDeviceImageMemoryRequirementsKHR( VkDevice                                device,
         ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan.hpp:5566:10: note: previous definition is here
    void vkGetDeviceImageMemoryRequirementsKHR( VkDevice                                device,
         ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan.hpp:5902:10: error: class member cannot be redeclared
    void vkGetDeviceImageSparseMemoryRequirementsKHR(
         ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan.hpp:5573:10: note: previous definition is here
    void vkGetDeviceImageSparseMemoryRequirementsKHR( VkDevice                                device,
         ^
In file included from C:/Users/Sebastian/AndroidStudioProjects/AndroidVulkanDemo/Renderer/Source/StRenderer/Renderer.cpp:1:
In file included from C:/Users/Sebastian/AndroidStudioProjects/AndroidVulkanDemo/Renderer/Include/StRenderer/Renderer.hpp:8:
In file included from C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan.hpp:6176:
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:1841:14: error: redefinition of 'MemoryMapFlagBits'
  enum class MemoryMapFlagBits : VkMemoryMapFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:1828:14: note: previous definition is here
  enum class MemoryMapFlagBits : VkMemoryMapFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:1941:14: error: redefinition of 'SemaphoreCreateFlagBits'
  enum class SemaphoreCreateFlagBits : VkSemaphoreCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:1928:14: note: previous definition is here
  enum class SemaphoreCreateFlagBits : VkSemaphoreCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2061:14: error: enumeration redeclared with different underlying type 'int' (was 'VkQueryPoolCreateFlags' (aka 'unsigned int'))
  enum class QueryPoolCreateFlagBits
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2048:14: note: previous declaration is here
  enum class QueryPoolCreateFlagBits : VkQueryPoolCreateFlags
             ^                         ~~~~~~~~~~~~~~~~~~~~~~
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2171:14: error: redefinition of 'BufferViewCreateFlagBits'
  enum class BufferViewCreateFlagBits : VkBufferViewCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2158:14: note: previous definition is here
  enum class BufferViewCreateFlagBits : VkBufferViewCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2772:14: error: redefinition of 'PipelineDynamicStateCreateFlagBits'
  enum class PipelineDynamicStateCreateFlagBits : VkPipelineDynamicStateCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2681:14: note: previous definition is here
  enum class PipelineDynamicStateCreateFlagBits : VkPipelineDynamicStateCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2781:14: error: redefinition of 'PipelineInputAssemblyStateCreateFlagBits'
  enum class PipelineInputAssemblyStateCreateFlagBits : VkPipelineInputAssemblyStateCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2694:14: note: previous definition is here
  enum class PipelineInputAssemblyStateCreateFlagBits : VkPipelineInputAssemblyStateCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2799:14: error: redefinition of 'PipelineMultisampleStateCreateFlagBits'
  enum class PipelineMultisampleStateCreateFlagBits : VkPipelineMultisampleStateCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2707:14: note: previous definition is here
  enum class PipelineMultisampleStateCreateFlagBits : VkPipelineMultisampleStateCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2808:14: error: redefinition of 'PipelineRasterizationStateCreateFlagBits'
  enum class PipelineRasterizationStateCreateFlagBits : VkPipelineRasterizationStateCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2720:14: note: previous definition is here
  enum class PipelineRasterizationStateCreateFlagBits : VkPipelineRasterizationStateCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2817:14: error: redefinition of 'PipelineTessellationStateCreateFlagBits'
  enum class PipelineTessellationStateCreateFlagBits : VkPipelineTessellationStateCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2733:14: note: previous definition is here
  enum class PipelineTessellationStateCreateFlagBits : VkPipelineTessellationStateCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2826:14: error: redefinition of 'PipelineVertexInputStateCreateFlagBits'
  enum class PipelineVertexInputStateCreateFlagBits : VkPipelineVertexInputStateCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2746:14: note: previous definition is here
  enum class PipelineVertexInputStateCreateFlagBits : VkPipelineVertexInputStateCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2835:14: error: redefinition of 'PipelineViewportStateCreateFlagBits'
  enum class PipelineViewportStateCreateFlagBits : VkPipelineViewportStateCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2759:14: note: previous definition is here
  enum class PipelineViewportStateCreateFlagBits : VkPipelineViewportStateCreateFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2978:14: error: redefinition of 'DescriptorPoolResetFlagBits'
  enum class DescriptorPoolResetFlagBits : VkDescriptorPoolResetFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:2965:14: note: previous definition is here
  enum class DescriptorPoolResetFlagBits : VkDescriptorPoolResetFlags
             ^
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:3354:14: error: redefinition of 'CommandPoolTrimFlagBits'
  enum class CommandPoolTrimFlagBits : VkCommandPoolTrimFlags
             ^
he command: "C:\Program Files\CMake\bin\cmake.EXE" --build C:/Users/Sebastian/AndroidStudioProjects/AndroidVulkanDemo/out/build/Android-armeabi-v7a-Debug --config Debug --target StRenderer exited with code: 1
C:/Users/Sebastian/AppData/Local/Android/Sdk/ndk/26.1.10909125/sources/third_party/vulkan/src/include/vulkan/vulkan_enums.hpp:3340:14: note: previous definition is here
  enum class CommandPoolTrimFlagBits : VkCommandPoolTrimFlags
             ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.

Some errors look like typical merge mistakes, because the function code itself is the same, only the formatting is different:

Vulkan.hpp:
image

image

Vulkan_enums.hpp:
image

Affected versions

r26

Canary version

No response

Host OS

Windows

Host OS version

19045.3693

Affected ABIs

armeabi-v7a, arm64-v8a, x86, x86_64

Build system

CMake

Other build system

No response

minSdkVersion

24

Device API level

No response

@sstarosz sstarosz added the bug label Nov 26, 2023
@DanAlbert
Copy link
Member

@cnorthrop

I'm not actually sure what that header is doing there. That directory I thought was just for the validation layers, which are no longer bundled with the NDK itself (#1374). As best as I can tell this was only ever included because the validation layers needed it to build?

You probably don't want the vulkan headers from that directory. You want the one from the sysroot, which is automatically on the #include search path. vulkan.hpp is not something the graphics team currently supports. #1767 tracks that, but it's not actually something in my power to fix. That needs to come from the OS (it needs to exactly match the version of vulkan.h that ships with the OS headers).

The one time I tried using vulkan.hpp with the NDK, I did that by checking the vulkan version in the NDK (VK_HEADER_VERSION in $NDK/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/include/vulkan/vulkan_core.h), downloaded the matching version from https://github.com/KhronosGroup/Vulkan-Hpp/releases, and vendored that into my source tree. It was annoying that it wasn't included, but it worked fine (though tbh I actually found it easier to work with the C API since the hpp version was very poorly documented at the time, and the conversion between the two was not always obvious).

I suspect the actual fix you want here is "please support vulkan.hpp", which is that other bug I linked, and that in at least the short term we should be less confusing and delete the directory with the stale headers?

@cnorthrop
Copy link

I looks like there was a merge error at some point. @trevordblack, can you take a look? I can't find the bad merge, but you can see the end result by looking at the history here:

https://cs.android.com/android/platform/superproject/+/master:external/vulkan-headers/include/vulkan/vulkan.hpp;l=1161;bpv=1

@DanAlbert
Copy link
Member

Discussed more with cnorthrop over IM, and it sounds like my guess is probably correct: these were part of the validation layer source and were accidentally left behind when the rest of the validation layers were removed. We'll drop the headers from that location in r27 since those are obviously misleading.

I'd very much prefer to do that alongside vulkan.hpp being added to the sysroot though. We may not have meant to ship that, but if people had found it and were using it, that'd still look like a feature regression :(

Within r26, if @trevordblack can resolve the bad merge, we can probably cherry-pick the fix. If the main branch has moved forward since r26 that might be difficult. Alternatively we could just accept that the band-aid has been ripped off and leave it broken since we're going to remove it anyway.

@DanAlbert DanAlbert changed the title [BUG] VulkanHpp headers shipped with NDK containe duplicated function definitions. Nov 28, 2023
@sstarosz
Copy link
Author

sstarosz commented Nov 28, 2023

I wasn't sure about Vulkan-Hpp support, because on the one side there is a issue whether to support Vulkan-Hpp, on the other side you can find updated Vulkan-Hpp in NDK. So far, I've been using the headers from this folder because it was the easiest way for me to add Vulkan-Hpp to my project, but if you wants to remove it, I don't mind.

If it's not supported ofically, I agree that it should be removed because, some people probably use these headers.....

I managed to find a way to add Vulkan-Hpp to my Android project using vcpkg and it seems to work for me
Vcpkg: Using Vulkan SDK

Thank you for clarifying the situation

@DanAlbert
Copy link
Member

DanAlbert commented Nov 28, 2023

Ah, I didn't realize that was in vcpkg. Yeah, that's probably your best bet until we can support it officially. You may need to be cautious about matching the version to what's in the NDK sysroot as before though.

vcpkg does support AAR generation, which might be easier to integrate than the normal vcpkg output depending on what your build looks like (hard if you're not using gradle, but definitely easier to keep builds reproducible across your team if you are): https://learn.microsoft.com/en-us/vcpkg/users/platforms/android#consume-libraries-using-vcpkg-and-android-prefab-archives-aar-files

@trevordblack
Copy link

"Ugh. who was the idiot that caused the merge error".
Checks blame...
"Well I mean anyone could have made this mistake"
yeesh.

It looks like external/vulkan-headers is on 1.3.237

From my understanding there's a couple of actions

  • Fix the 3 duplicates outlined in [BUG] vulkan.hpp accidentally included in sources/third_party/vulkan, very broken #1974 (comment) and hope that fixes it. But, where the fix lands is problematic. I can land the fix in external/vulkan-headers but that's problematic for a few reasons. If I "just" fix the duplicates then that will make downstreaming from the vulkan headers problematic the next time we do that. If we try to downstream the vulkan headers first and then fix the duplicates, well... downstreaming is a not-painless process and there is more than a few reasons we're still on 1.3.237. It's honestly better if this can just be band-aided in the NDK repo. Not sure where the right place for that fix is.
  • I can compare external/vulkan-headers to https://github.com/KhronosGroup/Vulkan-Headers/tree/main/include/vulkan and see if anything else looks suspect

Honestly the "fix" here is the painful one (downstream from vulkan-headers to external/vulkan-headers), and I don't know when I'm going to find the time to do that... January?

If there's a 5 minute band-aid NDK thing to do, I'm happy to do that soon....

Hmm. I guess, is the explicit ask for me (as person with OS changing power) to just remove the duplicates above in external/vulkan-headers? It'll be annoying later on, but later-on Trevor has a lot more bandwidth than today-now Trevor.

@DanAlbert
Copy link
Member

DanAlbert commented Dec 4, 2023

"Ugh. who was the idiot that caused the merge error".
Checks blame...
"Well I mean anyone could have made this mistake"
yeesh.

Been there :)

It's honestly better if this can just be band-aided in the NDK repo. Not sure where the right place for that fix is.

We ship directly from external/vulkan-headers. A band aid in the NDK repo would be a band aid in external/vulkan-headers. If we want to fix this in a point release of r26c, we could do that in the aosp/ndk-r26-release branch of that project, which wouldn't auto merge anywhere. That's an option, but since we didn't support this on purpose, and the OP sounds happy just using vcpkg until we do, we may not need to.

and I don't know when I'm going to find the time to do that... January?

We're not shipping anything until January anyway.

Hmm. I guess, is the explicit ask for me (as person with OS changing power) to just remove the duplicates above in external/vulkan-headers? It'll be annoying later on, but later-on Trevor has a lot more bandwidth than today-now Trevor.

The best thing would be to fix external/vulkan-headers when you're able, but also update https://cs.android.com/android/platform/superproject/main/+/main:external/vulkan-headers/Android.bp;l=51-56;drc=1783d5ae806ca53d8f180dc03f470712527072eb to include this header, which would let us close #1767. You may also want to add a test that does nothing but #include <vulkan.hpp> to make sure this doesn't regress :)

A fix for a future release of the NDK has even more time than until January, btw. I'd only need a fix by late Feb for that?

@trevordblack
Copy link

OK

I'll try to get to this late December or early January. I'd rather not push it to the end of the month.

@DanAlbert
Copy link
Member

@trevordblack we're going to be cutting a release in the near future. I'm going to drop the headers that are being installed to <NDK>/sources/third_party since those weren't ever supposed to be there in the first place, but it'd be nice if we could pick up a new sysroot that has a proper vulkan.hpp at the same time :)

@DanAlbert
Copy link
Member

https://android-review.googlesource.com/c/platform/ndk/+/2924824 removes the broken and misleading stuff. Unclear whether or not we're going to be able to fix #1767 in r27, but it doesn't sound too likely right now.

@trevordblack
Copy link

We're working on fixing the broken build and should have that done within a week or two.
When it lands into the NDK is more Dan's department.

I suspect the actual fix you want here is "please support vulkan.hpp", which is that other bug I linked, and that in at least the short term we should be less confusing and delete the directory with the stale headers?

After internal discussion we're not in a position to support the .hpp files, and so aren't going to be explicitly adding them

@DanAlbert
Copy link
Member

We're working on fixing the broken build and should have that done within a week or two.
When it lands into the NDK is more Dan's department.

It'll go out in r27 beta 1 if that's the case 👍

@timprepscius
Copy link

For future readers of this thread. Maybe it's fixed in r27. But maybe you can't use that version for whatever reason.

If you go to the official VulkanHeaders on github, go through the release tags to match the one google is using. I think it's 237 at the moment, not sure of new release.

You can get get that repository-tag and then set up a directory with all of the ndk's vulkan/.h and the VulkanHeaders/vulkab/.hpp symbolically linked in, and it seems to work.

@DanAlbert
Copy link
Member

The much better fix is to use vcpkg like sstarosz suggested above. The NDK's vulkan.hpp was only ever shipped by accident and thus never tested.

@timprepscius
Copy link

Possibly so, I would rather not. Package managers are great until they aren't.

@DanAlbert
Copy link
Member

In that case you're still better off getting it from somewhere else, like the official repo: https://github.com/KhronosGroup/Vulkan-Hpp. Just make sure you download the version that's matched to the vulkan.h in whatever NDK you're using.

@timprepscius

This comment was marked as off-topic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 participants