When Quarkus meets Virtual Threads

[giscus[bot]]

When Quarkus meets Virtual Threads

Quarkus: Supersonic Subatomic Java

https://quarkus.io/blog/virtual-thread-1/

[dorianm]

Thanks for this good introduction post. Simple (maybe stupid) question, what is the future of Smallrye Mutiny with Virtual Threads in Quarkus ? Is it still relevant with them ? Are Vert.X reactive drivers (for DB/Redis/...) still relevant too ?

[cescoffier]

That's a great question and a topic we will address in another post.
All our reactive clients do not pin, and the Mutiny await() feature has been developed not to pin. So, you can use every reactive client we propose and use await() from a virtual thread. It's safe (in the sense it does not pin) and simplifies the code as you do not have to use continuation (.map, .flatMap...)

Also, virtual threads are great for "one-shot" methods, represented in Mutiny using Uni; for streams, it's trickier. for loops do not have the same semantics (as well as blocking queues). I have been working on an API to bridge stream and virtual threads, but it's a WIP. Thus, Multi is still very relevant.

[dorianm]

Thank you for this explanation ! So the Mutiny and the Multi API won't go away soon. If your WIP API to bridge stream and virtual threads is open-source, I'll be interested to look at it if you don't mind.

[cescoffier]

Even Uni won't go away. There are many things you can do with Uni that are hard to do with virtual threads. But yes, we will see a decrease in terms of usage.

About streams, I will go back to it after the blog post series. It's basically an implementation of the Tokyo channels (an excellent Rust toolkit) in Java. My initial implementation was still on Java 19... so I need to return to it before making it open source.

[starksm64]

Thanks Clement. On the Monopolization where the suggestion is to utilize a separate platform for these long running computations, might it be possible in the future to have a hint or some type of flag on the @RunOnVirtualThread annotation that either indicated an expectation of a long running op, or even a average execution time/range such that Quarkus could add an interceptor to potentially switch the computation to another thread pool automatically? I suppose we are missing some JVM APIs to effectively manage such a switch?

[cescoffier]

Hello @starksm64!

Yes, this is something I had in mind for a long time, but also something I need to discuss with more people. There are multiple issues (or tasks):

• I would like to detect monopolization, like we do in Vert.x (and Quarkus). I may have a lead to do that, but the JVM does not provide the right signals (JFR events) to have something 100% reliable. Note that the approach we use for reactive today is not 100% reliable either. When we have something, it will be added to loom unit (the junit extension to detect pinning, added to Quarkus last week) and Quarkus metrics.
• I was thinking of introducing a new annotation (named @LongRunning on my notebook, so nothing written in stone), which would indicate that you should not execute that method on a virtual thread, and in this case, we would invoke it on the worker thread pool. The calling virtual thread would be suspended (without pinning) until the method completes. It's relatively easy, but we must consider a few aspects: do you want the worker thread pool or another thread pool? What about context propagation (request scope, tracing, transaction...)? Do we want/support fault-tolerance annotation (like retry, fallback, or bulkhead, which would delay the execution until a slot is available (not the current behavior, if I'm not mistaken))...
• We need to document the mechanism carefully as it introduces a concurrency limit (the number of threads able to execute your long-running processing concurrently).

Note that this will be required for some intelligent applications. For example, if your application interacts with an AI model using a native call (for example, if you call tensor flow), you would need this mechanism (because it's long-running and because the native part will make it pinning). There are also several drivers (database, brokers..., etc.) relying on native, which would not be virtual-thread friendly at all (there is an emerging trend to provide Java clients using a thin layer using JNI/JNA to invoke a native library). Now, the question (and it's always the hardest) is the annotation's name.

[andrey-nakin]

if the virtual thread executes intensive and long computation, it monopolizes that carrier

OpenJDK documentation claims the following: "Virtual threads do not require or expect application code to explicitly hand control back to the scheduler; in other words, virtual threads are not cooperative."

https://openjdk.org/jeps/444

[cescoffier]

Yes, in the sense that they cannot explicitly ‘yield’. Also, the scheduling is not preemptive. Thus, monopolization can happen.

[tsabirgaliev]

@cescoffier is Thread.yield() noop for Virtual Threads?

[cescoffier]

It will try to yield without any guarantee.

[tsabirgaliev]

Hmm.. good to know. My (probably wrong) perception was that it WILL yield to scheduler but scheduler MAY NOT guarantee fairness, that is scheduler can just resume current thread without switching. So technically, there is room for improvement already covered by Threads API, no?

[cescoffier]

Yes, your perception is correct. It may continue running the same virtual thread, instead of putting it back in the queue.

[ennishol]

@cescoffier Thanks for a great post!
@RunOnVirtualThread is not working with rest-easy classic, right? Maybe quarkus could log a warning if the annotation is present but cannot be applied to the method or bean.

It would be also great if @RunOnVirtualThread could accept a custom thread name e.g. @RunOnVirtualThread("mytask). It will probably make debugging and jfr analysis easier in case of problems. Just guessing

[cescoffier]

Yes, adding a warning when using RESTEAsy Classic is a good idea. For several reasons, it's not possible to get RESTEasy Classic to use virtual threads without an all-or-nothing switch (which we want to avoid as it can affect your applications in unexpected ways).

About the naming, it isn't effortless, as our internal scheduler manages the naming, and we have a single one. Multiple virtual schedulers would be rather expensive, as we would need to track them, even if they are used to create a single virtual thread.

[rodrigorodrigues]

Hi @cescoffier,

Would you have a sample of Quarkus App using new Jdk 21 in docker?

I'm trying to dockerize one app but couldn't find a valid image to use jdk 21 then getting the error below.

company-api  | Starting the Java application using /opt/jboss/container/java/run/run-java.sh ...
company-api  | INFO exec  java -Dquarkus.http.host=0.0.0.0 -Djava.util.logging.manager=org.jboss.logmanager.LogManager -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -cp "." -jar /deployments/quarkus-run.jar 
company-api  | Exception in thread "main" java.lang.UnsupportedClassVersionError: com/microservice/quarkus/QuarkusApplication has been compiled by a more recent version of the Java Runtime (class file version 65.0), this version of the Java Runtime only recognizes class file versions up to 61.0
company-api  | 	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
company-api  | 	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1012)
company-api  | 	at io.quarkus.bootstrap.runner.RunnerClassLoader.loadClass(RunnerClassLoader.java:105)
company-api  | 	at io.quarkus.bootstrap.runner.RunnerClassLoader.loadClass(RunnerClassLoader.java:65)
company-api  | 	at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:60)
company-api  | 	at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:32)

[charleslip]

Try docker.io/bellsoft/liberica-runtime-container:jre-21-glibc .

[cescoffier]

Unfortunately, we are still awaiting a UBI image providing Java 21 - See https://adoptium.net/en-GB/blog/2023/09/temurin21-delay/. Now, I just heard that it would just be a matter of days.

Be aware that most existing containers do not provide run-java. So, you need to configure your application (JVM parameters) correctly.

[rodrigorodrigues]

Thanks cescoffier, I've fixed following this blog then used openjdk:21 image, more details here

[charleslip]

I noticed that Virtual-Threads support is now available for "quarkus-resteasy-reactive" and "quarkus-smallrye-reactive-messaging".

Would Virtual Threads support be coming to "quarkus-reactive-routes" soon?

[cescoffier]

Can you open an issue about that? We should definitely integrate virtual threads with reactive routes. I'm actually also thinking of integrating within Quarkus (lots of routes like openapi, health... could leverage virtual threads).

[ennishol]

@cescoffier loom-unit is really nice, thanks for your efforts! Would it be somehow possible to ignore some particular pinning events based on stack trace or at least count of pinning events?
The built-in http client sometimes pins the carrier thread (but not always) while closing an input stream. It would be great to be able to ignore such a case, WDYT?

[cescoffier]

You can do the following:

@Test
public void testThatShouldNotPin(ThreadPinnedEvents events) { // Inject an object to check the pin events
        Assertions.assertTrue(events.getEvents().isEmpty());
        codeUnderTest.pin();
        await().until(() -> events.getEvents().size() > 0);
        Assertions.assertEquals(events.getEvents().size(), 1);
}

[0ceanCat]

Hi, is there some way to use virtual thread in Service level classes? i tried use @RunOnVirtualThread on my service code, but it said that it could only be applied for endpoints.

When i use Thread.startVirtualThread(....) i dont have the correct context inside the thread, that is, i can not access @Intejct UserInfo userInfo inside the thread. (already tried with Arc.container().requestContext()).

Thanks

[ramiswailem]

@Inject
@VirtualThreads
ExecutorService vThreads;

[0ceanCat]

thank you, it works!

[dabkhazi]

@cescoffier could you comment this configuration parameter

smallrye.messaging.worker.<virtual-thread>.max-concurrency

What is <virtual-thread> ? Is this a method name or something else ?

Is 1024 maximum concurrency for each method annotated with @RunOnVirtualThread or for all methods ?

How limit executor service concurrency annotated with @VirtualThreads?

Thanks!

[ozangunalp]

smallrye.messaging.worker.<virtual-thread>.max-concurrency limits the concurrency for all Quarkus messaging @RunOnVirtualThread methods that are not configured with a worker name : https://quarkus.io/guides/messaging-virtual-threads#control-the-maximum-concurrency

To limit the concurrency of the @VirtualThreads ExecutorService you can use semaphores.

[dabkhazi]

smallrye.messaging.worker.<virtual-thread>.max-concurrency limits the concurrency for all Quarkus messaging @RunOnVirtualThread methods that are not configured with a worker name : https://quarkus.io/guides/messaging-virtual-threads#control-the-maximum-concurrency

To limit the concurrency of the @VirtualThreads ExecutorService you can use semaphores.

Ok, each method annotated with @RunOnVirtualThread has 1024 maximum concurrency by default. Am i right? Or is this the total count for all such methods? What happens if this limit is increased? Will i get an error that can be handled somehow (can't find it in docs)?

When i asked about @VirtualThreads concurrency, i meant is there a built-in mechanism in the quarkus?

Thanks a lot!

[cescoffier]

No, the concurrency limit is only for the messaging part. It was mostly designed for Kafka as, if you create one virtual thread per incoming record, you can eat a lot of memory to create a very high number of virtual threads.

[dabkhazi]

Thanks, i didn't notice that this is only for messaging.

[dabkhazi]

But I still have questions
If my code can run entirely on virtual threads, but quarkus still creates event loop threads. Do I need to set 0 for quarkus.vertx.event-loops-pool-size in the configuration so that these threads are not created?

I have to create a lot of soap services. The only extension is quarkus cxf. But this extension does not support either the reactive approach or virtual threads. Why? Why does this extension contradict the general principle of qaurkus to be reactive?

[cescoffier]

No you still need event loops as the I/O work use event loop and not virtual threads. There are many reason why we keep this architecture (maturity, performance, ecosystem...).

[dabkhazi]

No you still need event loops as the I/O work use event loop and not virtual threads. There are many reason why we keep this architecture (maturity, performance, ecosystem...).

Hm, I guess I didn't express myself clearly. When my code marked @RunOnVirtualThread is executed, I see that it is not executed on the Vert.x event loop thread, but on another thread (virtual), since I displayed its name for debugging. But the general reason now is clear to me.

But what I don’t like most is that the Quarkus CXF extension can’t work with virtual threads :(.

[cescoffier]

Let's me ping @ppalaga about that :-)