6 ms·

This problem is not going to go away so easily. Numerous core Java classes (like BufferedInputStream) use synchronized. I count 1600+ usages in java.base.

This problem is not going to go away so easily. Numerous core Java classes (like BufferedInputStream) use synchronized. I count 1600+ usages in java.base. The blocking issue means it's _much_ easier to accidentally run into this, rather than waving it away as an unlikely edge case. I personally ran into this Using the built in com.sun webserver, with a virtual thread executor. My VPS only has two CPUs which means the FJP that virtual threads run on only have 2 active threads at a time. I ran into this hang when some of the connection hung, blocking any further requests from being processed.

he0001 3y ago
BufferdInputStream is rewritten and is only using synchronized if subclassed. In fact there has been a lot of work removing the synchronized keyword.
jillesvangurp 3y ago
People always forget that things that only happen every few million times, can happen fairly frequently on a busy server. This has bitten me numerous times. The nature of a lot of these types of issues is that they are hard to detect and hard to reproduce. Virtual threads are nice for unblocking legacy code but they aren't without issues. There are better options for new code with less trade offs on the jvm as well. I've recently been experimenting with jasync-postgresql (there's a mysql variant as well) as an alternative to JDBC in Kotlin. It's a nice library. It does have some limitations and is a bit on the primitive side. But it appears to be somewhat widely used in various database frameworks for Scala, Java, and Kotlin. Databases and database frameworks are an area on the JVM where there just is a huge amount of legacy code built on threads and blocking IO. It's probably one of the reasons Oracle worked on virtual threads as migrating away from these frameworks is unlikely to ever happen in a lot of code bases. So, waving a magic wand and making all that code non blocking is very attractive. But of course that magic has some hard limitations and synchronize blocks are one of those. I imagine they are working on improving that further.
- kaba0 3y ago
  Synchronized blocks are not a problem. Synchronized blocks that later don’t unblock the thread may sometimes be.
- mike_hearn 3y ago
  The lack of support for synchronized isn't a fundamental or hard limit, it's just that the HotSpot implementation is complicated for performance reasons and they put off rewriting that code until later. They're indeed working on that now and in some future version I guess wait/notify and synchronized blocks will start to work. After all, you can easily transform such code into an equivalent that does work.
- tveita 3y ago
  There are ways to find problem sections without having to trigger a full deadlock: https://openjdk.org/jeps/444 https://openjdk.org/jeps/444 The system property jdk.tracePinnedThreads triggers a stack trace when a thread blocks while pinned. Running with -Djdk.tracePinnedThreads=full prints a complete stack trace when a thread blocks while pinned, highlighting native frames and frames holding monitors. Running with -Djdk.tracePinnedThreads=short limits the output to just the problematic frames.
- vincnetas 3y ago
  Was curious what it is "jasync". And man it hurts me to see documentation like this (when compared to classic javadocs) https://github.com/jasync-sql/jasync-sql/wiki/API-Overview https://github.com/jasync-sql/jasync-sql/wiki/API-Overview From project WIKI (https://github.com/jasync-sql/jasync-sql/wiki https://github.com/jasync-sql/jasync-sql/wiki)
- simiones 3y ago
  > Virtual threads are nice for unblocking legacy code but they aren't without issues. There are better options for new code with less trade offs on the jvm as well. The designers of Project Loom would say the exact opposite. The whole push behind Project Loom and similar models (Go's oft-praised "goroutines" runtimes being another one) is motivated by Threads being a much better fit for async behavior in a fundamentally procedural language like Java or Go than promise-based frameworks like async/await. The whole motivation of Project Loom is to make the simple thing (spawning threads to handle blocking IO) the fast thing as well (by actually replacing the blocking IO with efficient async IO OS calls and managing the threads internally). Project Loom will be considered a full success if the next generation Java web server does something akin to "new Thread(() -> {executeHandlerFunc(conn); }.Start(); " for each incoming connection, just like the Go built-in web server.
anthony88 3y ago
I've written an open source library to easily replace synchronized with something more virtual thread friendly: https://github.com/japplis/Virtually https://github.com/japplis/Virtually
pron 3y ago
As the JEP states, pinning due to synchronized is a temporary issue. We didn't want to hold off releasing virtual threads until that matter is resolved (because users can resolve it themselves with additional work), but a fix already exists in the Loom repository, EA builds will be offered shortly for testing, and it will be delivered in a GA release soon. Those who run into this issue and are unable or unwilling to do the work to avoid it (replacing synchronized with j.u.c locks) as explained in the adoption guide [1] may want to wait until the issue is resolved in the JDK. I would strongly recommend that anyone adopting virtual threads read the adoption guide. [1]: https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html#GUID-8AEDDBE6-F783-4D77-8786-AC5A79F517C0 https://docs.oracle.com/en/java/javase/21/core/virtual-threa...
- cesarb 3y ago
  > unable or unwilling to do the work to avoid it The problem is that it's rare to write code which uses no third-party libraries, and these third-party libraries (most written before Java virtual threads ever existed) have a good chance of using "synchronized" instead of other kinds of locks; and "synchronized" can be more robust than other kinds of locks (no risk of forgetting to release the lock, and on older JVMs, no risk of an out-of-memory while within the lock implementation breaking things), so people can prefer to use it whenever possible. To me, this is a deal breaker; it makes it too risky to use virtual threads in most cases. It's better to wait for a newer Java LTS which can unmount virtual threads on "synchronized" blocks before starting to use it.
  - josefx 3y ago
    > have a good chance of using "synchronized" instead of other kinds of locks; and "synchronized" can be more robust than other kinds of locks (no risk of forgetting to release the lock, and on older JVMs, no risk of an out-of-memory while within the lock implementation breaking things), I haven't professionally written Java in years, however from what I remember synchronized was considered evil from day one. You can't forget to release it, but you better got out of your way to allocate an internal object just for locking because you have no control who else might synchronize on your object and at that point you are only a bit of syntactic sugar away from a try { lock.lock();}finally{lock.unlock();} .

he0001 3y ago

BufferdInputStream is rewritten and is only using synchronized if subclassed. In fact there has been a lot of work removing the synchronized keyword.

jillesvangurp 3y ago

People always forget that things that only happen every few million times, can happen fairly frequently on a busy server. This has bitten me numerous times. The nature of a lot of these types of issues is that they are hard to detect and hard to reproduce. Virtual threads are nice for unblocking legacy code but they aren't without issues. There are better options for new code with less trade offs on the jvm as well. I've recently been experimenting with jasync-postgresql (there's a mysql variant as well) as an alternative to JDBC in Kotlin. It's a nice library. It does have some limitations and is a bit on the primitive side. But it appears to be somewhat widely used in various database frameworks for Scala, Java, and Kotlin. Databases and database frameworks are an area on the JVM where there just is a huge amount of legacy code built on threads and blocking IO. It's probably one of the reasons Oracle worked on virtual threads as migrating away from these frameworks is unlikely to ever happen in a lot of code bases. So, waving a magic wand and making all that code non blocking is very attractive. But of course that magic has some hard limitations and synchronize blocks are one of those. I imagine they are working on improving that further.

kaba0 3y ago
Synchronized blocks are not a problem. Synchronized blocks that later don’t unblock the thread may sometimes be.
mike_hearn 3y ago
The lack of support for synchronized isn't a fundamental or hard limit, it's just that the HotSpot implementation is complicated for performance reasons and they put off rewriting that code until later. They're indeed working on that now and in some future version I guess wait/notify and synchronized blocks will start to work. After all, you can easily transform such code into an equivalent that does work.
tveita 3y ago
There are ways to find problem sections without having to trigger a full deadlock: https://openjdk.org/jeps/444 https://openjdk.org/jeps/444 The system property jdk.tracePinnedThreads triggers a stack trace when a thread blocks while pinned. Running with -Djdk.tracePinnedThreads=full prints a complete stack trace when a thread blocks while pinned, highlighting native frames and frames holding monitors. Running with -Djdk.tracePinnedThreads=short limits the output to just the problematic frames.
vincnetas 3y ago
Was curious what it is "jasync". And man it hurts me to see documentation like this (when compared to classic javadocs) https://github.com/jasync-sql/jasync-sql/wiki/API-Overview https://github.com/jasync-sql/jasync-sql/wiki/API-Overview From project WIKI (https://github.com/jasync-sql/jasync-sql/wiki https://github.com/jasync-sql/jasync-sql/wiki)
simiones 3y ago
> Virtual threads are nice for unblocking legacy code but they aren't without issues. There are better options for new code with less trade offs on the jvm as well. The designers of Project Loom would say the exact opposite. The whole push behind Project Loom and similar models (Go's oft-praised "goroutines" runtimes being another one) is motivated by Threads being a much better fit for async behavior in a fundamentally procedural language like Java or Go than promise-based frameworks like async/await. The whole motivation of Project Loom is to make the simple thing (spawning threads to handle blocking IO) the fast thing as well (by actually replacing the blocking IO with efficient async IO OS calls and managing the threads internally). Project Loom will be considered a full success if the next generation Java web server does something akin to "new Thread(() -> {executeHandlerFunc(conn); }.Start(); " for each incoming connection, just like the Go built-in web server.

anthony88 3y ago

I've written an open source library to easily replace synchronized with something more virtual thread friendly: https://github.com/japplis/Virtually https://github.com/japplis/Virtually

pron 3y ago

As the JEP states, pinning due to synchronized is a temporary issue. We didn't want to hold off releasing virtual threads until that matter is resolved (because users can resolve it themselves with additional work), but a fix already exists in the Loom repository, EA builds will be offered shortly for testing, and it will be delivered in a GA release soon. Those who run into this issue and are unable or unwilling to do the work to avoid it (replacing synchronized with j.u.c locks) as explained in the adoption guide [1] may want to wait until the issue is resolved in the JDK. I would strongly recommend that anyone adopting virtual threads read the adoption guide. [1]: https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html#GUID-8AEDDBE6-F783-4D77-8786-AC5A79F517C0 https://docs.oracle.com/en/java/javase/21/core/virtual-threa...

cesarb 3y ago
> unable or unwilling to do the work to avoid it The problem is that it's rare to write code which uses no third-party libraries, and these third-party libraries (most written before Java virtual threads ever existed) have a good chance of using "synchronized" instead of other kinds of locks; and "synchronized" can be more robust than other kinds of locks (no risk of forgetting to release the lock, and on older JVMs, no risk of an out-of-memory while within the lock implementation breaking things), so people can prefer to use it whenever possible. To me, this is a deal breaker; it makes it too risky to use virtual threads in most cases. It's better to wait for a newer Java LTS which can unmount virtual threads on "synchronized" blocks before starting to use it.
- josefx 3y ago
  > have a good chance of using "synchronized" instead of other kinds of locks; and "synchronized" can be more robust than other kinds of locks (no risk of forgetting to release the lock, and on older JVMs, no risk of an out-of-memory while within the lock implementation breaking things), I haven't professionally written Java in years, however from what I remember synchronized was considered evil from day one. You can't forget to release it, but you better got out of your way to allocate an internal object just for locking because you have no control who else might synchronize on your object and at that point you are only a bit of syntactic sugar away from a try { lock.lock();}finally{lock.unlock();} .