Can i update only readme.txt file without release a new version for WordPress.org Plugin?

Yes! You don’t need to release a new version. Whenever a major version of WordPress is released, WordPress.org will send a reminder to plugin authors to test their plugins with the latest version. After testing your plugins and ensuring compatibility, it only takes a few moments to change the readme.txt “Tested up to:” value to latest wp version. This information provides peace of mind to users and encourages them to update to the latest version. So just change the stable version’s readme.txt value.