Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linear performance test times out intermittently on JRuby #1310

Open
headius opened this issue Mar 10, 2025 · 3 comments
Open

Linear performance test times out intermittently on JRuby #1310

headius opened this issue Mar 10, 2025 · 3 comments

Comments

@headius
Copy link
Contributor

headius commented Mar 10, 2025

The linear timing test "test_read_directive_linear_performance" occasionally times out on JRuby. This does not likely indicate a failure to run the example in linear time as much as it reflects a sensitivity to very short timeouts. Until we fully understand why this test times out intermittently, I will omit it on JRuby.

The timeout failure produces the following output in CI:

Error: test_read_directive_linear_performance(TestRDocParserRuby): Timeout::Error: [100]: in 0.01s (tmin: 3.2501000021056825e-05, tmax: 7.660400001441303e-05, tbase: 1/1000)
org/jruby/ext/thread/Mutex.java:119:in `lock'
org/jruby/ext/thread/Mutex.java:181:in `synchronize'
/home/runner/.rubies/jruby-head/lib/ruby/stdlib/timeout.rb:93:in `finished'
/home/runner/.rubies/jruby-head/lib/ruby/stdlib/timeout.rb:191:in `block in timeout'
/home/runner/.rubies/jruby-head/lib/ruby/stdlib/timeout.rb:196:in `timeout'
/home/runner/work/rdoc/rdoc/vendor/bundle/jruby/3.1.0/gems/test-unit-ruby-core-1.0.6/lib/core_assertions.rb:836:in `block in assert_linear_performance'
org/jruby/RubyArray.java:2009:in `each'
/home/runner/work/rdoc/rdoc/vendor/bundle/jruby/3.1.0/gems/test-unit-ruby-core-1.0.6/lib/core_assertions.rb:831:in `assert_linear_performance'
/home/runner/work/rdoc/rdoc/test/rdoc/test_rdoc_parser_ruby.rb:[33](https://github.com/ruby/rdoc/actions/runs/13748754286/job/38447004956#step:5:34)87:in `test_read_directive_linear_performance'
     3384: 
     3385:   def test_read_directive_linear_performance
     3386:     pre = ->(i) {util_parser '# ' + '0'*i + '=000:'}
  => 3387:     assert_linear_performance((1..5).map{|i|10**i}, pre: pre) do |parser|
     3388:       assert_nil parser.read_directive []
     3389:     end
     3390:   end
org/jruby/RubyKernel.java:1426:in `catch'
org/jruby/RubyKernel.java:1421:in `catch'
org/jruby/RubyArray.java:2009:in `each'
org/jruby/RubyArray.java:2009:in `each'
org/jruby/RubyArray.java:2009:in `each'
org/jruby/RubyKernel.java:1426:in `catch'
org/jruby/RubyKernel.java:1421:in `catch'
@headius
Copy link
Contributor Author

headius commented Mar 10, 2025

See comments here discussing this failure: #1305 (comment)

headius added a commit that referenced this issue Mar 10, 2025
JRuby's perf does not suddenly go non-linear, but this test is
occasionally timing out. We will investigate separtely and disable
it for now.

See #1310
@headius
Copy link
Contributor Author

headius commented Mar 10, 2025

@enebo This exercises our Java-based parser through Ripper. Can you think of any reason why it might occasionally parse slower, other than VM variability issues?

@enebo
Copy link

enebo commented Mar 11, 2025

@headius It is a huge lists of lambdas being called from a single method. Any change in performance in executing the same code I think would be from the JVM deciding to compile stuff (which should happen up front). I would not expect it to be linear from that alone. Your comment (in linked issue) sounds right that they need a warmup cycle to measure linear performance.

If that does not explain it then we probably need to generate a histogram and see if there is something weird happening like a growing but removable data structure which is causing a large GC spike.

@kou kou changed the title LInear performance test times out intermittently on JRuby Linear performance test times out intermittently on JRuby Mar 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants