How to fetch URLs in parallell with EventMachine and Ruby
Ruby posted 10 months ago by christian
Save time by doing things in parallell:
1 require 'rubygems' 2 require 'eventmachine' 3 require 'open-uri' 4 require 'pp' 5 require 'thread' 6 require 'benchmark' 7 8 class Worker 9 include EM::Deferrable 10 11 def run 12 get_google 13 set_deferred_status :succeeded 14 end 15 end 16 17 def get_google 18 # sorry for spamming you 19 open('http://www.google.com/') do |f| 20 #pp f.meta 21 end 22 end 23 24 def asynchronous(i) 25 worker = Worker.new 26 # on success 27 worker.callback do 28 p "#{Thread.current} done #{i}!" 29 end 30 worker.errback do 31 p "Unexpected error" 32 EM.stop 33 end 34 # 35 Thread.new do 36 worker.run 37 EM.stop 38 end 39 #puts "scheduling done!" 40 end 41 42 def synchronous(i) 43 get_google 44 end 45 46 # on error 47 EM.error_handler do |e| 48 p "Unexpected error: #{e}" 49 end 50 51 EM.run do 52 seconds = Benchmark.realtime do 53 50.times do |i| 54 asynchronous i 55 end 56 end 57 p "With EventMachine: #{seconds} elapsed..." 58 59 seconds = Benchmark.realtime do 60 50.times do |i| 61 synchronous i 62 end 63 end 64 p "Without EventMachine: #{seconds} elapsed..." 65 end
Output:
1 With EventMachine: 9.05974316596985 elapsed... 2 Without EventMachine: 19.1381118297577 elapsed...
Conclusion
- Speeds up blocking operations.
- EventMachine is currently limited to one CPU core (native thread) per process.