Ryan Angilly's Blog

How Google tracks clicks for backpropagation

I was wondering a few nights ago: How does Google track clicks for their backpropagation algorithms? So I cracked open Firebug, did a search and watched the Console tab while I clicked a link. I was expecting to see a nice little XMLHttpRequest go across the screen… and didn’t.

Upon further inspection, I found that the links in search results all had inline onclick handlers:

A call to clk() in the console tab just returned true, so off to the source for clk() I went.

Cmd+U… Cmd+F… ‘clk’…:

  window.clk=function(d,e,f,j,k,l,m){}

Those sons of bitches. Back to Firebug, and the Net tab. At this point, I noticed a whole bunch of requests that had responded with 204s. The request for these 204’d documents had image/png and image/jpg in the accepts header, and it hits me: they’re pixel tagging. Very clever. But I still want to find clk(), so I see in the Net tab where there was a 19k JS file requested, open it up, do a search, and with a little bit of manual de-minifying:

function(d,e,f,j,k,l,m) {
  if(document.images){
    var a=encodeURIComponent||escape,b=new Image,
        g=window.google.cri++;
    window.google.crm[g]=b;
    b.onerror=b.onload=b.onabort=function() {
      delete window.google.crm[g];
    };

    var c,h,i;

    if(google.v6) {
      c=google.v6.src;
      h=google.v6.complete||google.v6s?2:1;
      i=(new Date).getTime()-google.v6t;i
      delete google.v6;
    }

    b.src=["/url?sa=T","&source="+google.sn,
      e?"&oi="+a(e):"",f?"&cad="+a(f):"","&ct=",
      a(j||"res"),"&cd=",a(k),"&ved=",a(m),
      d?"&url="+a(d.replace(/#.*/,"")).replace(/\+/g,"%2B"):"",
      "&ei=",google.kEI,c?"&v6u="+a(c)+"&v6s="+h+
      "&v6t="+i:"",l].join("")
  }
  return true
};

Jackpot.

When you think about it, this makes WAY more sense than an XHR. It’s cross browser, probably a hell of a lost faster (even though I haven’t taken the time to benchmark it), and just so… Google. This may be old hat to some, but I never though of using pixel tagging dynamically in this way. Something tells me we’ll be using this technique in Engineering Alley at Punchbowl HQ in the next few weeks.

Ruby statement modifers behave differently than conditional statements

If you’re a Ruby guy or gal, you know about statement modifiers. They are lovely little bits of syntax that let us do stuff like:

raise "You're an idiot" if params.nil?
call_some_awesome_method unless it_was_already_called

Which, we’ve all been told, is the exact same thing as doing:

if params.nil?
  raise "You're an idiot"
end

unless it_was_already_called
  call_some_awesome_method
end

Well, as it turns out, this isn’t true. Statement modifiers in Ruby behave differently than their conditional brethren. Something that caused me quite a bit of pain a couple weeks ago. Take the following bit of code. We’ll go through it line by line.

ra:~$ irb
irb(main):001:0> a
NameError: undefined local variable or method `a' for main:Object
  from (irb):1
irb(main):002:0> defined? a
=> nil
irb(main):003:0> a
NameError: undefined local variable or method `a' for main:Object
  from (irb):3
irb(main):004:0> a = 5 unless defined? a
=> nil
irb(main):005:0> a
=> nil
irb(main):006:0> defined? a
=> "local-variable"
irb(main):007:0> 

1. crack open an irb console and try to access the variable a. It hasn’t been defined yet, so irb appropriately complains.
2. use the defined? keyword to check what the interpreter currently thinks of that variable. nil is returned. The interpreter has never seen it. Undefined as expected.
3. lets check a again to see if calling defined? on it did antything. Nope. Still undefined. So far so good.
4. Now, I write some code that I think means “Hey Ruby, if you have never seen a before, then set it equal to 5.
5. … a is nil… wtf?
6. ?

Thought I knew Ruby. Let’s rewrite this using conditionals instead of modifiers.

irb(main):007:0> b
NameError: undefined local variable or method `b' for main:Object
  from (irb):7
irb(main):008:0> defined? b
=> nil
irb(main):009:0> unless defined? b
irb(main):010:1>   b = 5
irb(main):011:1> end
=> 5
irb(main):012:0> b
=> 5
irb(main):013:0> defined? b
=> "local-variable"
irb(main):014:0> 

Which is what you’d expect. So what’s going on here? My first thought was that maybe there was an operator binding precedence thing going on. Like maybe it was evaluating: a = (5 unless defined?(a)). But even that should assign 5 to a. Then I tried (a = 5) unless defined?(a). No luck. Same behavior. a gets assigned nil.

I think the explanation goes something like this: In the first example, the interpreter sees a and realizes right away that it’s an lvalue. As such, it adds a to the local variable list. Then, it continues on and sees an assignment operation (=), an expression (5) and a statement modifier (unless defined? a). It delays evaluating the expression and executing the assignment because of the statement modifier. Once it evaluates that modifier, it decides to not execute the expression or assignment.

So I think about this for a while wondering how I could test the theory and it dawns on me that there is a much simpler way to illustrate this problem:

irb(main):001:0> a
NameError: undefined local variable or method `a' for main:Object
	from (irb):1
irb(main):002:0> a = 5 unless (res = defined?(a))
=> nil
irb(main):003:0> a
=> nil
irb(main):004:0> res
=> "local-variable"
irb(main):005:0> 

I’m not sure if I’m missing something super obvious, something super obscure, or if this is a bug in Ruby. But I know that res should be nil and a == 5.

Just something to keep in mind down the road. Statement modifiers behave differently than conditional statements.

Dynamically adding class methods in Ruby

Even though there is technically no such thing as a class method in Ruby, I’m going to call them that for the sake of clarity.  When I say class method, I mean something like this:

class A
  class << self
    def yo
      "wassup"
    end
  end
end

A.yo #=> "wassup"

Every now and then, you may want to dynamically generate these things.  Thanks to instance_eval and define_method, dynamically defining methods in Ruby is trivial, but these operate in what someone coming from Java or C++ would call the instance context.  Take the following example:

class A
  class << self
    def create_method(name)
      define_method(name) { puts "Nice!  I'm #{name}" }
    end
  end
end


A.create_method('mine')
A.mine     #Raises NoMethodError
A.new.mine #prints out "Nice! I'm mine"

The define_method creates a method that is only accessible on instances of the class, not a class method. I’ve seen people struggle with this, and in some code you’ll end up with stuff like this:

class A
  class << self
    def create_method(name)
      self.class.instance_eval do
        define_method(name) { puts "Nice!  I'm #{name}" }
      end
    end
  end
end

A.create_method('mine')
A.mine     #prints out "Nice! I'm mine"

Great success! Case closed! The problem is that this is crap. It’s wrong. To understand why it’s wrong, check out the following example:

class A
  class << self
    def create_method(name)
      self.class.instance_eval do
        define_method(name) { puts "Nice!  I'm #{name}" }
      end
    end
  end
end
A.create_method('mine')
A.mine     # prints out "Nice! I'm mine"

class B
end
B.mine     # prints out "Nice! I'm mine"

What?

Calling self.class.instance_eval will evaluate the block on self.class, which in this case is Class itself, the object from which all classes descend.  Thanks to inheritance, that means that every class will get this method:

"1".class.mine    # prints out "Nice! I'm mine"
1.class.mine      # prints out "Nice! I'm mine"

The big problem with this is that it does work. It deceives you. And the real kick in the balls can come when you want to define this class method in a class, and then call it from subclasses:

class Main
  def Main.create_method(name, args)
    klass = self.to_s
    self.class.instance_eval do
      define_method(name) { return "nice! from #{klass} with #{args.inspect}" }
    end
  end
end

class A < Main
  create_method :delete, :only => 7
end
A.delete  #=> "nice! from A with {:only=>7}"

class B < Main
  create_method :delete, :only => 8
end

B.delete  #=> "nice! from B with {:only=>8}"
A.delete  #=> "nice! from B with {:only=>8}"

Yeah. The method delete doesn’t get defined separately on A and B. It gets defined on Class once, and then redefined. When A to tries and run #delete, Ruby looks in A, then Main, then Class. When B tries to run #delete, Ruby looks in B, then Main, then Class. Same method.

This technique can ruin your day.

So, Ryan, what’s the solution?

Simple. The metaclass. The metaclass is defined as:

metaclass = class << self
              self
            end

I’m not going to explain the metaclass (aka eigenclass, aka singleton class) in this post, there’s plenty of material on it. Rails ActiveSupport module gives you a method called metaclass that you can use anywhere, without ActiveSupport you’ll need to get at it yourself. But once you have it, you can do:

class Object
  def metaclass
    class << self; self; end
  end
end

class Main
  def Main.create_method(name, args)
    klass = self.to_s
    metaclass.instance_eval do
      define_method(name) { return "nice! from #{klass} with #{args.inspect}" }
    end
  end
end

class A < Main
  create_method :delete, :only => 7
end

A.delete  #=> "nice! from A with {:only=>7}"

class B < Main
  create_method :delete, :only => 8
end

B.delete  #=> "nice! from B with {:only=>8}"
A.delete  #=> "nice! from A with {:only=>7}"

You win.

Go forth and metaprogram.