17 ms·

~22:23: To reduce the CPU time your app uses, make sure all your inner loops are in C Also, "Optimizing CPU is easy, Optimizing Memory is Hard"

• jedsmith 16y ago
  I was at the keynote and his explanation about using any() to move a tight loop to C blew my mind. I had never thought about optimizing that way, and that is a very enlightening look at making tight loops faster. Quite a dramatic improvement, too.
  • riobard 16y ago
    Correct me if I got it wrong, but does it mean that putting any Python function in any() makes it run in C??? From my understanding, the any() version is faster is primarily because it avoids looking up .update() method each iteration. The usual Python way of optimizing such cases is: # the original slow version # 2.87753200531 sec def run(): _md5 = hashlib.md5() for i in itertools.repeat("foo", 10000000): _md5.update(i) # slight change # 1.82029104233 sec def run(): _md5 = hashlib.md5() update = _md5.update for i in itertools.repeat("foo", 10000000): update(i) # using any() # 1.50683498383 sec def run(): _md5 = hashlib.md5() any(itertools.imap(_md5.update, itertools.repeat("foo", 10000000)))
    • jedsmith 16y ago
      The iterator is consumed in C using any(), instead of the Python bytecode being executed to perform the loop. I don't know if it'd work for "any Python function," but I'd definitely look at it for long-running iterators.
  • bl4k 16y ago
    does anybody know if AppEngine supports any() and itertools ?
    • sunqiang 16y ago
      Python 2.5 has any (and all) Python 2.3 has itertools so Google App Engine(2.5) should have both of them.
  • pdhborges 16y ago
    Just a little not so readers don't get the wrong impression: - imap returns an iterator, any is used to "force" the iterator run through all elements. - the loop runs on C because of any. Instead of a for loop where jump and body are repeatedly evaluated by the interpreter, any consumes the iterator through a loop implemented in C.