This is just really cool.

I’ve been loving my ergonomic Kinesis keyboard. I got my employer to buy me one for work, too. It has been a major help with my repetitive strain injury. I have the Pro model, which comes with a footswitch, which I mapped to Shift — a lifesaver for my pinkies.

My only real complaint with the Kinesis is that the key action is not padded! At the bottom of the stroke, the key hits the stop, giving my achy fingers a jolt they don’t appreciate. This has bugged me for a while, and I finally came up with a fix.

1. Get a pack of twisty balloons (you know, the skinny kind that clowns make animals out of).
2. Cut an inch-long section of balloon (a tube).
3. Use the inner part of a pen (the ink tube) to roll the balloon tube up into a little grommet (a small torus).
4. Remove a key from the keyboard (using the key removal tool) and roll the grommet over the shaft inside the key.
5. Replace the key.
6. Repeat from step 2.

It took me about 1 hour to do the whole keyboard, but my fingers are now happily bouncing.

Hi. I’m Jeff, and I’m a compuholic.

Due to working at the computer for about 12 hours a day, I’ve recently been suffering from repetitive strain injury. As a result, I have had to wean myself off the machine in order to prevent long-term nerve damage. It’s been incredibly difficult since the computer is an integral tool in my work and hobbies. How can I maintain productivity without using my computer all the time?

As I have stepped back from my normal pattern of computer usage, I’ve gained some fruitful insights.

1. The appearance of productivity enabled by computers is intoxicating.

Computers are automation machines. They help us do two things: (a) generate information, and (b) process information. Generating information is great if it is useful information. But more often computers are used to generate junk. It is easy and rewarding to create impressive reports, draw pretty pictures, and write nifty software. But consider this: it might be more productive to do nothing, rather than waste other people’s time with useless content. Be certain that you are creating something because it is really needed — not just because you can.

Processing information is wonderful if you learn something from it. However, due to item (a), most information is worthless. You may feel accomplished, but actually achieve nothing, by spending your morning reading tons of email, articles, posts, and news. Screen the information you spend time on. Be certain that it is worth processing.

2. Dependence on references for knowledge subverts subconscious problem solving.

The brain works in parallel — many ideas are simultaneously assessed. But you are never conscious of more than one or two. When information is not known from memory, it must be retrieved via the slow, single-process method of looking it up (e.g. using an internet search). This bottleneck inhibits the rapid problem-solving that is possible when the brain has all the information it needs. If you memorize information, your brain can perform optimally as a massive parallel idea filter.

The capacity of the human brain is astonishing. Kim Peek, the inspiration for Rain Man, can read a book in one hour and recall 98% of it. He has memorized 12,000 books in many different subjects — history, literature, geography, math, sports, music. Your brain has more computational power than the greatest supercomputer on Earth. Use it.

3. The quality of published books far exceeds that of free material on the web.

A search engine is the first place I go for answers. For commonplace information, this is great. However, when exploring a subject in depth, the internet is often a spiderweb of interesting-looking links that reel you in and distract you from your original purpose. Due to the investment required to print a book, authors and editors spend a lot of time making their material complete, correct, and interesting. Publishers usually won’t invest in something that isn’t high quality.

When you really want to learn about a subject, read authoritative books and seek answers from experts. If they don’t know the answer, they can usually point you to someone who knows more than they do. You are only six degrees of separation from the answer.

Productivity can be increased by replacing habitual patterns of computer usage with more introspective offline approaches.

I’ve been having a bout with “pinky pain” lately, as recently I’ve been working at the computer pretty much continuously. After reading several related blog posts, this seems to be a perpetual problem among programmers.

In my quest to solve this problem, here’s a few useful things I’ve come across. (I’m not an expert on ergonometry or anything — this is just what’s been helpful for me.) Hopefully it will help someone else out there.

Get a good mouse, or ban the rodent altogether

This is number one. I can’t believe that all computers (AFAIK) still ship with the standard “top-click/slide” mouse. Just use one of these for 12+ hours a day in grad school writing code for your robot competition project and you will come to hate them. Now if I happen to use one of these for only a few hours, my hand will punish me the whole next day. A while back I switched over to a Logitech TrackMan Wheel mouse. I’ve been pretty happy with it. Another interesting option is the Evoluent VerticalMouse, which some people swear by. Or you can ban the rodent altogether and go mouse-free using Ratpoison and Conkeror.

Get a good keyboard

I had been using a standard Dell-provided keyboard, which was OK, except that the keys began getting hard to press. Enter more hand pain. Just getting a new keyboard with easy-to-press keys made a big difference. I’ve thought about getting a specialty keyboard, but haven’t yet. Interesting options include the HHK (Happy Hacking Keyboard), the pricey Contoured Kinesis, and Jon Aquino’s “K’nexis” hack. Another possible idea is to use a miniature keyboard — putting all the keys closer to your hands for less reach-strain — I haven’t tried this, but it sounds promising.

Snuggle up with your keyboard

I used to have my keyboard on the desk in front of my monitor. But I found that when working for extended periods, it was causing strain to keep my hands positioned over the keyboard. I have a wristrest, but it doesn’t seem to help much. I experimented with keeping my keyboard on my lap, in order to be able to be comfortable while typing. Immediately, I could work for longer without getting that numbness in my wrists and pinkies. Since my legs were kind of restricted in having to keep the keyboard positioned, I rigged up a little platform for my keyboard that attaches to my chair’s armrests. The beautiful thing about this setup is that my fingers are on the home row when my arms are perfectly at rest in my lap — and remain so even when I recline in my chair, or scoot away from the desk. I use the mouse as little as possible, but find that for some tasks it is easier on the hands. For such occasions, I keep my mouse (a trackball) also in my lap. (Another interesting setup is Karl Pfleger’s split-keyboard on his armrests.)

Swap Control-key with Caps-Lock

The standard PC keyboard layout is really terrible for programmers. Or anyone. Particularly inane is the position of the Caps-Lock key, in prime position next to the home row. It’s actually closer to your hand than Return. Since Control is used very frequently, a simple mod that alleviates some of the idiocy of this layout is to swap Control with Caps-Lock. Manicai.net has instructions on how to do this for various OS’s.

Use “Sticky-Keys”

A significant part of the problem, for me, is key combinations, like CTRL+v. You have to hold one key with your pinky-finger and press another with the same hand. OK, so, Sticky-Keys. With this enabled, you can sequentially hit CTRL and then v. (It’s backwards-compatible, too — you can still do CTRL+v if you want.) I thought at first that this would get confusing and I’d make a lot of mistakes, but it actually works really well. I was able to be completely proficient within only a few minutes.

Get a good editor

OK, not intending to start any flame wars here, but personally I find Vim to be very easy on the hands. The multiple modes enable me to do most editing commands using the letter keys. The other major alternative is, of course, Emacs, which you could probably customize to use the keystrokes you want. If you’re not using one of these yet, you’re in for a treat — give them a try.

Get a good chair

This isn’t really about hand pain, as much as lower back pain (which I was experiencing for a while…). The Steelcase Criterion chair (high-back) is the standard office desk chair at my work, and it is fabulous. I shelled out the bucks to get one at home also because I can basically work in it indefinitely without pain in my back (or butt). It rolls and swivels, and basically everything is customizable — height, armrest height, armrest position, recline position, recline lock, recline resistance, seat angle, seat depth… is there anything else?

Random thoughts on keyboard layouts

I’ve been thinking it would be cool if there were a keyboard layout that used only the letter keys, spacebar, and ;/., (and maybe number keys), in order to minimize pinky-finger usage. These keys could be used in a multi-modal way (like Vim) to enable the functionality other keys provide, but without having to torque your hands/fingers at all. This may be possible to do with Xkb. Also, some people really like the Dvorak keyboard layout. I’ve tried it briefly, but had to switch back to qwerty for now due to my pinky-pain… I’ll probably switch over to Dvorak in the near future.

Update: After writing this and doing some more research (and getting fed up with this hand pain, and becoming quite concerned ), I just ordered a Kinesis Contoured keyboard. It seems to be the best ergonomic choice, especially for the pinky peril. I’ll probably post more about how well it works out.

Just uploaded my Python implementation of the Mersenne Twister random number generator.

Check it out here.

Shed Skin is a new tool that takes Python code and compiles it to C++. And not just any C++, but good, readable, well-formatted C++. I’d go so far as to say that the automatically generated code is as lucid as most hand-written C++ code I’ve seen. And Shed Skin itself is very cleanly implemented (consisting of only about 6000 lines of code). My compliments to creator Mark Dufour.

Shed Skin does require that you use a restricted set of Python features, since the compiled code uses static typing for speed. However, in practice, this should not be too much of a limitation. (Personally, I’ve found that when I start trying to get really clever by using Python’s dynamic typing and introspection, my code becomes hopelessly complex.)

What about speed? Mark writes, “For a set of 16 non-trivial test programs, measurements show a typical speedup of 2-40 over Psyco, about 12 on average, and 2-220 over CPython, about 45 on average.”

Here’s a simple benchmark I ran to compare Python-ShedSkin to Python, C, C++, and Fortran. The test is just a loop on which the compiler can’t take any shortcuts.

Source Code

Python code (used by both Python-ShedSkin and regular Python)

from math import cos
x = 0.5
for i in range(100):
    for j in range(1000000):
        x = cos(x**2)
print “x = %f\n” % x

C code (used by both C (gcc) and C++ (g++))

#include <stdio.h>
#include <math.h>
int main() {
    int i,j;
    float x=0.5;
    for (i=0;i<100;i++) {
        for (j=0;j<1000000;j++) {
            x = cos(pow(x,2));
        }
    }
    printf("x = %f\n", x);
}

Fortran code (compiled with gfortran/gfc)

program main
real:: x = 0.5
do i=1,100
    do j=1,1000000
        x = cos(x**2)
    end do
end do
write(*,*) ‘x = ‘, x
end program main

Here’s the time trials

Python-ShedSkin

$ ss speed && make all && time ./speed
x = 0.801071

real    0m9.039s
user    0m9.030s
sys     0m0.000s

Python

$ time python speed.py
x = 0.801071

real    3m19.260s
user    3m17.270s
sys     0m0.660s

C (gcc)

$ gcc speed.c -O3 -lm && time ./a.out
x = 0.801071

real    0m9.170s
user    0m9.140s
sys     0m0.000s

C++ (g++)

$ g++ speed.c -O3 -lm && time ./a.out
x = 0.801071

real    0m9.110s
user    0m9.110s
sys     0m0.000s

Fortran 90 (gfc)

$ gfc speed.f90 -O3 && time ./a.out
 x =   0.8010705

real    0m8.787s
user    0m8.790s
sys     0m0.000s

So, in summary…

Python-ShedSkin         9.039s
Python              3m 19.260s
C                       9.170s
C++                     9.110s
Fortran 90              8.787s

In this test, Shed Skin produces an executable that is actually a shade faster than C/C++ — and only a hair slower than Fortran 90. And the reader will note that the improvement over Python was a speedup by a factor of 22. (Insert smiley face here.)

The classic Emacs script psychoanalyze-pinhead, generates a conversation between the famous ELIZA psychoanalyst AI algorithm and Zippy from the cartoon, ‘Zippy the Pinhead’.

Check out this page which generates a new psychotherapy session each time…

I don’t get it. There’s these great high-level languages like Python and Ruby, with succinct, clear syntax, and everybody loves them. They’re so easy to prototype in, because you can rapidly express your algorithms. They scale well because you can easily use the most natural data structure for your needs. They are maintainable because you can go back and immediately understand code you wrote 6 months ago.

But… they’re only interpretable — you can’t compile them. So what? So, they’re slooow (well, at least, compared to compiled languages.) So here’s what I don’t get: Why can’t we have the best of both worlds? What I mean is: Why can’t there be a language that is (A) clear and succinct, and (B) either interpretable OR compilable.

C, C++, Java, BASIC, and Fortran are the leading compiled languages in use (I don’t have any stats behind this — just what I’ve observed.) When compared to a language like Python, these are all verbose and, at times, cryptic.

“Wait!”, you say. (Go ahead, say it…) What about OCaml, Pyrex, Jython, and Lua? Yes! This is on the right track. OCaml is either interpretable or compilable, so it meets criteria B. But in my (humble?) opinion, it’s syntax is pretty cryptic. (Maybe I’m just not used to functional programming.)

Pyrex is interesting. It’s a subset of Python that can be automatically compiled to C. Perfect! Well, sort of. From what I’ve seen, the code that Pyrex generates is rather inefficient — it’s doing a lot to try to interface with Python, rather than focusing on generating high-performance code.

Jython is also interesting. Again, a subset of Python, and it can be compiled to Java. Cool! So it’s very portable. Unfortunately, Jython can’t handle most Python packages - most notably the Numpy packages (Numeric, NumArray). So, high-performance numerical computing is out.

Lua seems promising at first. Python-like syntax… nice. Interpreter… nice! Automatically converted to C-code for compilation… sweet. So, it should be fast, right? No. All arrays are associative arrays - that is, array values aren’t stored sequentially in memory, making them very inefficient for numerical computation.

With some work (or maybe just a reduction of my own ignorance?), any these might be made into something workable.

Or, I could start from scratch. I’m toying with the idea of implementing a new language to meet my criteria - and learning about lex and yacc for doing this.

Correction: Lua is not converted to C, it is directly compiled. It is written in, and can be extended with C.

OK, so I knew about Google Video before, and have seen a couple random videos there from links people sent me in emails.

But I just discovered that there’s a host of videos from academic conferences. That’s right. Actual mental stimulation, as opposed to mind-numbing “funny” crap.

Right now, while writing this, I’m watching a presentation made at Google’s EngEDU conference called Hacking the brain by predicting the future and inverting the un-invertible. Awesome!

Here’s some fascinating videos of Jeff Hawkins talking about his wonderful theory of how the brain works.

While watching a video of Jeff Hawkin’s presenting his Hierarchical Temporal Memory approach, I started thinking about the various aspects of machine learning problems - e.g. are you handling discrete or continuous data? Are you providing domain knowledge or not? When defining the task an algorithm is trying to solve, you inevitably make simplifying assumptions (whether you are aware of them or not). This has a huge impact on the applicability of the solution you eventually come up. In other words, if you try to solve the wrong problem, you’ll certainly have the wrong solution.

This reminded me of a list of machine learning “scope variables” I had started keeping a while back. I extended it with things I thought of as I watched the video (which is quite interesting, by the way.)

For each variable, I describe the choices you might make in defining your task, in order of increasing difficulty. I believe that the scope of the problem which the brain faces is composed of the more difficult of many of these cases. Here’s the list.

Scope variables of unsupervised learning problems

Variable type
- Finite set (e.g. boolean)
- Discrete values (infinite set)
- Continuous (infinite set)

Dimensionality of input patterns
- 1 dimension (scalar)
- 2 dimensions (e.g. planar coordinates)
- …
- N dimensions (e.g. image with N pixels)

Use of domain-knowledge
- Prototype patterns are provided
- …
- No domain knowledge is provided

Number of input patterns
- 2 patterns (binary classification)
- 2 < k <= 10
- 10 < k <= 100
- 100 < k <= 1000000
- 1000000 < k

Known number of input patterns
- Known number of patterns, k
- Unknown number of patterns, k

Probability distribution function of pattern occurrence
- Uniform
- Non-uniform
(e.g. one pattern may be presented only once, while another is frequent)

Presentation period of each pattern
- N time-steps
- …
- 1 time-step

Shape of probability distribution function of each pattern
- Spherical (equal variance in all dimensions)
- Aligned (covariance between any two dimensions is zero)
- General (covariance may be non-zero)

Probability distribution function of each pattern
- Uniform
- Gaussian
- (Other?)

Temporal patterns
- No temporal component (each pattern can be recognized in a 1 time-step)
- Temporal component (may require synthesis over multiple time-steps)

Sequential temporal patterns
- Sequential (parts of a temporal pattern occur at successive time steps)
- Non-sequential (parts may be temporally separated)

Regularity of temporal patterns
- Regular (patterns consist of steps which always occur in the same order)
- Irregular (the parts of a pattern may be presented in varying order)

Co-presence of patterns
- One-at-a-time
- Two-at-a-time
- …
- N-at-a-time
(e.g. in analyzing one scene, you see a dog, a person, a fire hydrant,…)

Efficiency of implementation
What is the complexity of the solution, in big-O notation, for:
- O(f(n)) (where n is the dimensionality of the input patterns)
- O(f(m)) (where m is the number of input patterns)

Statistical significance of patterns
- Highly distinct (each pattern is very different from any other)
- Very similar (patterns may have largely overlapping distributions)
- (Superimposed? - not relevant - may be regarded as temporal patterns)

Drifting patterns
- Stationary (true statistics of each pattern are not changing over time)
- Drifting (patterns may change/evolve over time)
- (Discontinuous? - Is it possible to maintain a constant representation
for a pattern that undergoes a sharp discontinuity in change?)

(Observability?)
- Full observability of state (Markov)
- Partial observability (non-Markov)

Variable dimensionality
- Fixed dimensionality (constant number of dimensions in the input pattern)
- Variable dimensionality (input pattern may increase or decrease # of dims)

Online / offline
- Data set is known in entirety
- New data arrives periodically (must act before all data is experienced)

(Size of data set?)

This encompasses only the perception task. While certainly not the only the brain is doing, I think it’s the most difficult. If we can first identify the problem that the brain is faced with, we’ll be much further toward designing an algorithm to model it.