AI, or Postings from a Train, part 1

[davv]

In the domain of artificial intelligence, I've been thinking about what one may call a "General Optimizer" (or perhaps Great Optimizer, though there can be no one single Great - in the Platonic/Vasai sense - Optimizer). The General Optimizer is a system that you provide with a fitness function, and then it acts to maximize the value of that fitness function, given the manipulators and actions available to it. What makes the optimizer General is that the fitness function can be anything you can evaluate. It might be the proportion of red pixels in a picture the Optimizer gets from a camera. It might be the fraction of satisfied clauses in a SAT instance; or it might be the average return on some stock market data according to a strategy it devises.

The interesting thing about the General Optimizer is that now we have an idea of how it would work. The algorithm is completely impractical, but at least we have an algorithm and a general idea of what modules are necessary.

The impractical algorithm consists of two components. First, there is an explanation aspect, which tries to produce a predictor of reality (the domain where changes may be affected) and the fitness function. Second, there is a planner aspect, which determines the action to perform so that the best prediction so far, when combined with the action, predicts as good an improvement in the fitness function as is possible.

The predictor then uses a clever trick. It assumes that reality is computable, and that simpler models are more likely to be true. In other words, all laws of physics (or mathematics) can be run through an ordinary computer, and simple laws are more likely to be true than are complex laws. (There's a related proof that every kind of predictor has to make *some* kind of assumption, and so the reasoning goes that this assumption of computability seems to be a good one in practice.)

So the impractical predictor takes a sum of every computer program that produces what it has seen so far, and weights the programs by their length in base 1. This is like taking a vote, but one where short programs (simple laws) have a much stronger vote than long programs. Yes, this is uncomputable since it can be used to find a set of data's ultimate compression (Kolmogorov complexity) which is known to be impossible to do with an ordinary computer -- but the point here is to find the ideal.

The impractical planner is much simpler. Since the predictor now have a correctly weighted "committee" (as one may say, of all programs), it can extend the prediction however far into the future it desires. This may take a very long time, but again. Ideal! Thus it goes through every possible action it can make, and checks what the predictor says will happen if it performs that action. One of these actions will eventually give the best return on the fitness function, and it then picks that one.

If reality diverges from the prediction, that's no problem. As long as reality is computable, that only provides additional data for the next step of the impractical predictor.

Comments

[davv]

In the method I was talking about, you derive the ODE itself. For instance, say you want to find y = y'. Then you can take the initial value (say y(0) = 1) and just insert into the Taylor sequence. Right off, you know that y'(0) = y(0) = 1, y''(0) = 1 etc., and you get the Taylor expansion for e directly. Remember how I stumbled across sinh by imagining a function that had y(0) == 0, y'(0) == 1, y''(0) == 0 and so on? Kinda like that.

For more complex ODEs, you'll have to actually differentiate to get the second, third, etc. derivative.

This particular approach isn't all that common, though; you're much more likely to encounter something like RK4.