### Archive

Archive for August 10th, 2010

## On Lanchester's Differential Equations and their Transform into a Markov Transition Matrix (or, the Markovization of the Lanchester Equations)

I have been very curious as to whether the Markovization of differential equations is applicable to all systems, linear or not.  In the case of the SIR differential equations, Markovization was possible even when the system is both non-linear and coupled.  I think that perhaps there is an algorithm to frame the differential equations thusly... I haven't fleshed it out completely (other than by using quite a bit of intuition, as in the SIR case).  Markovization, I think, is distinct from putting differential equations in state-space form; the vector pulled through the state-space matrix represents the trajectories of a function and its derivatives, not the trajectories of a set of functions related in that their sum is one: probabilities or proportions.  Who knows, though, a happy mix may be possible.

I was examining a Calculus book, specifically by Deborah Hughes-Hallet, Andrew M. Gleason, et al. (New York, John Wiley & Sons, Inc., 1994), and came across this very interesting problem that models how armies or soldiers fight it out... and tries to predict who wins.  I hope you don't judge: even after having read the most complicated books in mathematics, I honestly believe that revisiting more basic books can bring about insights that one didn't think of before.  Maybe even ground the complicated knowledge one acquired, or refine it.  Such has been my own experience... so I think it's a useful exercise, for all of those who have resisted "going back to the basics." If you are one, perhaps you are missing the opportunity to bridge the gaps you may (or not) still have!  :-\

The problem on page 550 says that, if $x$ represents the number of soldiers from army $X$ and $y$ represents the number of soldiers from army $Y$, the rate at which soldiers in one army are injured or die can be thought of as proportional to the number of "firepower" the other army can concentrate on them.  The model was first thought by F.W. Lanchester:

$\frac{dx}{dt} = -ay$

$\frac{dy}{dt} = -bx$ with $a, b > 0$.

In order to Markovize Lanchester's differential equations, we need to flesh out what the states are: clearly, you can be alive and belong to army $X$, or army $Y$, or be dead (from a different vantage point, we could have "Dead from $X$" and "Dead from $Y$" if we wanted to count each individually).  The Markov transition matrix should look something like this, then:

$L = \left[ \begin{array}{ccc} something & 0 & something \\ 0 & something & something \\ 0 & 0 & 1 \end{array} \right]$

where $L_{1,1}$ represents the probability of being alive in army $X$ (if you are from army $X$) after a time step (or the proportion of alive soldiers in $X$ after a time step), $L_{1,2}$ represents the probability of "defecting" to army $Y$ if you are in army $X$ (I assume such doesn't happen), $L_{1,3}$ represents the probability of dying if you are in army $X$.  Then $L_{2,1}$ represents the probability of defecting to army $X$ if you are in army $Y$ (I also assume this doesn't happen), $L_{2,2}$ represents the probability of being alive in army $Y$, if you are from army $Y$, and $L_{2,3}$ is the probability of dying if you are in army $Y$.  The third row of $L$ simply represents the fact that one cannot come back from the dead once dead:  there are no zombies (such may be in fact possible in videogames, for example, hence why I make a point of saying this).

The next order of business is to note that the total population does not change (alives and deads).  Thus we would have that:

$p_{alive,x} = x/N$

$p_{alive,y} = y/N$, and

$p_{alive,x} + p_{alive,y} + p_{dead} = 1$.  Rewriting Lanchester's equations in terms of proportions, we would have that:

$\frac{d p_{alive,x}}{dt} = -a p_{alive,y}$

$\frac{d p_{alive,y}}{dt} = -b p_{alive,x}$

To be complete, we must model what the rate of change of the dead is, which we can figure out by differentiating in time the equation $p_{alive,x} + p_{alive,y} + p_{dead} = 1$.  In effect, what I'm saying is that, Lanchester's equations are incomplete - they are missing the key information that:

$\frac{d p_{dead}}{dt} = a p_{alive,y} + b p_{alive,x}$

This equation basically states that the dead increases as the rate at which soldiers in army $X$ are put down and the rate at which soldiers in army $Y$ are put down.

Euler's method suggests that to write the approximation to these differential equations, we need think of the derivative not at the limit, but taking tiny time steps.  So in the following equation

$lim_{\Delta t \rightarrow \infty} \frac{\Delta p_{alive,x}}{\Delta t} = -a p_{alive,y}$

let's forget about the limit for a second, and solve thus:

$\Delta p_{alive,x} \approx -a p_{alive,y} \Delta t$

and

$p_{alive,x} (t+\Delta t) \approx p_{alive,x} (t) - a p_{alive,y} \Delta t$.  Writing all differential equations thus:

$p_{alive,y} (t+\Delta t) \approx p_{alive,y} (t) -b p_{alive,x} \Delta t$ and

$p_{dead} (t+\Delta t) \approx p_{dead} (t) + a p_{alive,y} \Delta t + b p_{alive,x} \Delta t$

Euler's method gives us just what we need for our transition matrix construct.  If the approximations are in fact equalities, from the point-of-view of Markov chain theory, the left-hand side is the next-step vector, and the right-hand side explains how this vector would be obtained by a transformation on the at-step vector.  In terms of Markov dynamics:

$\textbf{p(t +} \Delta t \textbf{)} = \textbf{p(t)} \cdot L$

Playing a bit with the matrix $L$, it should look like:

$L = \left[ \begin{array}{ccc} 1 - a \cdot \frac{p_{alive,y}}{p_{alive,x}} \Delta t & 0 & a \cdot \frac{p_{alive,y}}{p_{alive,x}} \Delta t \\ 0 & 1 - b \cdot \frac{p_{alive,x}}{p_{alive,y}} \Delta t & b \cdot \frac{p_{alive,x}}{p_{alive,y}} \Delta t \\ 0 & 0 & 1 \end{array} \right]$

Letting $\Delta t$ be unity,

$\textbf{p(t +1)} = \textbf{p(t)} \cdot L$

with

$L = \left[ \begin{array}{ccc} 1 - a \cdot \frac{p_{alive,y}}{p_{alive,x}} & 0 & a \cdot \frac{p_{alive,y}}{p_{alive,x}} \\ 0 & 1 - b \cdot \frac{p_{alive,x}}{p_{alive,y}} & b \cdot \frac{p_{alive,x}}{p_{alive,y}} \\ 0 & 0 & 1 \end{array} \right]$

and, I think, we have successfully Markovized the differential equations, in effect discretizing them at unit time steps.  We should be cautious though, because now both $a, b$ have additional restrictions in order that the entries of $L$ remain below or equal to 1 (and specifically, the sum of each row must be equal to one).

Euler's method in effect is the link here between the continuous differential equations and the Markov transition matrix.  In the SIR discretization of my previous post, I went backwards, showing how we can write continuous differential equations from the Markov transition matrix.

Several interesting things to note about the matrix $L$: the probability of death, if one is in army $X$ is proportional to the attrition $a$ and by how many soldiers army $Y$ exceeds $X$ at the time.  A similar argument works for the death probability of soldiers in $Y$, with attrition $b$ and excedent of $X$ to $Y$.

As with the SIR equations, since $L$ is really changing every time step, it is $L(t)$.  So, from initial conditions, the amount of soldiers alive in army $X$, army $Y$, and dead at time step $n$ can be calculated using the discretized matrix by pulling the original vector through, as:

$\textbf{p(t +} n \Delta t \textbf{)} = \textbf{p(t)} \cdot L(t) \cdot L(t + \Delta t) \cdot \ldots \cdot L(t + (n-1) \Delta t)$

In my next post, I want to model, as in the Calculus book I mentioned, the Iwo Jima encounter during World War II, the fight between Japan and the US, and solving the trajectory numerically using the discretization above.

An additional note, from examining our assumptions of matrix $L$, we realize now simple ways in which we could extend Lanchester's differential equations, by filling in "probabilities that a soldier will defect to the other army."  We have indeed now enriched the model in the sense that we can now take into account other complicated dynamics (the least of which is the "creation of zombies").  I think this is very useful in any model, because it allows us to contemplate transitions into states we wouldn't have considered otherwise, and, more powerfully, allows us to transform our mindset to the realm of differential equations, where we can have DE-related insights, and back again, where we can have Markovian insights.